Ng<'ddlZddlmZmZmZmZddlmZddlm Z ddl m Z dZ ej eZGdde ZdS) N)AnyListMappingOptional)CallbackManagerForLLMRun)LLM) ConfigDictgpt2cpeZdZUdZeZeed< dZe e ed< dZ e ed< dZ e ed< dZeed< ed Ze d!dddd dede e d e ed ede ede defdZe d!dddede e d e ede def dZe d"ded e ed ede edede e de e de fdZede dedede de de f dZedeee ffdZedefdZ d#dede eede ede def d ZdS)$IpexLLMzIpexLLM model. Example: .. code-block:: python from langchain_community.llms import IpexLLM llm = IpexLLM.from_model_id(model_id="THUDM/chatglm-6b") model_idN model_kwargsmodel tokenizerT streamingforbid)extra) tokenizer_id load_in_4bitload_in_low_bitrrrkwargsreturnc :|||d||||S)a Construct object from model_id Args: model_id: Path for the huggingface repo id to be downloaded or the huggingface checkpoint folder. tokenizer_id: Path for the huggingface repo id to be downloaded or the huggingface checkpoint folder which contains the tokenizer. load_in_4bit: "Whether to load model in 4bit. Unused if `load_in_low_bit` is not None. load_in_low_bit: Which low bit precisions to use when loading model. Example values: 'sym_int4', 'asym_int4', 'fp4', 'nf4', 'fp8', etc. Overrides `load_in_4bit` if specified. model_kwargs: Keyword arguments to pass to the model and tokenizer. kwargs: Extra arguments to pass to the model and tokenizer. Returns: An object of IpexLLM. Fr r low_bit_modelrrrr _load_model)clsr rrrrrs ]/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/llms/ipex_llm.py from_model_idzIpexLLM.from_model_id's5@%%+%   )rc :|||ddd||S)a Construct low_bit object from model_id Args: model_id: Path for the ipex-llm transformers low-bit model folder. tokenizer_id: Path for the huggingface repo id or local model folder which contains the tokenizer. model_kwargs: Keyword arguments to pass to the model and tokenizer. kwargs: Extra arguments to pass to the model and tokenizer. Returns: An object of IpexLLM. TFNrr)rr rrrs rfrom_model_id_low_bitzIpexLLM.from_model_id_low_bitQs40% %   r!Frc ddlm}m} ddlm} m} n#t $rt dwxYw|pi} |pi}|p|} d| vrd| d<| ddvrtd| dd | d} | j | fi| }n#t$r| j | fi| }YnwxYwd | vrd | D} d d d }|s|d}||d<n d}||d<nd} | | |||| }n*#t$r| ||||| }YnwxYw| ||d|||| d|S)Nr) AutoModelAutoModelForCausalLM) AutoTokenizerLlamaTokenizerzCould not import ipex-llm. Please install `ipex-llm` properly following installation guides: https://github.com/intel-analytics/ipex-llm?tab=readme-ov-file#install-ipex-llm.devicecpu)r*xpuzXIpexLLMBgeEmbeddings currently only supports device to be 'cpu' or 'xpu', but you have: .trust_remote_codec&i|]\}}|dk ||S)r-).0kvs r z'IpexLLM._load_model..s0A!?R:R:R1:R:R:Rr!T) use_cacher-from_pretrainedrr load_low_bit)load_function_namer load_kwargsr)r rrrr/)ipex_llm.transformersr%r& transformersr'r( ImportError ValueErrorpopr5 Exceptionitems_load_model_generalto)rr rrrrrrr%r&r'r( _model_kwargs _tokenizer_idr)rr8r7rs rrzIpexLLM._load_modelss          C B B B B B B B B   c  %* 2$0 = ( (&+M( #  ". 8 8L1>x1HLLL ""8,, W5 5mUU}UUII W W W66}VV VVIII W - / /!.!4!4!6!6M !%   0*%6"1@ -..%6".: N++!/  ++$#5!'* ,EE   ++#5!'* ,EEE  s &       s,->B B('B()D$D+*D+ model_classr7r8c  t||}||fii||S#t$r3}td|jd|d|Yd}~dSd}~wwxYw)z,General function to attempt to load a model.zFailed to load model using r,z: N)getattrr>loggererror__name__)rDr7r r8r load_functiones rr@zIpexLLM._load_model_generals #K1CDDM =MM-L -L|-LMM M    LLD'DD*<DD@ADD          s A(AAc |j|jdS)zGet the identifying parameters.r rrMselfs r_identifying_paramszIpexLLM._identifying_paramss  -   r!cdS)Nzipex-llmr/rNs r _llm_typezIpexLLM._llm_typeszr!promptstop run_managerc |jrddlm}|j|d}||jj}||jdd}|(ddlm }ddl m } || ||jg} nd} |jj |f|| d |} |j | dd } | S|j|d}||jj}|(ddlm }ddl m } || ||jg} nd} |jj |fd | i|} |j | dd t|d} | S) Nr) TextStreamerpt)return_tensorsT) skip_promptskip_special_tokens)StoppingCriteriaList)StopSequenceCriteria)streamerstopping_criteria)r[r_)rr:rWrencoderArr))transformers.generation.stopping_criteriar\transformers.tools.agentsr]generatedecodelen) rOrSrTrUrrW input_idsr^r\r]r_outputtexts r_callz IpexLLM._calls: >1  1 1 1 1 1 1--fT-JJI! TZ%677I#|DdHKJJJJJ%9$8))$??@%%!!%)!(TZ(!"3 F >(((MMDK--fT-JJI! TZ%677IKJJJJJ$8$8))$??@%%!!%)!(TZ(->BHF>(((MMF DKr!)N)NFNFNN)NN) rI __module__ __qualname____doc__DEFAULT_MODEL_IDr str__annotations__rrdictrrrrboolr model_config classmethodrr r#r staticmethodr@propertyrrPrRrrrir/r!rr r sT%Hc$$$*#'L(4.'''0E3Is&It8:L(,' '+!)-' ' ' ' tn' sm '  ' "#' '  ' ' ' [' R(, '+     tn sm        [ B'+")-#'+!%] ] ] sm]  ] "# ]  ] tn] ]  ] ] ] [] ~    \" WS#X%6   X 3X %):> 888tCy!867 8  8 888888r!r )loggingtypingrrrrlangchain_core.callbacksr#langchain_core.language_models.llmsrpydanticr rm getLoggerrIrGr r/r!rr|s////////////======333333  8 $ $[[[[[c[[[[[r!