Ng"ddlmZddlmZmZmZmZmZmZm Z ddlmZddl m Z ddlmZddl m Z mZddlmZddlmZddlmZddlmZddlmZddlmZGdd eZd S) )pre_init)Any AsyncIteratorDictIteratorListOptionalUnion)root_validator)AsyncCallbackManagerForLLMRunCallbackManagerForLLMRun)LLM)enforce_stop_tokens)GenerationChunkc eZdZUdZeed<eed< dZee eefed< dZ e dee fed< dZ e ed< ed e eeffd Zed efd Zed e d e fd Z ddedeeedeeded ef dZ ddedeeedeeded ef dZ ddedeeedeeded eef dZ ddedeeedeeded eef dZdS) DeepSparseaHNeural Magic DeepSparse LLM interface. To use, you should have the ``deepsparse`` or ``deepsparse-nightly`` python package installed. See https://github.com/neuralmagic/deepsparse This interface let's you deploy optimized LLMs straight from the [SparseZoo](https://sparsezoo.neuralmagic.com/?useCase=text_generation) Example: .. code-block:: python from langchain_community.llms import DeepSparse llm = DeepSparse(model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base_quant-none") pipelinemodelNmodel_configurationgeneration_configF streamingreturnc8|j|j|j|jdS)zGet the identifying parameters.)r model_configrr)rrrrselfs _/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/llms/deepsparse.py_identifying_paramszDeepSparse._identifying_params1s)Z 4!%!7    cdS)zReturn type of llm. deepsparsers r _llm_typezDeepSparse._llm_type;s |rvaluesc ddlm}n#t$rtdwxYw|dpi}|jd d|dd||d<|S) z2Validate that ``deepsparse`` package is installed.r)Pipelinez[Could not import `deepsparse` package. Please install it with `pip install deepsparse[llm]`rtext_generationr)task model_pathrr")r!r& ImportErrorcreate)clsr$r&rs rvalidate_environmentzDeepSparse.validate_environment@s  + + + + + + +   G   34: ,X_ "g    z  s #promptstop run_managerkwargsc |jr#d}|jd|||d|D] }||jz } |}n$|jdd|i|jjdj}|t ||}|S)Generate text from a prompt. Args: prompt: The prompt to generate text from. stop: A list of strings to stop generation when encountered. Returns: The generated text. Example: .. code-block:: python from langchain_community.llms import DeepSparse llm = DeepSparse(model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base_quant-none") llm.invoke("Tell me a joke.") r.r/r0 sequencesrNr")r_streamtextrr generationsrrr.r/r0r1combined_outputchunkr8s r_callzDeepSparse._callTs& >  O%DkEK . . 5:-"DD III$2HIIQ    &tT22D rcK|jr)d}|jd|||d|23d{V}||jz }6|}n$|jdd|i|jjdj}|t ||}|S)r3r4r5Nr6rr")r_astreamr8rrr9rr:s r_acallzDeepSparse._acallzs& >  O,t} Dk  EK   . . . . . . .e 5:- #DD III$2HIIQ    &tT22D s/c+K|jd|dd|j}|D]C}t|jdj}|r||j|VDdSaYields results objects as they are generated in real time. It also calls the callback manager's on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. Returns: A generator representing the stream of tokens being generated. Yields: A dictionary like object containing a string token. Example: .. code-block:: python from langchain_community.llms import DeepSparse llm = DeepSparse( model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base_quant-none", streaming=True ) for chunk in llm.stream("Tell me a joke", stop=["'"," "]): print(chunk, end='', flush=True) # noqa: T201 T)r6rr)r8)tokenNr"rrrr9r8on_llm_new_tokenrr.r/r0r1 inferencerCr<s rr7zDeepSparse._streams8"DM   040F    E#):1)=)BCCCE ?,,5:,>>>KKKK   rc K|jd|dd|j}|D]J}t|jdj}|r!||jd{V|WVKdSrBrDrFs rr?zDeepSparse._astreams8"DM   040F    E#):1)=)BCCCE E!222DDDDDDDDDKKKKK   r)NN)__name__ __module__ __qualname____doc__r__annotations__strrr rrr rboolpropertyrr#rr-rr r=r r@rrr7rr?r"rrrrs  MMM JJJV48$sCx.1888E15uT3_-444) It8  T#s(^   X 3X$4X,%):> $$$tCy!$67 $  $ $$$$R%)?C $$$tCy!$;< $  $ $$$$R%):> $$$tCy!$67 $  $ / " $$$$R%)?C $$$tCy!$;< $  $  ' $$$$$$rrN)langchain_core.utilsrtypingrrrrrr r pydanticr langchain_core.callbacksr r #langchain_core.language_models.llmsrlangchain_community.llms.utilsrlangchain_core.outputsrrr"rrrXsW))))))LLLLLLLLLLLLLLLLLL))))))######))))))*)))))333333))))))>>>>>>))))))222222WWWWWWWWWWr