Ng}rddlmZddlmZmZmZmZmZddlm Z m Z ddl m Z ddl mZGdde ZdS) )partial)AnyDictListOptionalSequence)AsyncCallbackManagerForLLMRunCallbackManagerForLLMRun)LLM)pre_initc eZdZUdZeed<eed< dZeeed< dZ eeed< dZ ee eefed< dZ eeed< e d e eeffd Ze d efd Zed e d e fd Z ddedeeedeeded ef dZ ddedeeedeeded ef dZdS) CTransformersa_C Transformers LLM models. To use, you should have the ``ctransformers`` python package installed. See https://github.com/marella/ctransformers Example: .. code-block:: python from langchain_community.llms import CTransformers llm = CTransformers(model="/path/to/ggml-gpt-2.bin", model_type="gpt2") clientmodelN model_type model_fileconfiglibreturnc8|j|j|j|jdS)zGet the identifying parameters.rrrrrselfs b/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/llms/ctransformers.py_identifying_paramsz!CTransformers._identifying_params-s'Z//k    cdS)zReturn type of llm. ctransformersrs r _llm_typezCTransformers._llm_type7s rvaluesc ddlm}n#t$rtdwxYw|dpi}|j|df|d|d|dd ||d <|S) z5Validate that ``ctransformers`` package is installed.r)AutoModelForCausalLMz\Could not import `ctransformers` package. Please install it with `pip install ctransformers`rrrrr)rrrr)rr# ImportErrorfrom_pretrained)clsr!r#rs rvalidate_environmentz"CTransformers.validate_environment<s  : : : : : : :   E   !'R?/? 7O l+l+u      x s #promptstop run_managerkwargsc g}|ptj}|||dD]3}|||||j4d|S)aVGenerate text from a prompt. Args: prompt: The prompt to generate text from. stop: A list of sequences to stop generation when encountered. Returns: The generated text. Example: .. code-block:: python response = llm.invoke("Tell me a joke.") Tr)streamverbose)r get_noop_managerrappendon_llm_new_tokenr0join)rr(r)r*r+text _run_managerchunks r_callzCTransformers._callQs*"Q&>&O&Q&Q [[d4[@@ G GE KK     ) )% ) F F F Fwwt}}rcKd}|rt|j|j}d}|||dD]}|r||d{V||z }|S)aAsynchronous Call out to CTransformers generate method. Very helpful when streaming (like with websockets!) Args: prompt: The prompt to pass into the model. stop: A list of strings to stop generation when encountered. Returns: The string generated by the model. Example: .. code-block:: python response = llm.invoke("Once upon a time, ") Nr/r1Tr-)rr4r0r)rr(r)r*r+ text_callbackr6tokens r_acallzCTransformers._acallms*  X#K$@$,WWWM[[d4[@@  E +#mE********* EMDD r)NN)__name__ __module__ __qualname____doc__r__annotations__strrrrrrrpropertyrr r r'rr r9rr r=rrrrr s   KKK JJJ!%J $$$ $J $$$:'+FHT#s(^ $+++;C#H  T#s(^   X 3X$4X.)-:> x}%67    >%)?C tCy!;<    rrN) functoolsrtypingrrrrrlangchain_core.callbacksr r #langchain_core.language_models.llmsr langchain_core.utilsr rrrrrJs66666666666666433333))))))@@@@@C@@@@@r