Ngf0ddlmZddlZddlmZddlmZmZmZm Z m Z m Z ddl m Z ddlmZddlmZddlmZmZdd lmZdd lmZmZejeZGd d eZdS) ) annotationsN)Path)AnyDictIteratorListOptionalUnion)CallbackManagerForLLMRun)LLM)GenerationChunk)get_pydantic_field_namespre_init)_build_model_kwargs)Fieldmodel_validatorceZdZUdZdZded<ded< dZded< dZded < ed d Z d ed < edd Z d ed< edd Z d ed< edd Z ded< edd Z ded< edd Zded< edd Zded< edd Zded< edd Zded< edd Zded< edZded< dZded< d Zd!ed"< d#Zd!ed$< edZded%< dZd&ed'< gZd(ed)< d*Zd!ed+< d,Zded-< d.Zded/< dZd&ed0< d1Zd2ed3< d4Zd2ed5< ee 6Z!d7ed8< dZ"ded9< dZ#d:ed;< dZ$d< e&dSdBZ'e(dCDe)dTdEZ*e+dUdFZ,e+dUdGZ-e+dVdHZ.dWdXdIZ/ dYdZdNZ0 dYd[dPZ1d\dRZ2dS)]LlamaCppallama.cpp model. To use, you should have the llama-cpp-python library installed, and provide the path to the Llama model as a named parameter to the constructor. Check out: https://github.com/abetlen/llama-cpp-python Example: .. code-block:: python from langchain_community.llms import LlamaCpp llm = LlamaCpp(model_path="/path/to/llama/model") Nrclientstr model_pathz Optional[str] lora_base lora_pathin_ctx)aliasintn_partsseedTf16_kvboolF logits_all vocab_only use_mlock n_threadsz Optional[int]n_batch n_gpu_layerssuffix max_tokensg?zOptional[float] temperaturegffffff?top_plogprobszOptional[bool]echoOptional[List[str]]stopg?repeat_penalty(top_k@last_n_tokens_sizeuse_mmapg?floatrope_freq_scaleg@rope_freq_base)default_factoryDict[str, Any] model_kwargs streamingzOptional[Union[str, Path]] grammar_pathzOptional[Union[str, Any]]grammarverbosevaluesrreturnc ddlm}m}n#t$rtdwxYwd}gd}fd|D}d d|d<|d  ||fi|d <n'#t $r}t d |d |d}~wwxYwd r.dr&d }d} t d|d| dtd tr| d d <n(dr| dd <n S)z4Validate that llama-cpp-python library is installed.r)Llama LlamaGrammarzCould not import llama-cpp-python library. Please install the llama-cpp-python library to use this embedding model: pip install llama-cpp-pythonr)r9r:rrrrrr r"r#r$r%r'r7r6rAc"i|] }|| SrH).0krBs ]/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/llms/llamacpp.py z1LlamaCpp.validate_environment..s@@@6!9@@@r(Nr=rz&Could not load Llama model from path: z. Received error r@r?zCCan only pass in one of grammar and grammar_path. Received grammar=z and grammar_path=.) llama_cpprErF ImportErrorupdate Exception ValueError isinstancer from_string from_file) clsrBrErFrmodel_param_names model_paramser@r?s ` rKvalidate_environmentzLlamaCpp.validate_environments   5 5 5 5 5 5 5 5 5   I  L)    $A@@@.?@@@ . ! -+1.+AL (F>2333 $uZ@@<@@F8     &&&"#&&   )  !7 Y'G!.1L333".333 y)3 / /  , 8 8 9J K KF9   N #  , 6 6vn7M N NF9    s  &2 A?? B# BB#before)modecDt|}t||}|S)z>Build extra kwargs from additional params that were passed in.)rr)rWrBall_required_field_namess rKbuild_model_kwargszLlamaCpp.build_model_kwargss'$_streamrvrnr) rcror1rprrcombined_text_outputchunkrdresults rK_callzLlamaCpp._calls. > 0$& %'  3 3 % 2$$' '))$//F))&)F T[999&99F)$Q'/ /rMIterator[GenerationChunk]c+DKi|||}|jd |dd|}|D]q}|dddd}t|dddd|i} |r"|| j|j| | VrdS) a\Yields results objects as they are generated in real time. It also calls the callback manager's on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name. Args: prompt: The prompts to pass into the model. stop: Optional list of stop words to use when generating. Returns: A generator representing the stream of tokens being generated. Yields: A dictionary like objects containing a string token and metadata. See llama-cpp-python docs and below for more. Example: .. code-block:: python from langchain_community.llms import LlamaCpp llm = LlamaCpp( model_path="/path/to/local/model.bin", temperature = 0.5 ) for chunk in llm.stream("Ask 'Hi, how are you?' like a pirate:'", stop=["'"," "]): result = chunk["choices"][0] print(result["text"], end='', flush=True) # noqa: T201 T)rostreamrurr.Nrv)rvgeneration_info)tokenrA log_probsrH)rnrgetr on_llm_new_tokenrvrA) rcror1rprrrdrzpartr.rys rKrwzLlamaCpp._stream,sJ:D((..9&9BF4BB6BB  DIq)--j$??H#)_Q'/!+X 6E ,,*dlh-KKKK  rMrvcz|j|d}t|S)Nzutf-8)rtokenizeencodelen)rcrvtokenized_texts rKget_num_tokenszLlamaCpp.get_num_tokens_s1--dkk'.B.BCC>"""rM)rBrrCr)rBr<rCr)rCr<)rCr)N)r1r0rCr<)NN) rorr1r0rprqrrrrCr) rorr1r0rprqrrrrCr|)rvrrCr)3__name__ __module__ __qualname____doc__r__annotations__rrrrrrr r"r#r$r%r'r(r)r+r,r-r.r/r1r2r4r6r7r9r:dictr=r>r?r@rArr[r classmethodr`propertyrerhrkrnr{rwrrHrMrKrrs  FOOO+#I####0#I####As'***E****59---G----?b'''D''''-5X...F....1uU,777J7777@uU,777J7777/eE555I5555,$uT===I====C#U1I666G6666/#(%N"C"C"CLCCCCF!E$KKF''''S #J####3#&K&&&&.!E!!!!.#eDkkH))))N D    % "D""""@&)N))))2E.(*****M#H####1 O    )#N####+#(5#>#>#>L>>>>?I8/3L3333 *.G---- G) :::X:x_(###[$# X"KKKXKX8%):> (0(0(0(0(0Z%):> 11111f######rMr) __future__rloggingpathlibrtypingrrrrr r langchain_core.callbacksr #langchain_core.language_models.llmsr langchain_core.outputsr langchain_core.utilsrrlangchain_core.utils.utilsrpydanticrr getLoggerrloggerrrHrMrKrs*""""""======================333333222222CCCCCCCC::::::++++++++  8 $ $P#P#P#P#P#sP#P#P#P#P#rM