Ng! 0ddlZddlmZmZmZmZmZmZmZddl Z ddl m Z m Z ddl mZddlmZddlmZmZddlmZddlmZd ZGd d eZd eed eefdZd e jd eefdZded eefdZded eefdZ dS)N)Any AsyncIteratorDictIteratorListMappingOptional)AsyncCallbackManagerForLLMRunCallbackManagerForLLMRun)LLM)GenerationChunk)get_from_dict_or_envpre_init) ConfigDict)Requestsz$meta-llama/Meta-Llama-3-70B-Instructc VeZdZUdZeZeed<dZe e ed<dZ e eed<e dZ ede d e fd Zed eeeffd Zed efd Zd efd Zd e fdZdeded e fdZdeded dfdZ ddede eede eded ef dZ ddede eede eded ef dZ ddede eede eded ee f dZ! ddede eede eded e"e f dZ#ded dfdZ$dS) DeepInfraaDeepInfra models. To use, you should have the environment variable ``DEEPINFRA_API_TOKEN`` set with your API token, or pass it as a named parameter to the constructor. Only supports `text-generation` and `text2text-generation` for now. Example: .. code-block:: python from langchain_community.llms import DeepInfra di = DeepInfra(model_id="google/flan-t5-xl", deepinfra_api_token="my-api-key") model_idN model_kwargsdeepinfra_api_tokenforbid)extravaluesreturnc2t|dd}||d<|S)z?Validate that api key and python package exists in environment.rDEEPINFRA_API_TOKEN)r)clsrrs ^/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/llms/deepinfra.pyvalidate_environmentzDeepInfra.validate_environment-s/3 )+@  )<$% c*id|jid|jiS)zGet the identifying parameters.rr)rrselfs r_identifying_paramszDeepInfra._identifying_params6s+ 4=) t01  r cdS)zReturn type of llm. deepinfrar"s r _llm_typezDeepInfra._llm_type>s {r cd|jS)Nz'https://api.deepinfra.com/v1/inference/)rr"s r_urlzDeepInfra._urlCsHHHHr cd|jddS)Nzbearer zapplication/json) Authorizationz Content-Type)rr"s r_headerszDeepInfra._headersFs"At'?AA.   r promptkwargsc,|jpi}i||}d|i|S)Ninput)r)r#r.r/rs r_bodyzDeepInfra._bodyLs7(.B 1,1&1  V   r codetextcT|dkrtd||dkrtd|dkrtd|dkrtd|j|dkrtd |d krtd ||d krtd |d|dS)NizDeepInfra Server: Error izDeepInfra Server: Unauthorizediiz"DeepInfra Server: Model not found iz%DeepInfra Server: Rate limit exceedediz'DeepInfra received an invalid payload: z6DeepInfra returned an unexpected response with status z: ) Exceptionr ValueError)r#r3r4s r_handle_statuszDeepInfra._handle_statusUs 3;;=t==>> > S[[<== = S[[<== = S[[PPPQQ Q S[[CDD D S[[MtMMNN N S[[""""" [r stop run_managerc Pt|}|||||}||j|j|}|dddS)alCall out to DeepInfra's inference API endpoint. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. Returns: The string generated by the model. Example: .. code-block:: python response = di("Tell me a joke.") headersurldataresultsrgenerated_text) rr-postr*r2r9 status_coder4jsonr#r.r:r;r/requestresponserAs r_callzDeepInfra._callhs,4==??333< 444tCy!467 4  4 4444B%)?C 8 8 8tCy! 8;< 8  8 8 8 8 8$%):> tCy!67    / " 0%)?C    tCy! ;<     '     *========r rrbodyrc#>K|D]}t|}||VdSN_parse_stream_helperrtrY_lines rrSrSs<$T**  KKKr cLK|23d{V}t|}||WV6dSrvrwrys rr]r]sVd$T**  KKKKees#rYc|r|drs|dr|tdd}n|tdd}|dkrdS|dSdS)Nsdata:sdata: s[DONE]zutf-8)rdrestripdecode)rYs rrxrxs ()) ( ??9 % % )I(()DDH (D ::<<9 $ $4;;w'' ' 4r c tj|}t|didS#t$rYdSwxYw)Ntokenr4)r4)rFrfr rgr7)rYobjs rrUrUspj"%%))&11     ttsA A AA)!rFtypingrrrrrrr aiohttplangchain_core.callbacksr r #langchain_core.language_models.llmsr langchain_core.outputsr langchain_core.utilsrrpydanticr&langchain_community.utilities.requestsrrnrbytesrorS StreamReaderr]rxrUr'r rrs NNNNNNNNNNNNNNNNNN433333222222????????;;;;;;9|=|=|=|=|=|=|=|=~%Xc]W%9mC>P u #     38O#<r