NgzddlZddlmZmZmZmZmZmZmZddl Z ddl m Z ddl m Z ddlmZddlmZmZddlmZmZddl mZGd d e ZGd d e ZdS) N)AnyDictIteratorListOptionalTupleUnion)CallbackManagerForLLMRun)LLM)GenerationChunk)convert_to_secret_strget_from_dict_or_env)Field SecretStr)Responsec eZdZUdZedZeed< eedZ eed< eddZ eed< eddZ eed < ed Z e ed < d Zeeeefed < GddZede fdZedeeeffdZedeeeffdZedefdZdedd ffd ZdedeeeffdZd#deeedeeeffdZ d$deeeefdeeed ee defdZ dedefdZ!dede"e#fdZ$ d%deeeefdeeed ee%dede"e#f d!Z& d%deeeefdeeed ee%dedef d"Z'xZ(S)& SambaStudiou{ SambaStudio large language models. Setup: To use, you should have the environment variables ``SAMBASTUDIO_URL`` set with your SambaStudio environment URL. ``SAMBASTUDIO_API_KEY`` set with your SambaStudio endpoint API key. https://sambanova.ai/products/enterprise-ai-platform-sambanova-suite read extra documentation in https://docs.sambanova.ai/sambastudio/latest/index.html Example: .. code-block:: python from langchain_community.llms.sambanova import SambaStudio SambaStudio( sambastudio_url="your-SambaStudio-environment-URL", sambastudio_api_key="your-SambaStudio-API-key, model_kwargs={ "model" : model or expert name (set for CoE endpoints), "max_tokens" : max number of tokens to generate, "temperature" : model temperature, "top_p" : model top p, "top_k" : model top k, "do_sample" : wether to do sample "process_prompt": wether to process prompt (set for CoE generic v1 and v2 endpoints) }, ) Key init args — completion params: model: str The name of the model to use, e.g., Meta-Llama-3-70B-Instruct-4096 (set for CoE endpoints). streaming: bool Whether to use streaming handler when using non streaming methods model_kwargs: dict Extra Key word arguments to pass to the model: max_tokens: int max tokens to generate temperature: float model temperature top_p: float model top p top_k: int model top k do_sample: bool wether to do sample process_prompt: wether to process prompt (set for CoE generic v1 and v2 endpoints) Key init args — client params: sambastudio_url: str SambaStudio endpoint Url sambastudio_api_key: str SambaStudio endpoint api key Instantiate: .. code-block:: python from langchain_community.llms import SambaStudio llm = SambaStudio=( sambastudio_url = set with your SambaStudio deployed endpoint URL, sambastudio_api_key = set with your SambaStudio deployed endpoint Key, model_kwargs = { "model" : model or expert name (set for CoE endpoints), "max_tokens" : max number of tokens to generate, "temperature" : model temperature, "top_p" : model top p, "top_k" : model top k, "do_sample" : wether to do sample "process_prompt" : wether to process prompt (set for CoE generic v1 and v2 endpoints) } ) Invoke: .. code-block:: python prompt = "tell me a joke" response = llm.invoke(prompt) Stream: .. code-block:: python for chunk in llm.stream(prompt): print(chunk, end="", flush=True) Async: .. code-block:: python response = llm.ainvoke(prompt) await response defaultsambastudio_urlsambastudio_api_keyT)rexcludebase_url streaming_urlF streamingN model_kwargsceZdZdZdS)SambaStudio.ConfigTN__name__ __module__ __qualname__populate_by_name^/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/llms/sambanova.pyConfigr{r&r(returncdS)9Return whether this model can be serialized by Langchain.Tr%clss r'is_lc_serializablezSambaStudio.is_lc_serializable~s tr&c dddS)Nrr)rrr%selfs r' lc_secretszSambaStudio.lc_secretss 1#8   r&c&d|jid|jiS)Return a dictionary of identifying parameters. This information is used by the LangChain callback system, which is used for tracing purposes make it possible to monitor LLMs. rr)rrr1s r'_identifying_paramszSambaStudio._identifying_paramssT^S@Q/RSSr&cdS)zReturn type of llm.zsambastudio-llmr%r1s r' _llm_typezSambaStudio._llm_types ! r&kwargsc t|dd|d<tt|dd|d<||d\|d<|d<tjdi|dS) 'init and validate environment variablesrSAMBASTUDIO_URLrSAMBASTUDIO_API_KEYrrNr%)rr _get_sambastudio_urlssuper__init__r2r9 __class__s r'r@zSambaStudio.__init__s$8 %'8% %  !)> )>@U V V) ) $%7;6P6P $ %7 7 3zF?3 ""6"""""r&urlcd|vr|}|}n[d|vr|dd}|}n>|}d|vr)d|d}ntd||fS)a3 Get streaming and non streaming URLs from the given URL Args: url: string with sambastudio base or streaming endpoint url Returns: base_url: string with url to do non streaming calls streaming_url: string with url to do streaming calls openaistreamzstream/rgenericzgeneric/streamUnsupported URL)replacejoinsplit ValueError)r2rCr stream_urls r'r>z!SambaStudio._get_sambastudio_urlss s??HJJ3;;y"55 ##!1!6!6syy7K7K!L!LJJ$%6777##r&stopc|g}|jpi}|dg|z}t|dkr||d<d|jvrd|vr|d|d<d|vr|d|d<d |vr|d |}nd |jvr_d|vr|d|d<d|vr|d|d<|}nd |jvr{d|vr|d|d<d|vr|d|d<d |D}ntd |jd|S)aD Get the tuning parameters to use when calling the LLM. Args: stop: Stop words to use when generating. Model output is cut off at the first occurrence of any of the stop substrings. Returns: The tuning parameters in the format required by api to use Nstop_sequencesrrE select_expertmodelmax_tokens_to_generate max_tokensprocess_promptapi/v2/predict/genericapi/predict/genericc^i|]*\}}|t|jt|d+S))typevalue)rYr!str).0kvs r' z2SambaStudio._get_tuning_params..sEAqDGG,s1vv>>r&rH9only openai, generic v1 and generic v2 APIs are supported)rgetlenrkeyspopitemsrL)r2rN _model_kwargs_stop_sequences tuning_paramss r'_get_tuning_paramszSambaStudio._get_tuning_paramssR <D)/R (++,1B1B71K1K o.}113333:G:K:K ;; 67*MM "d&: : :-,,....1>1B1B71K1K o.}113333:G:K:K ;; 67*0022MM L$"6LLL  r&promptct|tr|g}||}d|jvrRd|ddg}||d|}d|D}d|jdd }nCd |jvr|d d r#tj d dd|ddgd}n|d}d|dg}d|D}||d}d|ji}nd|jvr|d d rK|d ddkr#tj d dd|ddgd}n|d}n|d}|r||d}n|g|d}d|ji}ntd|jdtj } |r| |j||d} n| |j||d } | jdkr t#d | jd!| jd!| S)"a Performs a post request to the LLM API. Args: prompt: The prompt to pass into the model stop: list of stop tokens streaming: wether to do a streaming call Returns: A request Response object rEuserrrolecontent)messagesrFci|] \}}||| SNr%r\keyrZs r'r_z/SambaStudio._handle_request..s#SSS:3ARCARARARr&Bearer application/json Authorizationz Content-TyperVrUFzsambaverse-conversation-idN) message_idrnro)conversation_idrpitem0)idrZci|] \}}||| Srrr%rss r'r_z/SambaStudio._handle_request..1s#WWWZS%UEVc5EVEVEVr&)reparamsrtrWrZTrue)instancer~) instancesr~rHr`TheadersjsonrF2Sambanova / complete call failed with status code .) isinstancer[rirrerget_secret_valuerardumpsrLrequestsSessionpostrr status_code RuntimeErrortext) r2rjrNrr~ messages_dictdatarre http_sessionresponses r'_handle_requestzSambaStudio._handle_requests8$ fc " " XF((.. t+ + +&,CCDM -MMfMDSSSSSD"A+<<>>"A"A 2GG&)= = =zz*E22 #+G+/FSTIVV% #f556EWW6<<>>WWWF"f55Dd6GGIIJGG#d&: : :zz*E22 #*+//88FBB!Z/K37,2/5ay!"!")    FF$AYFF A$*f==&,X@@d6GGIIJGGL$"6LLL   '))  #(("G$t)HH$(( wT%)H  3 & &$'$$=$$$  r&rc |}n,#t$r}td|d|jd}~wwxYwd|jvr|dddd}nZd |jvr|d dd d }n6d |jvr|ddd }nt d|jd|S) Process a non streaming response from the api Args: response: A request Response object Returns completion: a string with model generation ;Sambanova /complete call failed couldn't get JSON response response: NrEchoicesrmessagerorVrerZ completionrW predictionsrHr`)r ExceptionrrrrLr2r response_dicters r'_process_responsezSambaStudio._process_responsegs $MMOOMM   -a--%]--   t+ + +&y1!4Y? JJJ %)= = =&w/27;LIJJ "d&: : :&}5a8FJJL$"6LLL  A;Ac#K ddl}n#t$rtdwxYwd|jvrf||}|D]9}|jdkr t d|jd|jd |jdkrt|jtrtj |j}n t d|jd|jd| d r t d|jd|jdt|d dkr|d dd d }nd }t|}|V#t $r}t d|d|jd}~wwxYwdSd|jvr|D]r} tj | }|ddddd}t|}|VL#t $r}t d|d| d}~wwxYwdSd|jvr|D]l} tj | }|dddd}t|}|VF#t $r}t d|d| d}~wwxYwdSt%d|jd) Process a streaming response from the api Args: response: An iterable request Response object Yields: GenerationChunk: a GenerationChunk with model partial generation rNTcould not import sseclient libraryPlease install it with `pip install sseclient-py`.rE error_event1Sambanova /complete call failed with status code r[DONE]errorrdeltarorr3Error getting content chunk raw streamed response: data: rVresultrerZ stream_tokenzline: rW responsesrHr`) sseclient ImportErrorr SSEClienteventseventrrrrr[rloadsrarbr r iter_linesrL) r2rrclientrrrogenerated_chunkrlines r'_process_stream_responsez$SambaStudio._process_stream_responses         E   t+ + +((22F# # ;-//&)#/)) :))) zX--%ej#66#':ej#9#9DD".!1#+#7!1!1#(:!1!1!1##  88G,,".!1#+#7!1!1#(:!1!1!1## tI//!33&*9oa&8&A)&LGG&(G*9w*G*G*G---- &.a..!&..?# # L&)= = = ++--   :d++D"8nW5a8A.QG&57&C&C&CO))))) &(a((!%((  #d&: : : ++--   :d++D"8n[9!<^LG&57&C&C&CO))))) &(a((!%((  L$"6LLL sS #CE(( F2F  F6AG?? H# HH#AJ  J/J**J/ run_managerc+K|||d}||D]"}|r||j|V#dS)a\Call out to Sambanova's complete endpoint. Args: prompt: The prompt to pass into the model. stop: a list of strings on which the model should stop generating. run_manager: A run manager with callbacks for the LLM. Yields: chunk: GenerationChunk with model partial generation TrNrron_llm_new_tokenrr2rjrNrr9rchunks r'_streamzSambaStudio._streamr '''EE228<<  E 9,,UZ888KKKK  r&c |jr"d}|jd|||d|D] }||jz } |S|||d}||}|S)a Call out to Sambanova's complete endpoint. Args: prompt: The prompt to pass into the model. stop: a list of strings on which the model should stop generating. Returns: result: string with model generation rrjrNrFrr%rrrrrr2rjrNrr9rrrs r'_callzSambaStudio._call > J%DkEK ) )ej(  '''FF++H55 r&rrNFNN))r!r"r#__doc__rrr[__annotations__rrrrrboolrrrrr( classmethodr/propertyr3r6r8r@rr>rrir rrrrr rr rr __classcell__rBs@r'rr sYYv!5,,,OS,,,%*U99R==%A%A%AAAAE"d333Hc333'r4888M3888#eE***It***K-1L(4S>*1112        4[ DcN   X TT#s(^TTTXT!3!!!X! # # # # # # # #$$sCx$$$$4??xS ':?d3PS8n????H%)$) ddd3in%dtCy!dD> d  ddddL"("s""""H]]h>W]]]]D%):> d3in%tCy!67    / " 2%):> d3in%tCy!67    r&rc eZdZUdZedZeed< eedZ eed< edZ eed< edZ e ed < ed Z eed < ed Zeed < edZeeed< edZeeed< eddiZeed< GddZede fdZedeeeffdZedeeeffdZedefdZdeddffd Z d&de e!eefdee!ed ee de"fdZ#d e"defd!Z$d e"de%e&fd"Z' d'de e!eefdee!ed#ee(dedef d$Z) d'de e!eefdee!ed#ee(dede%e&f d%Z*xZ+S)(SambaNovaCloudu SambaNova Cloud large language models. Setup: To use, you should have the environment variables: ``SAMBANOVA_URL`` set with SambaNova Cloud URL. defaults to http://cloud.sambanova.ai/ ``SAMBANOVA_API_KEY`` set with your SambaNova Cloud API Key. Example: .. code-block:: python from langchain_community.llms.sambanova import SambaNovaCloud SambaNovaCloud( sambanova_api_key="your-SambaNovaCloud-API-key, model = model name, max_tokens = max number of tokens to generate, temperature = model temperature, top_p = model top p, top_k = model top k ) Key init args — completion params: model: str The name of the model to use, e.g., Meta-Llama-3-70B-Instruct-4096 (set for CoE endpoints). streaming: bool Whether to use streaming handler when using non streaming methods max_tokens: int max tokens to generate temperature: float model temperature top_p: float model top p top_k: int model top k Key init args — client params: sambanova_url: str SambaNovaCloud Url defaults to http://cloud.sambanova.ai/ sambanova_api_key: str SambaNovaCloud api key Instantiate: .. code-block:: python from langchain_community.llms.sambanova import SambaNovaCloud SambaNovaCloud( sambanova_api_key="your-SambaNovaCloud-API-key, model = model name, max_tokens = max number of tokens to generate, temperature = model temperature, top_p = model top p, top_k = model top k ) Invoke: .. code-block:: python prompt = "tell me a joke" response = llm.invoke(prompt) Stream: .. code-block:: python for chunk in llm.stream(prompt): print(chunk, end="", flush=True) Async: .. code-block:: python response = llm.ainvoke(prompt) await response rr sambanova_urlsambanova_api_keyzMeta-Llama-3.1-8B-InstructrRFrirTgffffff? temperatureNtop_ptop_k include_usageTstream_optionsceZdZdZdS)SambaNovaCloud.ConfigTNr r%r&r'r(rzr)r&r(r*cdS)r,Fr%r-s r'r/z!SambaNovaCloud.is_lc_serializable}s ur&c ddiS)Nrr%r1s r'r3zSambaNovaCloud.lc_secretss#%899r&c\|j|j|j|j|j|j|jdS)r5rRrrTrrrrrr1s r'r6z"SambaNovaCloud._identifying_paramss8Z/+ZZ"1   r&cdS)z7Get the type of language model used by this chat model.zsambanovacloud-llmr%r1s r'r8zSambaNovaCloud._llm_types $#r&r9c t|ddd|d<tt|dd|d<tjdi|dS) r;r SAMBANOVA_URLz,https://api.sambanova.ai/v1/chat/completionsrrSAMBANOVA_API_KEYNr%)rr r?r@rAs r'r@zSambaNovaCloud.__init__su"6   B # # #  '< )<>Q R R' ' "# ""6"""""r&rjrNc  t|tr|g}d|ddg}|||j||j|j|j|jd}d|D}d|j dd}tj }|r| |j ||d }n| |j ||d }|jd kr td |jd|jd|S)z Performs a post request to the LLM API. Args: prompt: The prompt to pass into the model. stop: list of stop tokens Returns: A request Response object rlrrm)rprFrTrNrRrrrci|] \}}||| Srrr%rss r'r_z2SambaNovaCloud._handle_request..s#OOOzsEU=NU=N=N=Nr&rurvrwTrFrrr)rr[rTrRrrrrerrrrrrrrr) r2rjrNrrrrrrs r'rzSambaNovaCloud._handle_requestsa fc " " XF"(VAY??@ %/Z+ZZ   POTZZ\\OOOV4+A+R+R+T+TVV.    '))  #(("G$t)HH$(("G$u)H  3 & &$'$$=$$$  r&rc |}n,#t$r}td|d|jd}~wwxYw|dddd}|S)rrrNrrrro)rrrrrs r'rz SambaNovaCloud._process_responses $MMOOMM   -a--%]--   #9-a0;IF rc#K ddl}n#t$rtdwxYw||}|D]9}|jdkr t d|jd|jd |jdkrt|jtrtj |j}n t d|jd|jd| dr t d|jd|jdt|d dkr|d dd d }nd }t| }|V#t$r}t d|d|jd}~wwxYwdS)rrNrrrrrrrrrorrrr)rrrrrrrrrr[rrrarbr r) r2rrrrrrorrs r'rz'SambaNovaCloud._process_stream_responses6         E   $$X..]]__# # E{m++"%+%%z%%%  :))!%*c22#z%*55*-'3--$z--- xx((*-'3--$z--- 4 ?++a//"&y/!"4W"=i"H"$&57&C&C&CO))))   "*!**"Z** ?# # s! #>CE F(FFrc |jr"d}|jd|||d|D] }||jz } |S|||d}||}|S)Call out to SambaNovaCloud complete endpoint. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. Returns: The string generated by the model. rrFrr%rrs r'rzSambaNovaCloud._call0rr&c+K|||d}||D]"}|r||j|V#dS)rTrNrrs r'rzSambaNovaCloud._streamMrr&rr),r!r"r#rrrr[rrrrRrrrTintrfloatrrrrdictr(rr/rrr3rr6r8r@r rrrrrr rr rrrrs@r'rrs>>@r***M3***#(52#?#?#?y???!;<< 4  4444l(s077h>W7777x%):> d3in%tCy!67    @%):> d3in%tCy!67    / " r&r)rtypingrrrrrrr r langchain_core.callbacks.managerr #langchain_core.language_models.llmsr langchain_core.outputsr langchain_core.utilsr rpydanticrrrrrr%r&r'rs6 DDDDDDDDDDDDDDDDDDEEEEEE333333222222LLLLLLLL%%%%%%%%NNNNN#NNNbCCCCCSCCCCCr&