Ng'8ddlZddlZddlmZmZmZmZmZmZddl Z ddl m Z m Z ddl mZddlmZddlmZejeZGddeZdS) N)Any AsyncIteratorDictIteratorListOptional)AsyncCallbackManagerForLLMRunCallbackManagerForLLMRun)LLM)GenerationChunk)Fieldc eZdZUdZeed< dZeeed< dZee ed< e dd Z e ed< d Z eeed < d Zeeed < dZeeed< dZeeed< dZeeed< dZeeed< dZeeed< dZee ed< dZee ed< dZee ed< dZeeed< dZeeed< e dd Ze ed< e dd Ze ed< e dd Ze ed < d!Zee ed"< e dd# Ze ed#< e dd$ Ze ed$< gZ ee!eed%< dZ"e ed&< e#d'e$ee%ffd(Z&e#d'e$ee%ffd)Z'e#d'efd*Z(d4d+ee!ed'e$ee%ffd,Z) d5d-ed+ee!ed.ee*d/e%d'ef d0Z+ d5d-ed+ee!ed.ee,d/e%d'ef d1Z- d5d-ed+ee!ed.ee*d/e%d'e.e/f d2Z0 d5d-ed+ee!ed.ee,d/e%d'e1e/f d3Z2dS)6TextGenazText generation models from WebUI. To use, you should have the text-generation-webui installed, a model loaded, and --api added as a command-line option. Suggested installation, use one-click installer for your OS: https://github.com/oobabooga/text-generation-webui#one-click-installers Parameters below taken from text-generation-webui api example: https://github.com/oobabooga/text-generation-webui/blob/main/api-examples/api-example.py Example: .. code-block:: python from langchain_community.llms import TextGen llm = TextGen(model_url="http://localhost:8500") model_urlNpresetmax_new_tokensT do_sample)aliasg? temperatureg?top_p typical_prepsilon_cutoff eta_cutoffgzG?repetition_penalty(top_k min_lengthno_repeat_ngram_size num_beams penalty_alphalength_penaltyFearly_stoppingseed add_bos_tokenitruncation_length ban_eos_tokenskip_special_tokensstopping_strings streamingreturncLid|jd|jd|jd|jd|jd|jd|jd|jd |jd |j d |j d |j d |j d|j d|jd|jd|j|j|j|j|jdS)z/Get the default parameters for calling textgen.rrrrrrrrrrr r!r"r#r$r&r')r(r)r*r+)rrrrrrrrrrr r!r"r#r$r&r'r(r)r*r+selfs \/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/llms/textgen.py_default_paramszTextGen._default_paramsvs( d1   4+  TZ    d1   $/  !$"9  TZ  $/  #D$=    T/  d1  d1 DI! " T/# $"&!7!/#'#; $ 5+    c&id|ji|jS)zGet the identifying parameters.r)rr2r/s r1_identifying_paramszTextGen._identifying_paramssI;/H43GHHr3cdS)zReturn type of llm.textgenr/s r1 _llm_typezTextGen._llm_types yr3stopc|jr|td|j|j}n d|ji}|jp|pg|d<|S)a Performs sanity check, preparing parameters in format needed by textgen. Args: stop (Optional[List[str]]): List of stop sequences for textgen. Returns: Dictionary containing the combined parameters. Nz2`stop` found in both the input and default params.rr+)r+ ValueErrorrr2)r0r:paramss r1_get_parameterszTextGen._get_parameterssa   ST%5QRR R ; )FF ,F&*%:%Hd%Hb!" r3prompt run_managerkwargsc |jr#d}|jd |||d|D] }||jz } |}n|jd}||} | } || d<t j|| } | jdkr'| ddd }ntd | d}|S) Call the textgen web API and return the output. Args: prompt: The prompt to use for generation. stop: A list of strings to stop generation when encountered. Returns: The generated text. Example: .. code-block:: python from langchain_community.llms import TextGen llm = TextGen(model_url="http://localhost:5000") llm.invoke("Write a story about llamas.") r?r:r@/api/v1/generater?jsonresultsrtextERROR: response: r8) r,_streamrKrr>copyrequestspost status_coderHprint r0r?r:r@rAcombined_text_outputchunkresulturlr=requestresponses r1_callz TextGen._calls. > #% %DkEK 3 3% 2$$)FF^555C))$//FkkmmG &GH }Sw777H#s**!3A6v>4(44555 r3cK|jr)d}|jd |||d|23d{V}||jz }6|}n|jd}||} | } || d<t j|| } | jdkr'| dd d }ntd | d}|S) rCrDrENrFr?rGrIrJrrKrLr8) r,_astreamrKrr>rNrOrPrQrHrRrSs r1_acallzTextGen._acalls$. > #% ,t} Dk  EK   3 3 3 3 3 3 3e% 2$$ *FF^555C))$//FkkmmG &GH }Sw777H#s**!3A6v>4(44555 s/c+K ddl}n#t$rtdwxYwi|||}|jd}|}||d<|} | || tj | | } tj | } | ddkr9t| d d } |r| | j | Vn"| dd kr| dS aYields results objects as they are generated in real time. It also calls the callback manager's on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name. Args: prompt: The prompts to pass into the model. stop: Optional list of stop words to use when generating. Returns: A generator representing the stream of tokens being generated. Yields: A dictionary like objects containing a string token and metadata. See text-generation-webui docs and below for more. Example: .. code-block:: python from langchain_community.llms import TextGen llm = TextGen( model_url = "ws://localhost:5005" streaming=True ) for chunk in llm.stream("Ask 'Hi, how are you?' like a pirate:'", stop=["'"," "]): print(chunk, end='', flush=True) # noqa: T201 rNz9The `websocket-client` package is required for streaming.z/api/v1/streamr?Tevent text_streamrK)rKgeneration_info)token stream_end websocket ImportErrorr>rrN WebSocketconnectsendrHdumpsrecvloadsr on_llm_new_tokenrKclose r0r?r:r@rArfr=rWrXwebsocket_clientrVrUs r1rMzTextGen._streamsH         K   :D((..9&9///++--"$..00  %%%dj11222 %**,,FZ''Fg-//'$(C00uz0BBB L00 &&(((  #c K ddl}n#t$rtdwxYwi|||}|jd}|}||d<|} | || tj | | } tj | } | ddkr@t| d d } |r!| | j d{V| WVn"| dd kr| dSr_rerps r1r\zTextGen._astreamYsH         K   :D((..9&9///++--"$..00  %%%dj11222 %**,,FZ''Fg-//'$(I%66UZ6HHHHHHHHH L00 &&((( rr)N)NN)3__name__ __module__ __qualname____doc__str__annotations__rrrintr rboolrfloatrrrrrrrr r!r"r#r$r&r'r(r)r*r+rr,propertyrrr2r5r9r>r rZr r]rr rMrr\r8r3r1rrs$NNNJ FHSM   1$'NHSM'''3eD 444It444#&K%&&&M!E8E?   G"#Ix"""8'(NHUO'''"#J###*....F E8E??!"J !!!.*+(3-+++< !Ix}   %&M8E?&&&&'NHUO''' 5.>???ND???b'''D#'''%O<<:%):> ,,,tCy!,67 ,  , ,,,,b%)?C ,,,tCy!,;< ,  , ,,,,b%):> FFFtCy!F67 F  F / " FFFFV%)?C FFFtCy!F;< F  F  ' FFFFFFr3r)rHloggingtypingrrrrrrrOlangchain_core.callbacksr r #langchain_core.language_models.llmsr langchain_core.outputsr pydanticr getLoggerrtloggerrr8r3r1rs EEEEEEEEEEEEEEEE433333222222  8 $ $NNNNNcNNNNNr3