§ ¦ìNg’ãóÈ—dZddlZddlmZmZmZmZmZddlm Z ddl m Z ej e ¦«Z ddlmZn #e$rdZYnwxYwGd„de¦«ZGd „d e ¦«ZdS) z0Ratelimiting Handler to limit requests or tokenséN)ÚAnyÚDictÚListÚLiteralÚOptional)ÚBaseCallbackHandler)Ú LLMResult)Ú Ratelimitc ó^‡—eZdZdZ d dededdeedeefˆfd„ Z ˆxZ S) ÚUpstashRatelimitErrorzi Upstash Ratelimit Error Raised when the rate limit is reached in `UpstashRatelimitHandler` NÚmessageÚtype)ÚtokenÚrequestÚlimitÚresetcót•—t¦« |¦«||_||_||_dS)a¢ Args: message (str): error message type (str): The kind of the limit which was reached. One of "token" or "request" limit (Optional[int]): The limit which was reached. Passed when type is request reset (Optional[int]): unix timestamp in milliseconds when the limits are reset. Passed when type is request N)ÚsuperÚ__init__rrr)Úselfr rrrÚ __class__s €út/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/callbacks/upstash_ratelimit_callback.pyrzUpstashRatelimitError.__init__s6ø€õ$ ‰Œ×Ò˜Ñ!Ô!Ð!؈Œ ؈Œ ؈Œ ˆ ˆ ó)NN) Ú__name__Ú __module__Ú __qualname__Ú__doc__ÚstrrrÚintÚfloatrÚ __classcell__)rs@rr r sˆø€€€€€ððð $Ø!%ð ðàððÐ(Ô)ðð˜Œ}ð 𠘌ð ðððððððððrr c ó—eZdZUdZdZeed<dZeed<ddddœded e e d e e d efd „Z d e ee fde ee fde de fd„Zd e ee fdeede ddfd„Zdede ddfd„Zdde eddfd„ZdS)ÚUpstashRatelimitHandleraÈ Callback to handle rate limiting based on the number of requests or the number of tokens in the input. It uses Upstash Ratelimit to track the ratelimit which utilizes Upstash Redis to track the state. Should not be passed to the chain when initialising the chain. This is because the handler has a state which should be fresh every time invoke is called. Instead, initialise and pass a handler every time you invoke. TÚ raise_errorFÚ_checkedN)Útoken_ratelimitÚrequest_ratelimitÚinclude_output_tokensÚ identifierr&r'r(có~—t||g¦«std¦«‚||_||_||_||_dS)a@ Creates UpstashRatelimitHandler. Must be passed an identifier to ratelimit like a user id or an ip address. Additionally, it must be passed at least one of token_ratelimit or request_ratelimit parameters. Args: identifier Union[int, str]: the identifier token_ratelimit Optional[Ratelimit]: Ratelimit to limit the number of tokens. Only works with OpenAI models since only these models provide the number of tokens as information in their output. request_ratelimit Optional[Ratelimit]: Ratelimit to limit the number of requests include_output_tokens bool: Whether to count output tokens when rate limiting based on number of tokens. Only used when `token_ratelimit` is passed. False by default. Example: .. code-block:: python from upstash_redis import Redis from upstash_ratelimit import Ratelimit, FixedWindow redis = Redis.from_env() ratelimit = Ratelimit( redis=redis, # fixed window to allow 10 requests every 10 seconds: limiter=FixedWindow(max_requests=10, window=10), ) user_id = "foo" handler = UpstashRatelimitHandler( identifier=user_id, request_ratelimit=ratelimit ) # Initialize a simple runnable to test chain = RunnableLambda(str) # pass handler as callback: output = chain.invoke( "input", config={ "callbacks": [handler] } ) zhYou must pass at least one of input_token_ratelimit or request_ratelimit parameters for handler to work.N)ÚanyÚ ValueErrorr)r&r'r()rr)r&r'r(s rrz UpstashRatelimitHandler.__init__@sZ€õtOÐ%6Ð7Ñ8Ô8ð ÝðEñôð ð %ˆŒØ.ˆÔØ!2ˆÔØ%:ˆÔ"Ð"Ð"rÚ serializedÚinputsÚkwargsÚreturnc ó¼—|jrR|jsM|j |j¦«}|jst dd|j|j¦«‚d|_dSdSdS)a\ Run when chain starts running. on_chain_start runs multiple times during a chain execution. To make sure that it's only called once, we keep a bool state `_checked`. If not `self._checked`, we call limit with `request_ratelimit` and raise `UpstashRatelimitError` if the identifier is rate limited. zRequest limit reached!rTN)r'r%rr)Úallowedr r)rr-r.r/Úresponses rÚon_chain_startz&UpstashRatelimitHandler.on_chain_start…s{€ð Ô !ð !¨$¬-ð !ØÔ-×3Ò3°D´OÑDÔDˆHØÔ#ð Ý+Ø,¨i¸¼ÈÌñôðð!ˆDŒMˆMˆMð  !ð !ð !ð !rÚpromptsc ó‚—|jr5|j |j¦«}|dkrtdd¦«‚dSdS)z- Run when LLM starts running rzToken limit reached!rN)r&Ú get_remainingr)r )rr-r5r/Ú remainings rÚ on_llm_startz$UpstashRatelimitHandler.on_llm_start˜sS€ð Ô ð MØÔ,×:Ò:¸4¼?ÑKÔKˆIؘAŠ~ˆ~Ý+Ð,BÀGÑLÔLÐLð Mð Màˆ~rr3c óè—|jrj |jpi}|d}|jr|dn|d}n#t$rt d¦«‚wxYw|j |j|¬¦«dSdS)z¬ Run when LLM ends running If the `include_output_tokens` is set to True, number of tokens in LLM completion are counted for rate limiting Ú token_usageÚ total_tokensÚ prompt_tokenszþLLM response doesn't include `token_usage: {total_tokens: int, prompt_tokens: int}` field. To use UpstashRatelimitHandler with token_ratelimit, either use a model which returns token_usage (like OpenAI models) or rate limit only with request_ratelimit.)ÚrateN)r&Ú llm_outputr(ÚKeyErrorr,rr))rr3r/r?r;Ú token_counts rÚ on_llm_endz"UpstashRatelimitHandler.on_llm_end£s¹€ð Ô ð Jð Ø%Ô0Ð6°B Ø(¨Ô7 ðÔ1ð6K Ô/Ð/à$ _Ô5ð øõ ð ð ð Ý ðQñôðð øøøð Ô × &Ò & t¤¸[Ð &Ñ IÔ IÐ IÐ IÐ Ið+ Jð Js ‰(2²A cóT—t|p|j|j|j|j¬¦«S)zÐ Creates a new UpstashRatelimitHandler object with the same ratelimit configurations but with a new identifier if it's provided. Also resets the state of the handler. )r)r&r'r()r#r)r&r'r()rr)s rrzUpstashRatelimitHandler.resetÁs7€õ'Ø!Ð4 T¤_Ø Ô0Ø"Ô4Ø"&Ô"<ð  ñ ô ð r)N)rrrrr$ÚboolÚ__annotations__r%rrr rrrr4rr9r rBr©rrr#r#/sŽ€€€€€€ð ð ð€KÐÐÑØ€HˆdÐÐÑð 04Ø15Ø&+ð C;ðC;ðC;àðC;ð" )Ô,ð C;ð $ IÔ.ð C;ð $ð C;ðC;ðC;ðC;ðJ!ؘs C˜xœ.ð!Ø26°s¸C°x´.ð!ØLOð!à ð!ð!ð!ð!ð& Mؘs C˜xœ.ð MØ37¸´9ð MØHKð Mà ð Mð Mð Mð MðJ 9ðJ¸ðJÀðJðJðJðJð<  ð   ¨¤ ð  Ð9Rð  ð  ð  ð  ð  ð  rr#)rÚloggingÚtypingrrrrrÚlangchain_core.callbacksrÚlangchain_core.outputsr Ú getLoggerrÚloggerÚupstash_ratelimitr Ú ImportErrorÚ Exceptionr r#rFrrúrPsðØ6Ð6à€€€Ø5Ð5Ð5Ð5Ð5Ð5Ð5Ð5Ð5Ð5Ð5Ð5Ð5Ð5à8Ð8Ð8Ð8Ð8Ð8Ø,Ð,Ð,Ð,Ð,Ð,à ˆÔ ˜8Ñ $Ô $€ðØ+Ð+Ð+Ð+Ð+Ð+Ð+øØðððØ€I€I€Iðøøøððððð˜Iñôðð>_ ð_ ð_ ð_ ð_ Ð1ñ_ ô_ ð_ ð_ ð_ s²9¹AÁA