gYddlZddlZddlmZddlmZddlmZddl m Z ddl m Z e e ZdZGddZGd d ZGd d eZGd deZGddeZGddeZGddeZGddeZGddeZGddeZGddeZGddeZGdd eZGd!d"eZdS)#N)sparse)add_start_docstrings) get_loggerad Args: input_ids (`jnp.ndarray` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using [`PreTrainedTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) scores (`jnp.ndarray` of shape `(batch_size, config.vocab_size)`): Prediction scores of a language modeling head. These can be logits for each vocabulary when not using beam search or log softmax for each vocabulary token when using beam search kwargs (`Dict[str, Any]`, *optional*): Additional logits processor specific kwargs. Return: `jnp.ndarray` of shape `(batch_size, config.vocab_size)`: The processed prediction scores. cfeZdZdZeedejdejdejfdZdS)FlaxLogitsProcessorzSAbstract base class for all logit processors that can be applied during generation. input_idsscoresreturnc0t|jd)z"Flax method for processing logits.H is an abstract class. Only classes inheriting this class can be called.NotImplementedError __class__selfr r s g/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/generation/flax_logits_process.py__call__zFlaxLogitsProcessor.__call__6$"~ g g g   N __name__ __module__ __qualname____doc__r!LOGITS_PROCESSOR_INPUTS_DOCSTRINGjnpndarrayrrrrr3s`]];<< #+ s{ s{   =<   rrcfeZdZdZeedejdejdejfdZdS)FlaxLogitsWarperzjAbstract base class for all logit warpers that can be applied during generation with multinomial sampling.r r r c0t|jd)zFlax method for warping logits.r rrs rrzFlaxLogitsWarper.__call__ArrNrrrrr!r!>s`tt;<< #+ s{ s{   =<   rr!c jeZdZdZeedejdejdedejfdZ dS)FlaxLogitsProcessorLista. This class can be used to create a list of [`FlaxLogitsProcessor`] or [`FlaxLogitsWarper`] to subsequently process a `scores` input tensor. This class inherits from list and adds a specific *__call__* method to apply each [`FlaxLogitsProcessor`] or [`FlaxLogitsWarper`] to the inputs. r r cur_lenr c |D]}tj|jj}t |dkrt fdt |ddDs:tdt |d|j d||||fi}||||}|S)Nc3 K|]}|vV dSNr).0argkwargss r z3FlaxLogitsProcessorList.__call__..Us'SSS3&=SSSSSSrrz,Make sure that all the required parameters: z for z$ are passed to the logits processor.) inspect signaturer parameterslenalllistkeys ValueErrorr)rr r r%r, processor function_argss ` rrz FlaxLogitsProcessorList.__call__Ps ? ?I#-i.@AALM=!!A%%SSSSD9K9K9M9M4N4Nqrr4RSSSSS$UtML^L^L`L`GaGaUU$.UUU#9fgHHHH"9fg>> rN) rrrrrrrrintrrrrr$r$Iso ;<< #+ s{ S _b_j   =<   rr$cTeZdZdZdefdZdejdejdedejfdZ d S) FlaxTemperatureLogitsWarperz [`FlaxLogitsWarper`] for temperature (exponential scaling output probability distribution). Args: temperature (`float`): The value used to module the logits distribution. temperaturecnt|tr|dkstd|||_dS)Nrz:`temperature` has to be a strictly positive float, but is ) isinstancefloatr5r;)rr;s r__init__z$FlaxTemperatureLogitsWarper.__init__isC+u-- ikAoogZegghh h&rr r r%r c||jz }|Sr))r;rr r r%s rrz$FlaxTemperatureLogitsWarper.__call__os$** rN) rrrrr>r?rrr8rrrrr:r:`sq'E'''' #+s{SUXU`rr:cveZdZdZed dfdededefdZdejd ejd ed ejfd Z d S)FlaxTopPLogitsWarpera= [`FlaxLogitsWarper`] that performs top-p, i.e. restricting to top tokens summing to prob_cut_off <= prob_cut_off. Args: top_p (`float`): If set to < 1, only the smallest set of most probable tokens with probabilities that add up to `top_p` or higher are kept for generation. filter_value (`float`, *optional*, defaults to -inf): All filtered values will be set to this float value. min_tokens_to_keep (`int`, *optional*, defaults to 1): Minimum number of tokens that cannot be filtered. Inftop_p filter_valuemin_tokens_to_keepct|tr |dks|dkrtd|t|tr|dkrtd|||_||_||_dS)Nrg?z.`top_p` has to be a float > 0 and < 1, but is rEz:`min_tokens_to_keep` has to be a positive integer, but is )r=r>r5r8rFrGrH)rrFrGrHs rr?zFlaxTopPLogitsWarper.__init__s%'' WEAIIUeUUVV V,c22 p7IA7M7MnZlnnoo o ("4rr r r%r cFtj||jd\}}tj||j}t j|d d}||j k}tj |d}||j dddf dz}|j ddd|jf d}tj|||} t j|| d} | S)NaxisrErT)laxtop_kshaper full_likerGjaxnnsoftmaxcumsumrFrollatsetrHwhere sort_key_val) rr r r% topk_scores topk_indices mask_scorescumulative_probs score_masktopk_next_scores next_scoress rrzFlaxTopPLogitsWarper.__call__s$'Iffl26F$G$G! \mFD,=>> 6>>+B>??FFBFOO% 2 Xj!,, jmAAAqD)--d333  ]111&?(?&?#?@DDTJJ 9ZkJJg**<9IJJ2N rN rrrrr>r8r?rrrrrrrCrCts  =BE%LL=de55e555^a5555#+s{SUXU`rrCcveZdZdZed dfdededefdZdejd ejd ed ejfd Z d S)FlaxTopKLogitsWarpera [`FlaxLogitsWarper`] that performs top-k, i.e. restricting to the k highest probability elements. Args: top_k (`int`): The number of highest probability vocabulary tokens to keep for top-k-filtering. filter_value (`float`, *optional*, defaults to -inf): All filtered values will be set to this float value. min_tokens_to_keep (`int`, *optional*, defaults to 1): Minimum number of tokens that cannot be filtered. rDrErOrGrHct|tr|dkrtd|t|||_||_dS)Nrz6`top_k` has to be a strictly positive integer, but is )r=r8r5maxrOrG)rrOrGrHs rr?zFlaxTopKLogitsWarper.__init__sU%%% _!]V[]]^^ ^ 233 (rr r r%r c|j\}}tj||z|j}t |j|jd}t j||\}} tjtj||zdddf||f } | } | | z} |j |  | }| ||} | S)NrK) rPrfullrGminrOrN broadcast_toarangeflattenrWrXreshape)rr r r% batch_size vocab_sizenext_scores_flattopkr[r\shifttopk_scores_flattopk_indices_flatras rrzFlaxTopKLogitsWarper.__call__s!' J8J$;T=NOO4:v|B/00$'Ifd$;$;! \ #*Z"8"8:"Eqqq$w!OR\^bQcddllnn&..00(0022U:+./@AEEFVWW&..z:FF rNrbrrrrdrds  ;@%,,bc))c))\_)))) #+ s{ S UXU`      rrdcTeZdZdZdefdZdejdejdedejfdZd S) !FlaxForcedBOSTokenLogitsProcessorz [`FlaxLogitsProcessor`] that enforces the specified token as the first generated token. Args: bos_token_id (`int`): The id of the token to force as the first generated token. bos_token_idc||_dSr))rw)rrws rr?z*FlaxForcedBOSTokenLogitsProcessor.__init__s(rr r r%r ctj|jtd }dtj|dz z }tj||jdd|jfd|}|SNinfrEr) rrhrPr>bool_rYrWrwrXrr r r% new_scores apply_penaltys rrz*FlaxForcedBOSTokenLogitsProcessor.__call__spXflU5\\M:: CIgk222 =*-4;L8L*M*Q*QRS*T*TV\]] rN rrrrr8r?rrrrrrrvrvsq)S))))#+s{SUXU`rrvcXeZdZdZdedefdZdejdejdedejfd Zd S) !FlaxForcedEOSTokenLogitsProcessorae [`FlaxLogitsProcessor`] that enforces the specified token as the last generated token when `max_length` is reached. Args: max_length (`int`): The maximum length of the sequence to be generated. eos_token_id (`int`): The id of the token to force as the last generated token when `max_length` is reached. max_length eos_token_idc"||_||_dSr))rr)rrrs rr?z*FlaxForcedEOSTokenLogitsProcessor.__init__s$(rr r r%r ctj|jtd }dtj||jz dzz }tj||jdd|jf d|}|Srz) rrhrPr>r|rrYrWrrXr}s rrz*FlaxForcedEOSTokenLogitsProcessor.__call__sxXflU5\\M:: CIg&?!&CDDD =*-4;L8L*M*Q*QRS*T*TV\]] rNrrrrrrsx)3)c))))#+s{SUXU`rrcXeZdZdZdedefdZdejdejdedejfd Zd S) FlaxMinLengthLogitsProcessora3 [`FlaxLogitsProcessor`] enforcing a min-length by setting EOS probability to 0. Args: min_length (`int`): The minimum length below which the score of `eos_token_id` is set to `-float("Inf")`. eos_token_id (`int`): The id of the *end-of-sequence* token. min_lengthrct|tr|dkrtd|t|tr|dkrtd|||_||_dS)Nrz2`min_length` has to be a positive integer, but is z4`eos_token_id` has to be a positive integer, but is )r=r8r5rr)rrrs rr?z%FlaxMinLengthLogitsProcessor.__init__s*c** `j1nn^R\^^__ _,,, d q0@0@bT`bbcc c$(rr r r%r cdtj||jz ddz }tj||jdd|jftd |}|S)NrErr{)rcliprrYrWrrXr>rr r r%rs rrz%FlaxMinLengthLogitsProcessor.__call__sdCHWt%>1EEE =&)AAAt7H4H*I*M*MuUZ||m*\*\^dee rNrrrrrrsx)3)c))))#+s{SUXU`rrc$eZdZdZdZdefdZdS)(FlaxSuppressTokensAtBeginLogitsProcessora [`FlaxLogitsProcessor`] supressing a list of tokens as soon as the `generate` function starts generating using `begin_index` tokens. This should ensure that the tokens defined by `begin_suppress_tokens` are not sampled at the begining of the generation. Args: begin_suppress_tokens (`List[int]`): Tokens to not sample. begin_index (`int`): Index where the tokens are suppressed. c<t||_||_dSr))r3begin_suppress_tokens begin_index)rrrs rr?z1FlaxSuppressTokensAtBeginLogitsProcessor.__init__s!%)*?%@%@"&rr%cdtj||jz z }tj||jdd|jftd |}|S)NrEr{)rr|rrYrWrrXr>rs rrz1FlaxSuppressTokensAtBeginLogitsProcessor.__call__!seCIg0@&@AAA =&)AAAt7Q4Q*R*V*VX]^cXdXdWd*e*egmnn rN)rrrrr?r8rrrrrrsH  '''3rrcTeZdZdZdefdZdejdejdedejfdZ d S) !FlaxSuppressTokensLogitsProcessorz [`FlaxLogitsProcessor`] suppressing a list of tokens at each decoding step. The processor will set their log probs to be `-inf` so they are not sampled. Args: suppress_tokens (`list`): Tokens to not sample. suppress_tokensc.t||_dSr))r3r)rrs rr?z*FlaxSuppressTokensLogitsProcessor.__init__3s#O44rr r r%r cp|jd|jftd }|S)N.r{)rWrrXr>rAs rrz*FlaxSuppressTokensLogitsProcessor.__call__6s13 445995<<-HH rN) rrrrr3r?rrr8rrrrrr)sq55555#+s{SUXU`rrcNeZdZdZdZdejdejdedejfdZdS) FlaxForceTokensLogitsProcessora [`FlaxLogitsProcessor`] that takes a list of pairs of integers which indicates a mapping from generation indices to token indices that will be forced before sampling. The processor will set their log probs to 0 and all other tokens to `-inf` so that they are sampled at their corresponding index. Args: force_token_map (`list`): Map giving token ids and indices where they will be forced to be sampled. cXt|}tjt|dztjdz}|D]'\}}| |j||}(tj||_ dS)NrEdtyperK) dictronesrfr4int32itemsrWrXforce_token_array)rforce_token_maprindextokens rr?z'FlaxForceTokensLogitsProcessor.__init__Gs// Hc/*>*>*@*@&A&AA&EciXXX[]]+1133 K KLE5 $5$8$?$C$CE$J$J!!$+zerosrNdynamic_update_slice)generation_idxrn current_tokenr~updatesr rs r _force_tokenz=FlaxForceTokensLogitsProcessor.__call__.._force_tokenSswaJ 2>BMvV\BBBeEll]RJiQv|DDDG1*g=GYZZJ rrcSr)rr srz9FlaxForceTokensLogitsProcessor.__call__.._sFrc\tjjdkfdfdS)NrcSr)r)rr%srrzKFlaxForceTokensLogitsProcessor.__call__....ds W--rcSr)rrsrrzKFlaxForceTokensLogitsProcessor.__call__....fsr)rNcondr)rr%r rsrrz9FlaxForceTokensLogitsProcessor.__call__..as;CH&w/14----- r)rNrrrP)rr r r%rs` ``@rrz'FlaxForceTokensLogitsProcessor.__call__Rs}       t-3A6 6 NNN           rN) rrrrr?rrr8rrrrrr<se > > >#+s{SUXU`rrceZdZdZdZdZdS)#FlaxWhisperTimeStampLogitsProcessora{ Whisper specific Processor. This processor can be used to force a list of tokens. The processor will set their log probs to `inf` so that they are sampled at their corresponding index. Args: generate_config (`GenerateConfig`): The generate config used to generate the output. The following parameters are required: eos_token_id (`int`, *optional*, defaults to 50257): The id of the *end-of-sequence* token. no_timestamps_token_id (`int`, *optional*, defaults to 50363): The id of the `"<|notimestamps|>"` token. max_initial_timestamp_index (`int`, *optional*, defaults to 1): Used to set the maximum value of the initial timestamp. This is used to prevent the model from predicting timestamps that are too far in the future. c|j|_|j|_|jdz|_|dz|_|jr|xjdz c_t |dr |j|_n |j|_|j|j|_dSdS)NrErmax_initial_timestamp_index)rno_timestamps_token_idtimestamp_beginris_multilingualhasattrrro)rgenerate_config model_configdecoder_input_lengths rr?z,FlaxWhisperTimeStampLogitsProcessor.__init__}s+8&5&L#.EI/!3  * "    !   ?$A B B G/>/ZD , ,/;/FD ,  + 3/;/FD , , , 4 3rct|jddjftd }fd}t j|||}t jjkdd}t jj du|d}j j z}t j||jdd|dzdftd |}tj |d}fd}t j|||}|S) Nr{c Ttjjz dkdd}tj|dz jk|d}tjjz dkdd}tj|dz jkd|}tj|tj|dk|jjdt d |jdjt d |S)NrETFrrr{)rrYrrrWrXr>r) input_ids_kscores_klast_was_timestamppenultimate_was_timestampr%rs r handle_pairszBFlaxWhisperTimeStampLogitsProcessor.__call__..handle_pairss*!$Gd6F,F1+LdTY!Z!Z !$GaK(D,@@+""  ), 7T=M3MQR2RTXZ_(`(` %(+ GaK(D,@@))) % 9" -1K 4 6 67;;U5\\MJJK 3$"3 3488%,,GG  rTFrErKrLc<tj|jdd}t j|dj}t j||k|jdjtd |S)NrKrLr{) rRrS logsumexprrrfrYrWrXr>) logprobs_krtimestamp_logprobmax_text_token_logprobrs rhandle_cumulative_probszMFlaxWhisperTimeStampLogitsProcessor.__call__..handle_cumulative_probss # 0 0DrRvmaprrYrrrrS log_softmax) rr r r%rapply_max_initial_timestamp last_allowedlogprobsrs ` ` rrz,FlaxWhisperTimeStampLogitsProcessor.__call__sb111d99:>>e }MM      2(,'' 6::&)i4;K0KTSX&Y&Y#&)i  ,D 80 ' ' # +d.NN  ' Iaaa)+++ , 0 0%,, ? ?   6%%f2%66     31228VDD rN)rrrrr?rrrrrrls? GGG"<<<<<rrceZdZdZdefdZdejdedefdZdejd ejfd Z dejd ejded ejfd Z d S) FlaxNoRepeatNGramLogitsProcessora9 [`FlaxLogitsProcessor`] that enforces no repetition of n-grams. See [Fairseq](https://github.com/pytorch/fairseq/blob/a07cb6f40480928c9e0548b737aadd36ee66ac76/fairseq/sequence_generator.py#L345). Args: ngram_size (`int`): All ngrams of size `ngram_size` can only occur once. ngram_sizecnt|tr|dkrtd|||_dS)Nrz;`ngram_size` has to be a strictly positive integer, but is )r=r8r5r)rrs rr?z)FlaxNoRepeatNGramLogitsProcessor.__init__s@*c** ijAoog[egghh h$rr ror%c  j\ }|jdz z }|jdz z } fd} |zjdzf}tjd |z|t j|j} t j |z |zk d} tj | | f f|fjzzS)a get a matrix of size (batch_size,) + (vocab_size,)*n (for n-grams) that represent the n-grams that occurred previously. The BCOO representation allow to store only the few non-zero entries, instead of the full (huge) matrix rEc |z|z|j|tjgfdt jDzS)NcNg|]!}tj|zf"Sr)rarray)r*jbr poss r zZFlaxNoRepeatNGramLogitsProcessor.get_previous_ngrams..body_fun..s0XXXAsy++AsQwJ7XXXr)rWrXrrranger)ivalrrrnr rs @@rbody_funzFFlaxNoRepeatNGramLogitsProcessor.get_previous_ngrams..body_funszJAz/C6!9== YXXXXXtAWAWXXXY rrrfloat32)rP) rPrrRrN fori_looprrrrkastyperBCOO) rr ror%seq_len seq_ngrams cur_ngramsrrPall_update_indicesdatarns `` @rget_previous_ngramsz4FlaxNoRepeatNGramLogitsProcessor.get_previous_ngramss (o G! 34 ! 34        j($/A*=> W.. zJ&#)E2Y2Y2Y    : 233j:6MMUUV_``{D"45j]j]]a]lMl=lmmmmr latest_tokensr ctjtjd}tj|||S)zt Determines which tokens must be banned given latest tokens and the previously seen ngrams. c,|t|Sr))tuple)rprevious_ngramss rinner_fnzIFlaxNoRepeatNGramLogitsProcessor.get_banned_tokens_mask..inner_fns#5#7#78 8r)rsparsifyrRr bcoo_todense)rrrrs rget_banned_tokens_maskz7FlaxNoRepeatNGramLogitsProcessor.get_banned_tokens_masksJ   9 9   9"88M?#K#KLLLrr c~fd}tjjdz k|fd}|S)Nc j\}}|}tjjdjdz fj}t j|t j djdz z fjdjdz fd} || d}tj |td S)NrrEr)rrboolr{)rPrrrrrrRrNr dynamic_slicerrrYr>) _rorrbanned_tokens_indices_maskr%r r rs rtrue_fnz:FlaxNoRepeatNGramLogitsProcessor.__call__..true_fn s"LMAz"66y*gVVO Iyq'94?Q;N&OW`WfgggMG88%%7do.A#BCioVWFX[_[jmn[nEp M*.)D)D]Tc)d)d)k)klr)s)s &97%,,OO OrrEcSr)rrsrrz;FlaxNoRepeatNGramLogitsProcessor.__call__..sQWr)rRrNrr)rr r r%routputs```` rrz)FlaxNoRepeatNGramLogitsProcessor.__call__ sf P P P P P P P P&w$/A*==XX rN) rrrrr8r?rrrrrrrrrrs%3%%%% nS[ nc nTW n n n nD MCK MUXU` M M M M#+s{SUXU`rr)r.rRjax.laxrN jax.numpynumpyrjax.experimentalrutilsr utils.loggingrrloggerrrr!r3r$r:rCrdrvrrrrrrrrrrrs  ######((((((&&&&&& H  %!*                d."2()))))+)))X     +   F(;,(;2#6</B2(;&-----%8---`^^^^^*=^^^BSSSSS':SSSSSr