gdZddlmZddlmZddlmZddlmZej e Z GddeZ Gd d eZ d S) zT5 model configuration)Mapping)PretrainedConfig)OnnxSeq2SeqConfigWithPast)loggingc`eZdZdZdZdgZdddddZ dfd ZxZS)T5Configa This is the configuration class to store the configuration of a [`T5Model`] or a [`TFT5Model`]. It is used to instantiate a T5 model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the T5 [google-t5/t5-small](https://huggingface.co/google-t5/t5-small) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Arguments: vocab_size (`int`, *optional*, defaults to 32128): Vocabulary size of the T5 model. Defines the number of different tokens that can be represented by the `inputs_ids` passed when calling [`T5Model`] or [`TFT5Model`]. d_model (`int`, *optional*, defaults to 512): Size of the encoder layers and the pooler layer. d_kv (`int`, *optional*, defaults to 64): Size of the key, query, value projections per attention head. The `inner_dim` of the projection layer will be defined as `num_heads * d_kv`. d_ff (`int`, *optional*, defaults to 2048): Size of the intermediate feed forward layer in each `T5Block`. num_layers (`int`, *optional*, defaults to 6): Number of hidden layers in the Transformer encoder. num_decoder_layers (`int`, *optional*): Number of hidden layers in the Transformer decoder. Will use the same value as `num_layers` if not set. num_heads (`int`, *optional*, defaults to 8): Number of attention heads for each attention layer in the Transformer encoder. relative_attention_num_buckets (`int`, *optional*, defaults to 32): The number of buckets to use for each attention layer. relative_attention_max_distance (`int`, *optional*, defaults to 128): The maximum distance of the longer sequences for the bucket separation. dropout_rate (`float`, *optional*, defaults to 0.1): The ratio for all dropout layers. classifier_dropout (`float`, *optional*, defaults to 0.0): The dropout ratio for classifier. layer_norm_eps (`float`, *optional*, defaults to 1e-6): The epsilon used by the layer normalization layers. initializer_factor (`float`, *optional*, defaults to 1): A factor for initializing all weight matrices (should be kept to 1, used internally for initialization testing). feed_forward_proj (`string`, *optional*, defaults to `"relu"`): Type of feed forward layer to be used. Should be one of `"relu"` or `"gated-gelu"`. T5v1.1 uses the `"gated-gelu"` feed forward projection. Original T5 uses `"relu"`. use_cache (`bool`, *optional*, defaults to `True`): Whether or not the model should return the last key/values attentions (not used by all models). t5past_key_valuesd_model num_heads num_layersd_kv) hidden_sizenum_attention_headsnum_hidden_layershead_dim}@N 皙?ư>?reluTrc :||_||_||_||_||_||n|j|_||_||_| |_| |_ ||_ | |_ | |_ | |_ ||_|j d}|d|_|ddk|_t%|dkr |ddkst%|dkrt'd| d| d krd |_t)jd |||d |dS) N-rgatedr z`feed_forward_proj`: z is not a valid activation function of the dense layer. Please make sure `feed_forward_proj` is of the format `gated-{ACT_FN}` or `{ACT_FN}`, e.g. 'gated-gelu' or 'relu'z gated-gelugelu_new) pad_token_id eos_token_idis_encoder_decoder) vocab_sizer rd_ffrnum_decoder_layersr relative_attention_num_bucketsrelative_attention_max_distance dropout_rateclassifier_dropoutlayer_norm_epsiloninitializer_factorfeed_forward_proj use_cachesplit dense_act_fn is_gated_actlen ValueErrorsuper__init__)selfr,r rr-rr.r r/r0r1r3r4r5r*r6r(r)r2kwargsact_info __class__s c/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/models/t5/configuration_t5.pyr=zT5Config.__init__Ssj,%   $"4"@  do #.L+/N,("4"4"4!2")//44$RL$QK72 x==1  !!7!73x==1;L;L)(9)))   , , *D  %%1        )rrrrrNrrrrrrrTTrr r!) __name__ __module__ __qualname____doc__ model_typekeys_to_ignore_at_inference attribute_mapr= __classcell__)rAs@rBr r s,,\J#4"5 *) M  ')(+ '< < < < < < < < < < rCr cfeZdZedeeeeefffdZedefdZdS) T5OnnxConfigreturncddddddd}|jrd|dd<ddi|d <dd d|d <ndd d|d <dd d|d <|jr||d |S)Nbatchencoder_sequence)rr ) input_idsattention_maskz past_encoder_sequence + sequencerSr rdecoder_input_idsz past_decoder_sequence + sequencedecoder_attention_maskdecoder_sequenceinputs) direction)use_pastfill_with_past_key_values_)r> common_inputss rBrWzT5OnnxConfig.inputss%);<<").@AA   = Z1SM* +A .23WM- .:AFh6i6iM2 3 35s rBdefault_onnx_opsetzT5OnnxConfig.default_onnx_opsetsrrCN) rDrErFpropertyrstrintrWr^r+rCrBrMrMss WS#X%6 67X$CXrCrMN)rGtypingrconfiguration_utilsronnxrutilsr get_loggerrDloggerr rMr+rCrBrhs333333------  H % %t t t t t t t t n,rC