g"rdZddlmZmZmZddlmZddlmZej e Z GddeZ dS) zProphetNet model configuration)CallableOptionalUnion)PretrainedConfig)loggingc6XeZdZdZdZdgZddiZ d4deedee e e fdee dee dee dee dee dee dee dee d eed!eed"ee d#eed$ee d%ee d&ee d'ee d(ee d)ee d*ee d+eed,ee d-ee d.ee d/ee f4fd0 Zed1e fd2Zejd3ZxZS)5ProphetNetConfiga This is the configuration class to store the configuration of a [`ProphetNetModel`]. It is used to instantiate a ProphetNet model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the ProphetNet [microsoft/prophetnet-large-uncased](https://huggingface.co/microsoft/prophetnet-large-uncased) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: activation_dropout (`float`, *optional*, defaults to 0.1): The dropout ratio for activations inside the fully connected layer. activation_function (`str` or `function`, *optional*, defaults to `"gelu"`): The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`, `"relu"`, `"silu"` and `"gelu_new"` are supported. vocab_size (`int`, *optional*, defaults to 30522): Vocabulary size of the ProphetNET model. Defines the number of different tokens that can be represented by the `inputs_ids` passed when calling [`ProphetNetModel`]. hidden_size (`int`, *optional*, defaults to 1024): Dimensionality of the layers and the pooler layer. encoder_ffn_dim (`int`, *optional*, defaults to 4096): Dimensionality of the "intermediate" (often named feed-forward) layer in decoder. num_encoder_layers (`int`, *optional*, defaults to 12): Number of encoder layers. num_encoder_attention_heads (`int`, *optional*, defaults to 16): Number of attention heads for each attention layer in the Transformer encoder. decoder_ffn_dim (`int`, *optional*, defaults to 4096): Dimensionality of the `intermediate` (often named feed-forward) layer in decoder. num_decoder_layers (`int`, *optional*, defaults to 12): Number of decoder layers. num_decoder_attention_heads (`int`, *optional*, defaults to 16): Number of attention heads for each attention layer in the Transformer decoder. attention_dropout (`float`, *optional*, defaults to 0.1): The dropout ratio for the attention probabilities. dropout (`float`, *optional*, defaults to 0.1): The dropout probability for all fully connected layers in the embeddings, encoder, and pooler. max_position_embeddings (`int`, *optional*, defaults to 512): The maximum sequence length that this model might ever be used with. Typically set this to something large just in case (e.g., 512 or 1024 or 2048). init_std (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. add_cross_attention (`bool`, *optional*, defaults to `True`): Whether cross-attention layers should be added to the model. is_encoder_decoder (`bool`, *optional*, defaults to `True`): Whether this is an encoder/decoder model. pad_token_id (`int`, *optional*, defaults to 1) Padding token id. bos_token_id (`int`, *optional*, defaults to 0) Beginning of stream token id. eos_token_id (`int`, *optional*, defaults to 2) End of stream token id. ngram (`int`, *optional*, defaults to 2) Number of future tokens to predict. Set to 1 to be same as traditional Language model to predict next first token. num_buckets (`int`, *optional*, defaults to 32) The number of buckets to use for each attention layer. This is for relative position calculation. See the [T5 paper](see https://arxiv.org/abs/1910.10683) for more details. relative_max_distance (`int`, *optional*, defaults to 128) Relative distances greater than this number will be put into the last same bucket. This is for relative position calculation. See the [T5 paper](see https://arxiv.org/abs/1910.10683) for more details. disable_ngram_loss (`bool`, *optional*, defaults to `False`): Whether be trained predicting only the next first token. eps (`float`, *optional*, defaults to 0.0): Controls the `epsilon` parameter value for label smoothing in the loss calculation. If set to 0, no label smoothing is performed. use_cache (`bool`, *optional*, defaults to `True`): Whether or not the model should return the last key/values attentions (not used by all models). prophetnetpast_key_valuesnum_attention_headsnum_encoder_attention_heads皙?gelu:w {Gz?Tr Factivation_dropoutactivation_function vocab_size hidden_sizeencoder_ffn_dimnum_encoder_layersdecoder_ffn_dimnum_decoder_layersnum_decoder_attention_headsattention_dropoutdropoutmax_position_embeddingsinit_stdis_encoder_decoderadd_cross_attentiondecoder_start_token_idngram num_bucketsrelative_max_distancedisable_ngram_losseps use_cache pad_token_id bos_token_id eos_token_idc `||_||_||_||_||_||_| |_| |_| |_||_ ||_ ||_ ||_ ||_ ||_||_| |_||_| |_||_t)jd||||||d|dS)N)r3r4r5r*r+r,)rr r!r"rr#r$r%r(r)rr-r.r/r0r1r&rr'r2super__init__)selfrrrr r!r"rr#r$r%r&r'r(r)r*r+r,r-r.r/r0r1r2r3r4r5kwargs __class__s s/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/models/prophetnet/configuration_prophetnet.pyr9zProphetNetConfig.__init__fs<%&."4+F(."4+F('>$  #6  &%:""4"3"4 " %%%1 3#9        returnc |j|jzS)N)r"r$)r:s r=num_hidden_layersz"ProphetNetConfig.num_hidden_layerss&)@@@r>c td)NzyThis model does not support the setting of `num_hidden_layers`. Please set `num_encoder_layers` and `num_decoder_layers`.)NotImplementedError)r:values r=rAz"ProphetNetConfig.num_hidden_layerss! %   r>)rrrrrrrrrrrrrrTTrrrrFrTrrr)__name__ __module__ __qualname____doc__ model_typekeys_to_ignore_at_inference attribute_maprfloatrstrrintboolr9propertyrAsetter __classcell__)r<s@r=r r sCCJJ#4"5<M /2>D$)%))-,.57)-,.57-0#&14$(-1.201 %'/2-2"$(&'&'&'7@ @ $UO@ &eCM&:;@ SM @ c] @ "# @ %SM@ &.c]@ "#@ %SM@ &.c]@ $E?@ %@ "*#@ 5/@ %TN!@ "&d^#@ $!) %@ &}'@ (c])@ * (}+@ ,%TN-@ .e_/@ 0D>1@ 2sm3@ 4sm5@ 6sm7@ @ @ @ @ @ DA3AAAXA       r>r N) rHtypingrrrconfiguration_utilsrutilsr get_loggerrEloggerr r7r>r=rXs%$,,,,,,,,,,333333  H % %W W W W W 'W W W W W r>