gSdZddlZddlZddlZddlmZmZmZmZddl Z ddl Z ddl m Z ddl m Z mZmZddlmZddlmZdd lmZmZmZmZdd lmZmZmZmZmZmZm Z dd l!m"Z"dd l#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+d dl,m-Z-e(rddl.m/Z/e*j0e1Z2dZ3dZ4gdZ5dZ6dZ7dZ8dZ9dZ:dZ;de j<de=de=fdZ>Gdde j?Z@Gdd e j?ZAGd!d"e jBZCGd#d$eCZDGd%d&eCZEeCeEeDd'ZFGd(d)e jBZGGd*d+e jBZHGd,d-e jBZIGd.d/e"ZJGd0d1eJZKGd2d3eJZLd4ZMd5ZNd6ZOGd7d8eJZPGd9d:eJZQe&d;eMGd<d=eJZRe&d>eMGd?d@eJeZSe&dAeMGdBdCeJZTe&dDeMGdEdFeJZUGdGdHeJZVe&dIeMGdJdKeJeZWdS)LzPyTorch BART model.N)ListOptionalTupleUnion)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)GenerationMixin)_prepare_4d_attention_mask#_prepare_4d_attention_mask_for_sdpa!_prepare_4d_causal_attention_mask*_prepare_4d_causal_attention_mask_for_sdpa)BaseModelOutput)BaseModelOutputWithPastAndCrossAttentions!CausalLMOutputWithCrossAttentionsSeq2SeqLMOutputSeq2SeqModelOutput#Seq2SeqQuestionAnsweringModelOutputSeq2SeqSequenceClassifierOutput)PreTrainedModel)add_code_sample_docstringsadd_end_docstringsadd_start_docstrings%add_start_docstrings_to_model_forwardis_flash_attn_2_available#is_flash_attn_greater_or_equal_2_10loggingreplace_return_docstrings) BartConfig)_flash_attention_forwardzfacebook/bart-baser#)r"izvalhalla/bart-large-sst2z 'POSITIVE'z%valhalla/bart-large-finetuned-squadv1gzG?z' nice puppet' input_ids pad_token_iddecoder_start_token_idc||j}|ddddf|ddddf<||dddf<|td||dk||S)z1 Shift input ids one token to the right. Nr"rz1self.model.config.pad_token_id has to be defined.i) new_zerosshapeclone ValueError masked_fill_)r'r(r)shifted_input_idss b/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/models/bart/modeling_bart.pyshift_tokens_rightr3Qs"++IO<<(CRC06688aaae4aaadLMMM""#4#z'BartLearnedPositionalEmbedding.__init__fs3  $+5}EEEEEr4rr'past_key_values_lengthc|jdd\}}tj|||ztj|jj|d}t||j zS)z3`input_ids' shape is expected to be [bsz x seqlen].Nr;)dtypedevicer+) r-torcharangelongweightrDexpandr=forwardr<)r?r'rAbszseq_len positionsr@s r2rJz&BartLearnedPositionalEmbedding.forwardlsv!rr* WL "$:W$DEJ_c_j_q   &b// wwy4;6777r4)r) __name__ __module__ __qualname____doc__intr>rETensorrJ __classcell__r@s@r2r6r6asFsF3FFFFFF 888s8888888888r4r6c \eZdZdZd dedededeeffd Zdej ffd Z xZ S) BartScaledWordEmbeddingz\ This module overrides nn.Embeddings' forward by multiplying with embeddings scale. ?r7r8 padding_idx embed_scalec\t|||||_dSN)r=r>rZ)r?r7r8rYrZr@s r2r>z BartScaledWordEmbedding.__init__|s-  DDD&r4r'cVt||jzSr\)r=rJrZ)r?r'r@s r2rJzBartScaledWordEmbedding.forwards!wwy))D,<<rErSrJrTrUs@r2rWrWws''s'3'S'_ghm_n''''''===========r4rWcheZdZdZ ddededed ed ed ed eeffd Z de j dedefdZ dde j dee j dee e j dee j dee j dede e j ee j ee e j ffdZxZS) BartAttentionz=Multi-headed attention from 'Attention Is All You Need' paperr&FTN embed_dim num_headsdropout is_decoderbias is_causalconfigc t||_||_||_||z|_||_|j|z|jkrtd|jd|d|jdz|_||_ ||_ tj ||||_ tj ||||_tj ||||_tj ||||_dS)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).g࿩re)r=r>rarbrchead_dimrgr/scalingrdrfrLineark_projv_projq_projout_proj) r?rarbrcrdrerfrgr@s r2r>zBartAttention.__init__s "" !Y.  MI %$. 8 83dn33%.333 }d* $"i 94@@@ i 94@@@ i 94@@@  )YTBBB r4tensorrLrKc||||j|jddS)Nr"r;)viewrbrj transpose contiguousr?rqrLrKs r2_shapezBartAttention._shapes<{{3GGQQRSUVWWbbdddr4 hidden_stateskey_value_statespast_key_valueattention_masklayer_head_maskoutput_attentionsreturncZ |du}|\}} } |||jz} |r6|4|djd|jdkr|d} |d} n>|rU|||d|} |||d|} n||||d|} |||d|} tj|d| gd} tj|d| gd} nT|||d|} |||d|} |j r| | f}||j zd|j f}|| | |j |} | j |} | j |} | d}tj| | dd}|||j z| |fkr2t!d||j z| |fd||||d| |fkr+t!d |d| |fd|| ||j | ||z}| ||j z| |}t"j|d}|||j fkr-t!d |j fd|| dddd| ||j | |z}| ||j z| |}|r=| ||j | |}| ||j z| |}nd}t"j||j|j }tj|| }|||j z| |j fkr7t!d ||j z| |j fd|| ||j | |j }|dd}| || |j}||}|||fS) #Input shape: Batch x Time x ChannelNrr;r"r+dimz$Attention weights should be of size , but is z!Attention mask should be of size z/Head mask for a single layer should be of size ptraining `attn_output` should be of size )sizerorkr-rwrmrnrEcatrdrbrjrsreshapebmmrtr/r functionalsoftmaxrcrrarp)r?rxryrzr{r|r}is_cross_attentionrKtgt_len_ query_states key_states value_states proj_shapesrc_len attn_weightsattn_weights_reshaped attn_probs attn_outputs r2rJzBartAttention.forwards.T9',,..Wa{{=11DL@  L*q!'*.>.DQ.GGG(*J)!,LL  LT[[1A%B%BBLLJ;;t{{3C'D'Db#NNLL  'T[[%?%?SIIJ;;t{{='A'A2sKKLN1$5z#BJJJJ 9nQ&7%FANNNLLT[[%?%?SIIJ;;t{{='A'A2sKKL ? 8),7NDN*B > Ct{{<#>>CZP 'Z'4 +|+Z8 //!$$yz/C/CAq/I/IJJ     3#7'"J J J*dn8LgW^7_** %%''**   %""$$a'(BBB ta'8Rtt]k]p]p]r]rtt(,,S$.'7SSVddL',,S4>-A7GTTL},,\r,BB  &##%%$.)::: 1t~FW11',,..11+//2q!<<|?P?PQTVZVdfmov?w?wwL',,S4>-A7GTTL  ) %1$5$5c4>7T[$\$\ !055cDN6JGU\]]LL$( !]**<4AAr4)r&FTFNNNNNF)rNrOrPrQrRr^boolrr#r>rErSrwrrJrTrUs@r2r`r`sGG  '+CCCC C  C  CC$CCCCCC>eU\eCeceeee 488<1526"'vBvB|vB#5<0vB!u|!45 vB !. vB "%,/ vB vB u|Xel3XeEL>Q5RR SvBvBvBvBvBvBvBvBr4r`c2eZdZdZfdZdejdedefdZ dd ejd e ejd e e ejd e ejd e ejde de eje eje e ejffdZ xZ S)BartFlashAttention2aD Bart flash attention module. This module inherits from `BartAttention` as the weights of the module stays untouched. The only required change would be on the forward pass where it needs to correctly call the public API of flash attention and deal with padding tokens in case the input contains any of them. cbtj|i|t |_dSr\)r=r>r_flash_attn_uses_top_left_mask)r?argskwargsr@s r2r>zBartFlashAttention2.__init__*s9$)&))) 3V2W2W.W+++r4rqrLrKcF||||j|jSr\)rsrbrjrvs r2_reshapezBartFlashAttention2._reshape2s{{3GGGr4NFrxryrzr{r|r}r~c R|rtd|du}|\}} } |||d|} |r^|\|djd|jdkr:|ddd} |ddd} ng|rV|||d|} |||d|} n||||d|} |||d|} tj |ddd| gd} tj |ddd| gd} nT|||d|} |||d|} |j r,| dd| ddf}| jd}|||djdz }| j }|tj krtj rtj}n3t|jdr |jj}n|jjj }t&d |d | |} | |} | |} t-| | | || |jr|jnd |j|j }||| d}||}|sd}|||fS) Nz@BartFlashAttention2 attention does not support output_attentionsr+rr;r"r_pre_quantization_dtypezThe input hidden states seems to be silently casted in float32, this might be related to the fact you have upcasted embedding or layer norm layers in float32. We will cast back the input in .r&)rcrfuse_top_left_mask)r/rrror-rtrmrnrErrdrCfloat32is_autocast_enabledget_autocast_gpu_dtypehasattrrgrrHlogger warning_oncetor$rrcrfrrrp)r?rxryrzr{r|r}rrKq_lenrrrr kv_seq_len input_dtype target_dtyperrs r2rJzBartFlashAttention2.forward5s  a_`` `.T9%**,, UA}}T[[%?%?SII  N*q!'*.>.DQ.GGG(*44Q::J)!,66q!<rErSrRrrrrrJrTrUs@r2rr"s;XXXXXHu|HcHHHHH 488<1526"'i9i9|i9#5<0i9!u|!45 i9 !. i9 "%,/ i9 i9 u|Xel3XeEL>Q5RR Si9i9i9i9i9i9i9i9r4rc eZdZ d dejdeejdeeejdeejdeejded eejeejeeejfffd ZxZ S) BartSdpaAttentionNFrxryrzr{r|r}r~c|s|Atdt||||||S|du}|\}} } ||} |r6|4|djd|jdkr|d} |d} n>|rU|||d|} || |d|} n||||d|} || |d|} tj |d| gd} tj |d| gd} nT|||d|} || |d|} |j r| | f}|| | |} |j r || dkrd nd }tjj| | | ||jr|jnd | }|||j| |jfkr5t+d ||j| |jfd||dd}||| |j}||}|d|fS)rNaBartModel is using BartSdpaAttention, but `torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True` or `layer_head_mask` not None. Falling back to the manual attention implementation, but specifying the manual implementation will be required from Transformers version v5.0.0 onwards. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.)ryrzr{r|r}rr;r"r+rTFr&) attn_mask dropout_prfrr)rrr=rJrror-rwrmrnrErrdrfrrscaled_dot_product_attentionrrcrbrjr/rtrrarp)r?rxryrzr{r|r}rrKrrrrrrfrr@s r2rJzBartSdpaAttention.forwardsK   ;   l   77??!1-- /"3 # .T9',,..Wa{{=11  L*q!'*.>.DQ.GGG(*J)!,LL  LT[[1A%B%BBLLJ;;t{{3C'D'Db#NNLL  'T[[%?%?SIIJ;;t{{='A'A2sKKLN1$5z#BJJJJ 9nQ&7%FANNNLLT[[%?%?SIIJ;;t{{='A'A2sKKL ? 8),7N{{<#>> !N`~/E'TU++DD[` h)FF   $&*mQ5RR Sf1f1f1f1f1f1f1f1f1f1r4r)eagersdpaflash_attention_2ceZdZdeffd Z d dejdejdejdeede ejeejff d Z xZ S) BartEncoderLayerrgc*t|j|_t |j|j|j|j||_tj |j|_ |j |_ t|j|_|j|_tj|j|j|_tj|j|j|_tj |j|_dS)N)rarbrcrg)r=r>d_modelraBART_ATTENTION_CLASSES_attn_implementationencoder_attention_headsattention_dropout self_attnr LayerNormself_attn_layer_normrcr activation_function activation_fnactivation_dropoutrlencoder_ffn_dimfc1fc2final_layer_normr?rgr@s r2r>zBartEncoderLayer.__init__s /0KLn4,     %'L$@$@!~ #F$>?"(";9T^V-CDD9V3T^DD " T^ < <r4Frxr{r|r}r~c||}|||||\}}}tj||j|j}||z}||}|}|||}tj||j|j}| |}tj||j|j}||z}| |}|j tj krtj|s&tj|r9tj|j jdz }tj|| |}|f} |r| |fz } | S)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. layer_head_mask (`torch.FloatTensor`): mask for attention heads in a given layer of size `(encoder_attention_heads,)`. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. )rxr{r|r}ri)minmax)rrrrcrrrrrrrrCrEfloat16isinfanyisnanfinforclamp) r?rxr{r|r}residualrr clamp_valueoutputss r2rJzBartEncoderLayer.forward%s$!)-')+/ *8* * & |Q  --mt|VZVc-dd  =0 11-@@  **488M+B+BCC  --mt?Vaean-oo //  --mt|VZVc-dd  =0 --m<<  %- / / K & & * * , , 005 M0J0J0N0N0P0P 0 +m&9::>EK!K K<[YYYM "  '   &Gr4)F) rNrOrPr#r>rE FloatTensorrrrrJrTrUs@r2rrs=z======.-2 00(0)0* 0 $D> 0 u (5+<"== > 00000000r4rcbeZdZdeffd Z ddejdeejdeejd eejd eejd eejd eeejd ee dee deej eeej ej fffdZ xZ S)BartDecoderLayerrgct|j|_t |j|j|j|jdd||_|j |_ t|j |_ |j |_ tj|j|_t |j|j|j|jd||_tj|j|_tj|j|j|_tj|j|j|_tj|j|_dS)NT)rarbrcrdrfrg)rcrdrg)r=r>rrarrdecoder_attention_headsrrrcr rrrrrr encoder_attnencoder_attn_layer_normrldecoder_ffn_dimrrrrs r2r>zBartDecoderLayer.__init__Ys* /0KLn4,    ~ #F$>?"(";$&L$@$@!263NO N  *,    (*|DN'C'C$9T^V-CDD9V3T^DD " T^ < <r4NFTrxr{encoder_hidden_statesencoder_attention_maskr|cross_attn_layer_head_maskrzr} use_cacher~c x|} | |ddnd} ||| |||\}} } tj||j|j}| |z}||}d}d}|z|} | |ddnd}|||||||\}}}tj||j|j}| |z}||}| |z} |} || |}tj||j |j}| |}tj||j|j}| |z}| |}|f}|r|| |fz }| r|| fz }|S)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. encoder_hidden_states (`torch.FloatTensor`): cross attention input to the layer of shape `(batch, seq_len, embed_dim)` encoder_attention_mask (`torch.FloatTensor`): encoder attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. layer_head_mask (`torch.FloatTensor`): mask for attention heads in a given layer of size `(encoder_attention_heads,)`. cross_attn_layer_head_mask (`torch.FloatTensor`): mask for cross-attention heads in a given layer of size `(decoder_attention_heads,)`. past_key_value (`Tuple(torch.FloatTensor)`): cached past key and value projection states output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. Nr;)rxrzr{r|r}rr)rxryr{r|rzr}) rrrrcrrrrrrrrr)r?rxr{rrr|rrzr}rrself_attn_past_key_valueself_attn_weightspresent_key_valuecross_attn_present_key_valuecross_attn_weightscross_attn_past_key_valuers r2rJzBartDecoderLayer.forwardvs,<!:H9S>"1"#5#5Y] >Bnn'3)+/ ?M? ? ; (*; --mt|VZVc-dd  =0 11-@@ (,$! ,$H@N?Yrss(;(;_c %NRN_N_+!65 :8"3 O`OO KM-/KM11-4 >G  , )+ +Gr4)NNNNNNFT) rNrOrPr#r>rErSrrrrrJrTrUs@r2rrXsH=z======@268<9=26=A8<,1$(WW|W!.W ( 5 W !) 6 W "%,/ W%-U\$:W!u|!45W$D>WD>W u (51BEDU1U+V"WW XWWWWWWWWr4rcXeZdZdZdedededeffd Zdejdejfd Z xZ S) BartClassificationHeadz-Head for sentence-level classification tasks. input_dim inner_dim num_classespooler_dropoutcttj|||_tj||_tj|||_dS)N)r)r=r>rrldenseDropoutrcrp)r?rrrrr@s r2r>zBartClassificationHead.__init__sY Yy)44 zN333  )[99 r4rxr~c||}||}tj|}||}||}|Sr\)rcrrEtanhrp)r?rxs r2rJzBartClassificationHead.forwards[ ]33  =11  =11  ]33  m44 r4) rNrOrPrQrRr^r>rErSrJrTrUs@r2rrs77 : : : :  : : : : : :U\elr4rcReZdZeZdZdZddgZddgZdZ dZ dZ dZ e d Zd S) BartPreTrainedModelmodelTzencoder.versionzdecoder.versionrrpast_key_valuesc|jj}t|tjrJ|jjd||j |jj dSdSt|tj rS|jjd||j -|jj|j  dSdSdS)Nr&)meanstd) rginit_std isinstancerrlrHdatanormal_rezero_ EmbeddingrY)r?modulers r2 _init_weightsz!BartPreTrainedModel._init_weightssk" fbi ( ( ? M  & &CS & 9 9 9{&  &&((((('&  - - ? M  & &CS & 9 9 9!- "6#56<<>>>>> ? ?--r4c|jj}tjgddddd|gg|j}|||d}|S)N)r r;rr% r;rD)r{r')rgr(rErqrDne)r? pad_tokenr' dummy_inputss r2rz BartPreTrainedModel.dummy_inputssaK, L"2"2"2Q2q)4L!MVZVabbb 'll955"  r4N)rNrOrPr# config_classbase_model_prefixsupports_gradient_checkpointing"_keys_to_ignore_on_load_unexpected_no_split_modules_skip_keys_device_placement_supports_flash_attn_2_supports_sdparpropertyrr4r2rrsxL&*#*;=N)O&,.AB"3!N ? ? ?Xr4rceZdZdZdS)PretrainedBartModelc:tjdtdSNz_The class `PretrainedBartModel` has been depreciated, please use `BartPreTrainedModel` instead.warningswarn FutureWarningr?s r2__init_subclass__z%PretrainedBartModel.__init_subclass__ & m      r4NrNrOrPr%rr4r2rr#     r4rceZdZdZdS)BartPretrainedModelc:tjdtdSrr r$s r2r%z%BartPretrainedModel.__init_subclass__r&r4Nr'rr4r2r*r*r(r4r*aI This model inherits from [`PreTrainedModel`]. Check the superclass documentation for the generic methods the library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads etc.) This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. Parameters: config ([`BartConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. a! Summarization example: ```python >>> from transformers import AutoTokenizer, BartForConditionalGeneration >>> model = BartForConditionalGeneration.from_pretrained("facebook/bart-large-cnn") >>> tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large-cnn") >>> ARTICLE_TO_SUMMARIZE = ( ... "PG&E stated it scheduled the blackouts in response to forecasts for high winds " ... "amid dry conditions. The aim is to reduce the risk of wildfires. Nearly 800 thousand customers were " ... "scheduled to be affected by the shutoffs which were expected to last through at least midday tomorrow." ... ) >>> inputs = tokenizer([ARTICLE_TO_SUMMARIZE], max_length=1024, return_tensors="pt") >>> # Generate Summary >>> summary_ids = model.generate(inputs["input_ids"], num_beams=2, min_length=0, max_length=20) >>> tokenizer.batch_decode(summary_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] 'PG&E scheduled the blackouts in response to forecasts for high winds amid dry conditions' ``` Mask filling example: ```python >>> from transformers import AutoTokenizer, BartForConditionalGeneration >>> tokenizer = AutoTokenizer.from_pretrained("facebook/bart-base") >>> model = BartForConditionalGeneration.from_pretrained("facebook/bart-base") >>> TXT = "My friends are but they eat too many carbs." >>> input_ids = tokenizer([TXT], return_tensors="pt")["input_ids"] >>> logits = model(input_ids).logits >>> masked_index = (input_ids[0] == tokenizer.mask_token_id).nonzero().item() >>> probs = logits[0, masked_index].softmax(dim=0) >>> values, predictions = probs.topk(5) >>> tokenizer.decode(predictions).split() ['not', 'good', 'healthy', 'great', 'very'] ``` a Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) Bart uses the `eos_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). For translation and summarization training, `decoder_input_ids` should be provided. If no `decoder_input_ids` is provided, the model will create this tensor by shifting the `input_ids` to the right for denoising pre-training following the paper. decoder_attention_mask (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. If you want to change padding behavior, you should read [`modeling_bart._prepare_decoder_attention_mask`] and modify to your needs. See diagram 1 in [the paper](https://arxiv.org/abs/1910.13461) for more information on the default strategy. head_mask (`torch.Tensor` of shape `(encoder_layers, encoder_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules in the encoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. decoder_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. encoder_outputs (`tuple(tuple(torch.FloatTensor)`, *optional*): Tuple consists of (`last_hidden_state`, *optional*: `hidden_states`, *optional*: `attentions`) `last_hidden_state` of shape `(batch_size, sequence_length, hidden_size)`, *optional*) is a sequence of hidden-states at the output of the last layer of the encoder. Used in the cross-attention of the decoder. past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) and 2 additional tensors of shape `(batch_size, num_heads, encoder_sequence_length, embed_size_per_head)`. Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. decoder_inputs_embeds (`torch.FloatTensor` of shape `(batch_size, target_sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `decoder_input_ids` you can choose to directly pass an embedded representation. If `past_key_values` is used, optionally only the last `decoder_inputs_embeds` have to be input (see `past_key_values`). This is useful if you want more control over how to convert `decoder_input_ids` indices into associated vectors than the model's internal embedding lookup matrix. If `decoder_input_ids` and `decoder_inputs_embeds` are both unset, `decoder_inputs_embeds` takes the value of `inputs_embeds`. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. ceZdZdZddedeejffd ZdZ dZ dde j d ee j d ee j d ee jd eed eedeedeeeffdZxZS) BartEncoderz Transformer encoder consisting of *config.encoder_layers* self attention layers. Each layer is a [`BartEncoderLayer`]. Args: config: BartConfig embed_tokens (nn.Embedding): output embedding Nrg embed_tokensctj|_j|_j}j|_j|_ j rtj |nd}tj||j||_||j|j_t#j||_t'jfdt+jD|_jdk|_jdk|_t'j||_d|_|dS)NrXrZc.g|]}tSr)r.0rrgs r2 z(BartEncoder.__init__.."$d$d$d!%5f%=%=$d$d$dr4rrF)r=r>rcencoder_layerdrop layerdroprr(rYmax_position_embeddingsmax_source_positionsscale_embeddingmathsqrtrW vocab_sizer.rHr6embed_positionsr ModuleListrangeencoder_layerslayersr_use_flash_attention_2 _use_sdparlayernorm_embeddinggradient_checkpointing post_init)r?rgr.rarZr@s ` r2r>zBartEncoder.__init__sB    ~ 1N !.$*$B!.4.DMdi ***# 3  y$*:      #'3':D  $=  *   m$d$d$d$duVMbGcGc$d$d$dee &,&AEX&X#4>#%< #:#: &+# r4c|jSr\r.r$s r2get_input_embeddingsz BartEncoder.get_input_embeddings   r4c||_dSr\rIr?values r2set_input_embeddingsz BartEncoder.set_input_embeddings!r4r'r{ head_mask inputs_embedsr}output_hidden_states return_dictr~c||n |jj}||n |jj}||n |jj}||t d|$|}|d|jd}n!||dddddf}nt d|||}||} | |j } || z} | | } tj | |j|j} |F|jr d|vr|nd}n6|jr||st%||j}nt)||j}|rdnd} |rdnd} |p|dt-|jkr@t dt-|jd |dd t1|jD]\} }|r| | fz} d }|jr!t3jg}||jkrd }|rd }nX|jr0|jr)||j| |||| nd|}n|| |||| nd|}|d} |r | |dfz} |r| | fz} |st?d| | | fDStA| | | S)a~ Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(encoder_layers, encoder_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. NzDYou cannot specify both input_ids and inputs_embeds at the same timer+z5You have to specify either input_ids or inputs_embedsrrrz&The head_mask should be specified for  layers, but it is for rFT)NN)r|r}r"c3K|]}||V dSr\rr3vs r2 z&BartEncoder.forward..ks(eeqWXWdWdWdWdWdeer4last_hidden_staterx attentions)!rgr}rSuse_return_dictr/rsr-r.r>rrDrErrrcrrCrDrrCrrlenrB enumeraterErandr7rF_gradient_checkpointing_func__call__tupler)r?r'r{rQrRr}rSrTinput embed_posrxencoder_statesall_attentionsidx encoder_layerto_dropdropout_probability layer_outputss r2rJzBartEncoder.forwards\2C1N--TXT_Tq$8$D $+Jj &1%cdd d  "E!r9?2+>??II  &!!!!QQQ(+EETUU U   --i88M((// LL!566 % 1 00??  --mt|VZVc-dd   %* a343F3FD aI$5>O$5"E^UbUh!i!i">N fee]NN$Seeeee e+>Vd    r4r\)NNNNNNN)rNrOrPrQr#rrrr>rJrOrE LongTensorrSrrrrrrJrTrUs@r2r-r-s=z",9O>!!!""" '+15,059,0/3&*J J #J !.J EL) J   12 J $D> J 'tnJ d^J  uo% &J J J J J J J J r4r-ceZdZdZddedeejffd ZdZ dZ dde j d ee j d ee jd ee j d ee j d ee j deee jdee jdeedeedeedeedeeeffdZxZS) BartDecoderz Transformer decoder consisting of *config.decoder_layers* layers. Each layer is a [`BartDecoderLayer`] Args: config: BartConfig embed_tokens (nn.Embedding): output embedding Nrgr.ctj|_j|_j|_j|_j rtj j nd}tjj |j||_||j|j_t#jj |_t'jfdt+jD|_jdk|_jdk|_t'jj |_d|_|dS)NrXr0c.g|]}tSr)rr2s r2r4z(BartDecoder.__init__..r5r4rrF)r=r>rcdecoder_layerdropr7r(rYr8max_target_positionsr:r;r<rrWr=r.rHr6r>rr?r@decoder_layersrBrrCrDrrErFrG)r?rgr.rZr@s ` r2r>zBartDecoder.__init__zsC    ~ 1!.$*$B!393IRdi///s 3  v~t/?[     #'3':D  $=  * N  m$d$d$d$duVMbGcGc$d$d$dee &,&AEX&X#4>#%<#?#? &+# r4c|jSr\rIr$s r2rJz BartDecoder.get_input_embeddingsrKr4c||_dSr\rIrMs r2rOz BartDecoder.set_input_embeddingsrPr4r'r{rrrQcross_attn_head_maskrrRrr}rSrTr~c | | n |jj} | | n |jj} | | n |jj} | | n |jj} ||t d|&|} | j}|d|d}n=|,|dd}|dddddf} nt d||ddjdnd}|| | }|j r |d|vr|nd}n0|j r| s|t||||}nt||||}|X|V|j r d|vr|nd}nF|j r"| | st||j|d}nt!||j|d}|| |}||j}||z}||}t*j||j|j}|jr%|jr| rt4d d } | rd nd}| rd nd}| r|d nd}| rd nd}t9||gd d gD]z\}}|s|dt;|jkrCt d|dt;|jd|dd{t?|jD]\}}| r||fz }|jr tAj!g}||j"kr4|||nd}|jr?|jr8|#|j$|||||||nd|||ndd| | }n(||||||||nd|||nd|| |  }|d}| r||| rdndfz }| r||dfz }| ||dfz }| r||fz }| r|nd}| stKd|||||fDStM|||||S)a  Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) encoder_hidden_states (`torch.FloatTensor` of shape `(batch_size, encoder_sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder. Used in the cross-attention of the decoder. encoder_attention_mask (`torch.LongTensor` of shape `(batch_size, encoder_sequence_length)`, *optional*): Mask to avoid performing cross-attention on padding tokens indices of encoder input_ids. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder to avoid performing cross-attention on hidden heads. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) and 2 additional tensors of shape `(batch_size, num_heads, encoder_sequence_length, embed_size_per_head)`. Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. NzTYou cannot specify both decoder_input_ids and decoder_inputs_embeds at the same timer+zEYou have to specify either decoder_input_ids or decoder_inputs_embedsrr;)rrzZ`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...FrrQrxzThe `z` should be specified for rVr)r{rrr|rrzr}rr r"c3K|]}||V dSr\rrXs r2rZz&BartDecoder.forward..s0= ===r4)r\rrxr]cross_attentions)'rgr}rSrr^r/r-rsrr.rCrDrrrrCrr>rrDrErrrcrrFrrzipr_rBr`rErar7rbrcrdr)r?r'r{rrrQrxrrRrr}rSrTre input_shaperArMrxall_hidden_statesall_self_attnsall_cross_attentionsnext_decoder_cacher mask_nameri decoder_layerrlrzrm next_caches r2rJzBartDecoder.forwards`2C1N--TXT_Tq$8$D $+Jj "+!6IIDKstt t  "E+K!r;r?;;II  &',,..ss3K!!!!QQQ(+EEdee eDSC^!3A!6!0JqTbObOb^^imNN ^ $5 :N:VH& NN? ]>##A&3t{+;+;<<$3 33SEUEU33%NN,,Q/333 #,DK"8"8/ @/ @ C# 6!m%55!} &+jnn#&775D5P_S11VZN* t}  $ A A!*!")*&/&;IcNN1E1Q(--W[% ! ! !. !#1*?+A7@7LYs^^RV5I5U,S11[_#1&7' ! ! ! *!,M V"}:K5RQQQR'S&UU"  @=#3"55(4(]1-=,??(  2 -!1 1 +4>''$  '5FXlm  9+&+%1     r4r\) NNNNNNNNNNNN)rNrOrPrQr#rrrr>rJrOrErnrSrrrrrrrJrTrUs@r2rprpqsz",9O:!!!""" '+15=A=A,07;=A59$(,0/3&*p p #p !.p  ((9: p !))9 : p EL) p 'u|4p "$u'8"9:p   12p D>p $D>p 'tnp d^p  u?? @p p p p p p p p r4rpzRThe bare BART Model outputting raw hidden-states without any specific head on top.c%<eZdZddgZdeffd ZdZdZdZdZ d Z e e e eeee  dd ejd eejdeejdeejdeejdeejdeejdeeejdeeejdeejdeejdeedeedeedeedeeeff dZxZS) BartModelencoder.embed_tokens.weightdecoder.embed_tokens.weightrgctt||j|j}}|jrt j|jnd}t||j|||_ t||j |_ t||j |_ |dS)NrXr0)r=r>r(r=r:r;r<rrWsharedr-encoderrpdecoderrG)r?rgrYr=rZr@s r2r>zBartModel.__init__s    "("5v7HZ 393IRdi///s -j&.+cnooo "64;77 "64;77  r4c|jjrL||jj|j||jj|jdSdSr\)rgtie_word_embeddings_tie_or_clone_weightsrr.rrr$s r2 _tie_weightszBartModel._tie_weightss\ ; * O  & &t|'@$+ N N N  & &t|'@$+ N N N N N O Or4c|jSr\)rr$s r2rJzBartModel.get_input_embeddingss {r4cX||_|j|j_|j|j_dSr\)rrr.rrMs r2rOzBartModel.set_input_embeddingss' $(K !$(K !!!r4c|jSr\)rr$s r2 get_encoderzBartModel.get_encoder |r4c|jSr\rr$s r2 get_decoderzBartModel.get_decoderrr4) checkpoint output_typerexpected_outputNr'r{decoder_input_idsdecoder_attention_maskrQdecoder_head_maskrxencoder_outputsrrRdecoder_inputs_embedsrr}rSrTr~c|8| 6|tdt||jj|jj}| | n |jj} ||n |jj}| | n |jj} ||n |jj}|| |||| | ||}ne|rct|tsNt|dt|dkr|dndt|dkr|dnd}| |||d|||| | | | || }|s||zSt|j|j|j|j|j|j|j|jS) NzIf no `decoder_input_ids` or `decoder_inputs_embeds` are passed, `input_ids` cannot be `None`. Please pass either `input_ids` or `decoder_input_ids` or `decoder_inputs_embeds`.)r'r{rQrRr}rSrTrr"r;r[ r'r{rrrQrxrrRrr}rSrT)r\rdecoder_hidden_statesdecoder_attentionsr{encoder_last_hidden_staterencoder_attentions)r/r3rgr(r)r}rSrr^rrrr_rrr\rrxr]r{)r?r'r{rrrQrrxrrrRrrr}rSrTdecoder_outputss r2rJzBartModel.forwards6  $)>)F  U !34;3T[5W!! 2C1N--TXT_Tq$8$D $+Jj "+!6IIDKrrJrOrrrBART_INPUTS_DOCSTRINGr_CHECKPOINT_FOR_DOCr_CONFIG_FOR_DOC_EXPECTED_OUTPUT_SHAPErErnrrSrrrrrrJrTrUs@r2rrsX 89VW z      OOO 000 +*+@AA&&$. '+158<=A,0487;=A=A59=A$(,0/3&*!U U #U !.U $E$45 U !))9 : U EL) U $EL1U 'u|4U "$u'8"9:U "$u'8"9:U   12U  ((9:U D>U $D>U 'tnU d^!U " u(( )#U U U BAU U U U U r4rzLThe BART Model with a language modeling head. Can be used for summarization.c(eZdZdZgdZdgZdeffd ZdZdZ d$d e d e e d e j ffd Zd e d dfd ZdZdZeeeeeee d%dejde ejde ejde ejde ejde ejde ejde eejde eejde ejde ejde ejde ede ede ed e ed ee eff"d!Z!dejfd"Z"e#d#Z$xZ%S)&BartForConditionalGenerationr)rrlm_head.weightfinal_logits_biasrgclt|t||_|dt jd|jjjftj |j |jjjd|_ | dS)Nrr"Fri)r=r>rrregister_bufferrEzerosrr7rrlrlm_headrGrs r2r>z%BartForConditionalGeneration.__init__s    v&&  0%+q$*BSBb>c2d2deeey1B1QX]^^^  r4c4|jSr\)rrr$s r2rz(BartForConditionalGeneration.get_encoder'z%%'''r4c4|jSr\)rrr$s r2rz(BartForConditionalGeneration.get_decoder*rr4Nnew_num_tokenspad_to_multiple_ofr~ct||}||jjd|S)Nr)r=resize_token_embeddings_resize_final_logits_biasrHr-)r?rrnew_embeddingsr@s r2rz4BartForConditionalGeneration.resize_token_embeddings-sB88I[\\ &&~'<'B1'EFFFr4c|jjd}||kr|jddd|f}nBtjd||z f|jj}tj|j|gd}|d|dS)Nr+r"rrr)rr-rErrDrr)r?rold_num_tokensnew_bias extra_biass r2rz6BartForConditionalGeneration._resize_final_logits_bias2s/5b9 ^ + +-aaa..@AHHa.)H%IRVRhRopppJy$"8*!E1MMMH 0(;;;;;r4c|jSr\rr$s r2get_output_embeddingsz2BartForConditionalGeneration.get_output_embeddings;rr4c||_dSr\rr?rs r2set_output_embeddingsz2BartForConditionalGeneration.set_output_embeddings> % r4rrr'r{rrrQrrxrrrRrlabelsrr}rSrTc||n |jj}| G| rtdd} |'| %t | |jj|jj}|||||||||| | | | |||}||d}||j |j z}d}| e| |j } t}|| d|jj| d}|s|f|ddz}||f|zn|St|||j|j|j|j|j|j|j S) a labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Returns: NzJThe `use_cache` argument is changed to `False` since `labels` is provided.F)r{rrrrQrrxrrRrrr}rSrTrr+r" losslogitsrrrr{rrr)rgr^rwarningr3r(r)rrrrrDr rsr=rrrrr{rrr)r?r'r{rrrQrrxrrrRrrrr}rSrTr lm_logitsmasked_lm_lossloss_fctoutputs r2rJz$BartForConditionalGeneration.forwardAs<&1%BartForConditionalGeneration._reorder_cache..sCrrU_j--aZ=N1O1OPPrrrrrrr4r;rdrrreordered_past layer_pasts ` r2_reorder_cachez+BartForConditionalGeneration._reorder_cachesm)  J rrrrcmnpopnpcqrrrrrQRR.! NNr4r\NNNNNNNNNNNNNNNN)&rNrOrPrr_keys_to_ignore_on_load_missingr#r>rrrRrrrrrrrrrr!rrrBART_GENERATION_EXAMPLErErnrSrrrrrrJr staticmethodrrTrUs@r2rrs iii':&;#z((((((cxX[}hjht <<<<<<&&&+*+@AA?YYY/00'+158<=A,0487;=A=A59=A-1$(,0/3&*#O O #O !.O $E$45 O !))9 : O EL) O $EL1O 'u|4O "$u'8"9:O "$u'8"9:O   12O  ((9:O )*O D>O $D>O 'tn!O "d^#O $ uo% &%O O O 10ZYBAO bhELhhhh\r4rz Bart model with a sequence classification/head on top (a linear layer on top of the pooled output) e.g. for GLUE tasks. c%eZdZddgZdeffd Zeeee e e e e  ddejdeejd eejd eejd eejd eejd eejdeeejdeejdeejdeejdeedeedeedeedeee ff dZxZS)BartForSequenceClassificationrrrgc tj|fi|t||_t |j|j|j|j|_| dSr\) r=r>rrrr num_labelsclassifier_dropoutclassification_headrG)r?rgrr@s r2r>z&BartForSequenceClassification.__init__sq**6***v&& #9 N N    % $ $   r4)rrrr expected_lossNr'r{rrrQrrxrrRrrrr}rSrTr~cR||n |jj}| d} || td|jj|||||||||| | | | ||}|d}||jj|j }ttj | ddkrtd||ddf|dd|ddddddf}||}d}| | |j } |jjp|jjdkr d |j_nS|jjdkr7| jtjks| jtjkr d |j_n d |j_|jjd kr\t/}|jjdkr1||| }n||| }n|jjd krLt3}||d|jj| d}n*|jjd krt5}||| }|s|f|ddz}||f|zn|St7|||j|j|j|j|j |j!|j" S) a3 labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). NFz8Passing input embeddings is currently not supported for r{rrrQrrxrrRrrr}rSrTrr"z7All examples must have the same number of tokens.r+ regressionsingle_label_classificationmulti_label_classificationr)#rgr^NotImplementedErrorr@rNreq eos_token_idrrDr_rEunique_consecutivesumr/rsrr problem_typerrCrGrRr squeezer rrrrrr{rrr)r?r'r{rrrQrrxrrRrrrr}rSrTrrxeos_masksentence_representationrrrrs r2rJz%BartForSequenceClassification.forwardsR>&1%Kbdd ** )/#9/!5+'"7/!5#     << 899<<]=QRR u' Q88 9 9A = =VWW W"/!!! "<"A"A-BTBTUVBWBWY[]j]o]opr]s]s"t"t AAr111H# ))*ABB  YYv}--F{'/;)Q../;DK,,[+a//V\UZ5O5OSYS_chclSlSl/LDK,,/KDK,{'<77"99;)Q..#8FNN$4$4fnn6F6FGGDD#8FF33DD)-JJJ+--x B 0F G GUWYY)-III,..x// FY,F)-)9TGf$$v E.#3")"?&9$5&-&G")"?&9    r4r)rNrOrPrr#r>rrr'_CHECKPOINT_FOR_SEQUENCE_CLASSIFICATIONrr_SEQ_CLASS_EXPECTED_OUTPUT_SEQ_CLASS_EXPECTED_LOSSrErnrrSrrrrrrJrTrUs@r2rrs89VW z      +*+@AA:3$2. '+158<=A,0487;=A59=A-1$(,0/3&*!` ` #` !.` $E$45 ` !))9 : ` EL) ` $EL1` 'u|4` "$u'8"9:`   12`  ((9:` )*` D>` $D>` 'tn` d^!` " u55 6#` ` ` BA` ` ` ` ` r4rz BART Model with a span classification head on top for extractive question-answering tasks like SQuAD (a linear layer on top of the hidden-states output to compute `span start logits` and `span end logits`). c'*eZdZddgZfdZeeeee e e e  dde jdee jdee jd ee jd ee jd ee jd ee jd eee jdee jdee jdee jdee jdeedeedeedeedeee ff"dZxZS)BartForQuestionAnsweringrrct|d|_|j|_t||_t j|j|j|_| dSr:) r=r>rrrrrl hidden_size qa_outputsrGrs r2r>z!BartForQuestionAnswering.__init__0sm     +v&& )F$68IJJ r4)rrrrrNr'r{rrrQrrxrstart_positions end_positionsrRrrr}rSrTr~c||n |jj}| | d} |||||||||| | | |||}|d}||}|dd\}}|d}|d}d}| | t| dkr| d} t| dkr| d} |d}| d|} | d|} t|}||| }||| }||zd z }|s||f|ddz}||f|zn|St||||j |j |j|j|j|j|j S) a start_positions (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for position (index) of the start of the labelled span for computing the token classification loss. Positions are clamped to the length of the sequence (*sequence_length*). Position outside of the sequence are not taken into account for computing the loss. end_positions (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for position (index) of the end of the labelled span for computing the token classification loss. Positions are clamped to the length of the sequence (*sequence_length*). Position outside of the sequence are not taken into account for computing the loss. NFrrr"r+r) ignore_indexr;) r start_logits end_logitsrrrr{rrr)rgr^rrsplitrrur_rrr rrrrr{rrr)r?r'r{rrrQrrxrrrrRrrr}rSrTrsequence_outputrr r  total_loss ignored_indexr start_lossend_lossrs r2rJz BartForQuestionAnswering.forward<s[J&1%rrr_CHECKPOINT_FOR_QArr_QA_EXPECTED_LOSS_QA_EXPECTED_OUTPUTrErSrrnrrrrrrJrTrUs@r2rr&s89VW     +*+@AA%7$'+ #'158<=A,0487;=A6:4859=A$(,0/3&*#\ \ <\ !.\ $E$45 \ !))9 : \ EL) \ $EL1\ 'u|4\ "$u'8"9:\ "%"23\   01\   12\  ((9:\ D>\ $D>\ 'tn!\ "d^#\ $ u99 :%\ \ \ BA\ \ \ \ \ r4rc(eZdZdZfdZdZxZS)BartDecoderWrapperz This wrapper class is a helper class to correctly load pretrained checkpoints when the causal language model is used in combination with the [`EncoderDecoderModel`] framework. crt|t||_dSr\)r=r>rprrs r2r>zBartDecoderWrapper.__init__s.    "6** r4c|j|i|Sr\r)r?rrs r2rJzBartDecoderWrapper.forwardst|T,V,,,r4)rNrOrPrQr>rJrTrUs@r2rrsQ +++++-------r4rzu BART decoder with a language modeling head on top (linear layer with weights tied to the input embeddings). c eZdZdgZfdZdZdZdZdZdZ dZ e e e  dd ejd eejd eejdeejdeejdeejdeeejdeejdeejdeedeedeedeedeee ffdZedZxZS)BartForCausalLMrc*tj|}d|_d|_t |t ||_tj |j |j d|_ | dS)NTFri)copydeepcopyrdis_encoder_decoderr=r>rrrrlrr=rrGrs r2r>zBartForCausalLM.__init__sv&& $)!    '// y!3V5FUSSS  r4c$|jjjSr\rrr.r$s r2rJz$BartForCausalLM.get_input_embeddingssz!..r4c(||jj_dSr\rrMs r2rOz$BartForCausalLM.set_input_embeddingss*/ '''r4c|jSr\rr$s r2rz%BartForCausalLM.get_output_embeddingsrr4c||_dSr\rrs r2rz%BartForCausalLM.set_output_embeddingsrr4c||j_dSr\rr)r?rs r2 set_decoderzBartForCausalLM.set_decoders$ r4c|jjSr\r$r$s r2rzBartForCausalLM.get_decoders z!!r4rNr'r{rrrQrxrrRrrr}rSrTr~c<| | n |jj} | | n |jj} | | n |jj} |j||||||||| | | |  }||d}d}| e| |j} t}|| d|jj | d}| s|f|ddz}||f|zn|St|||j |j|j|jS)a Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) encoder_hidden_states (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder. Used in the cross-attention if the model is configured as a decoder. encoder_attention_mask (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on the padding token indices of the encoder input. This mask is used in the cross-attention if the model is configured as a decoder. Mask values selected in `[0, 1]`: head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) and 2 additional tensors of shape `(batch_size, num_heads, encoder_sequence_length, embed_size_per_head)`. The two additional tensors are only required when the model is used as a decoder in a Sequence to Sequence model. Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. Returns: Example: ```python >>> from transformers import AutoTokenizer, BartForCausalLM >>> tokenizer = AutoTokenizer.from_pretrained("facebook/bart-base") >>> model = BartForCausalLM.from_pretrained("facebook/bart-base", add_cross_attention=False) >>> assert model.config.is_decoder, f"{model.__class__} has to be configured as a decoder." >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt") >>> outputs = model(**inputs) >>> logits = outputs.logits >>> expected_shape = [1, inputs.input_ids.shape[-1], model.config.vocab_size] >>> list(logits.shape) == expected_shape True ```Nrrr+r")rrrrxr]r{)rgr}rSr^rrrrrDr rsr=rrrxr]r{)r?r'r{rrrQrxrrRrrr}rSrTrrrrrs r2rJzBartForCausalLM.forwardseL2C1N--TXT_Tq$8$D $+Jj &1%.nsCnnU_j--aZ=N1O1OPPnnnnnnr4rrs ` r2rzBartForCausalLM._reorder_cacheisQ)  J nnnncmnnnnn NNr4) NNNNNNNNNNNNN)rNrOrPrr>rJrOrrr%rr!rrrErnrrSrrrrrrJrrrTrUs@r2rrs++     ///000&&&%%%"""+L[jkkk'+15=A>B,07;=A59-1$(,0/3&*N N #N !.N  ((9: N !)): ; N EL) N 'u|4N "$u'8"9:N   12N )*N D>N $D>N 'tnN d^N  u77 8N N N lkN `\r4r)XrQrr;r!typingrrrrrEtorch.utils.checkpointrtorch.nnrr r activationsr generationr modeling_attn_mask_utilsrrrrmodeling_outputsrrrrrrrmodeling_utilsrutilsrrrrrrr r!configuration_bartr#modeling_flash_attention_utilsr$ get_loggerrNrrrrrrrrrrrSrRr3rr6rWModuler`rrrrrrrrr*BART_START_DOCSTRINGrrr-rprrrrrrrr4r2r8s //////////// AAAAAAAAAA!!!!!!)))))) .-----                    +*****KJJJJJJ  H % %*%+E')=& %, c [^     88888R\888, = = = = =bl = = =[B[B[B[B[BBI[B[B[B||9|9|9|9|9-|9|9|9~g1g1g1g1g1 g1g1g1V ,CCCCCryCCCLuuuuuryuuupRY0/@     -        -    )V_Dy y y y y %y y y x\ \ \ \ \ %\ \ \ ~X     #   DRThHHHHH#6HHHV x x x x x $7x x x v s s s s s 2s s s l - - - - -, - - -  yyyyy)?yy  yyyr4