gdZdZddlZddlZddlmZddlmZmZmZm Z ddl Z ddl Z ddl m Z ddl mZddlmZdd lmZmZdd lmZmZdd lmZmZmZmZdd lmZmZdd lm Z m!Z!ddl"m#Z#m$Z$m%Z%m&Z&m'Z'ddl(m)Z)m*Z*m+Z+e&j,e-Z.dZ/de j0de j0fdZ1de j0de j0fdZ2dZ3dDdZ4 dEdZ5eGdde#Z6eGdde#Z7Gd d!e j8Z9Gd"d#e j8Z:Gd$d%e j8Z;Gd&d'e j8Z<Gd(d)e j8Z=Gd*d+e j8Z>Gd,d-e j8Z?Gd.d/e j8Z@Gd0d1e j8ZAGd2d3eZBd4ZCd5ZDd6ZEGd7d8eBZFGd9d:eBZGe$d;eCGd<d=eBZHe$d>eCGd?d@eBeZIe$dAeCGdBdCeBeZJdS)FzPyTorch CLVP model.N) dataclass)DictOptionalTupleUnion)nn)CrossEntropyLoss)ACT2FN)GenerationConfigGenerationMixin)_prepare_4d_attention_mask!_prepare_4d_causal_attention_mask)BaseModelOutput)BaseModelOutputWithPastAndCrossAttentionsBaseModelOutputWithPooling!CausalLMOutputWithCrossAttentions)PreTrainedModelSequenceSummary)Conv1Disin_mps_friendly) ModelOutputadd_start_docstrings%add_start_docstrings_to_model_forwardloggingreplace_return_docstrings) ClvpConfigClvpDecoderConfigClvpEncoderConfigzsusnato/clvp_devlogitsreturnctj|tjt ||jS)Ndevice)r functional cross_entropytorcharangelenr%)r!s b/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/models/clvp/modeling_clvp.pycontrastive_lossr,;s3 = & &vu|CKKPVP]/^/^/^ _ __ similaritycrt|}t|}||zdz S)Ng@)r,t)r. caption_loss speech_losss r+ clvp_lossr3@s4#J//L":<<>>22K ; &# --r-c|dd|jddzf}|d|jddzdf}tj| |fdS)z*Rotates half the hidden dims of the input..Ndim)shaper(cat)xx1x2s r+ rotate_halfr>Gs] 3"!'"+"" " #B 3 q "" " #B 9rc2YB ' ' ''r-c|||}|||}||zt||zz}||zt||zz}||zt||zz} ||| fS)anApplies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. position_ids (`torch.Tensor`): The position indices of the tokens corresponding to the query and key tensors. For example, this can be used to pass offsetted position ids when working with a KV-cache. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. ) unsqueezer>) qkvcossin position_ids unsqueeze_dimq_embedk_embedv_embeds r+apply_rotary_pos_embrKNs* l  % %m 4 4C l  % %m 4 4C3w;q>>C/0G3w;q>>C/0G3w;q>>C/0G GW $$r-Tc6|rRtjj|d|}|'tjj|ddn|}|}|r=tj|jd|jddzf|j|j}t|D]\}} t| | rutj | |kd } tj | d| tj|g|j| | dg||<tjj| d|||<|'tjj|ddn|}||fS) z This method adds extra bos and eos tokens to input_ids and accordingly modifies the attention_mask which is used in `ClvpConditioningEncoder` and the generation loop of the `ClvpModelForConditionalGeneration`. )rrvalueNrrdtyper%r$)rr)r(rr&padzerosr9rQr% enumeratersumwheremin concatenatetensor) input_idsattention_mask pad_token_id bos_token_id eos_token_id add_bos_token add_eos_tokenmodified_input_idsi each_input_idposs r+_pad_extra_bos_eos_tokensreks H'++Iv\+RR HVHbEH  # #NF! # D D Dhv # "[ _Q !3a!7 8 XaXh   !*) 4 4 k k A}  ==AACC kk-<"?@@CGGII(-(9"4C4(%, ~iN^*_*_*_anorososatu))"1%% ).(;(?(? v]i(?(j(j"1%%HVHbEH  # #NF! # D D Dhv  ~ --r-ceZdZUdZdZeejed<dZ ejed<dZ eejed<dZ ee ejed<dZ ee ejed<dS)ClvpEncoderOutputak Base class for CLVP encoder's outputs that contains a pooling of the last hidden states as well as a projection output (a linear layer on top of the pooled output). Args: embeds (`torch.FloatTensor` of shape `(batch_size, output_dim)`, *optional*, returned when model is initialized with `with_projection=True`): The embeddings obtained by applying the projection layer to the pooler_output. last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): The hidden state of the last layer of the model. pooler_output (`torch.FloatTensor` of shape `(batch_size, hidden_size)`): Pooled output of the `last_hidden_state`. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. Nembedslast_hidden_state pooler_output hidden_states attentions)__name__ __module__ __qualname____doc__rhrr( FloatTensor__annotations__rirjrkrrlr-r+rgrgs*+/FHU& '...+/u(///15M8E-.5558