gw>dZddlmZddlmZmZmZmZddlZddl Zddlm Z ddl m Z m Z ddlmZdd lmZdd lmZmZmZmZmZmZd d lmZer dd lmZmZmZddlm Z m!Z!ej"e#Z$dZ% d)dej&de'de'dej(dej)de*dej&de'de+dej&fdZ,eGddeZ-Gdd e j.Z/d!Z0ed"e0Gd#d$eZ1d%Z2ed&e0Gd'd(e1eZ3dS)*zPyTorch PaliGemmamodel.) dataclass)ListOptionalTupleUnionN)nn)Cache StaticCache)GenerationMixin)PreTrainedModel) ModelOutputadd_start_docstrings%add_start_docstrings_to_model_forwardis_flash_attn_2_availableloggingreplace_return_docstrings)PaliGemmaConfig)index_first_axis pad_input unpad_input) AutoModelAutoModelForCausalLMrFattention_masksequence_length target_lengthdtypedevice min_dtypecache_position batch_size is_trainingtoken_type_idsc  ||dkr|} ntj||f|||} |dkr$|rtj| d} n d| ddd|f<| tj||j|ddkz} | ddddddf|ddd} || } |j d} | ddddddd| f|ddddddf | jz} | d k} | ddddddd| f | || ddddddd| f<|ra| ddddddd| f | ddddddf | jd kd | ddddddd| f<| S) aR Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. device (`torch.device`): The device to plcae the 4D attention mask on. min_dtype (`float`): The minimum value representable with the dtype `dtype`. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. is_training (`bool`): Whether the model is in training mode or in inference. The condition is checked by presence/absence of `token_type_ids/labels` N fill_valuerr rdiagonalr r) dimtorchfulltriuaranger reshapeexpandcloneshapeto masked_fill)rrrrr r!r"r#r$r%kwargs causal_mask mask_length padding_masks l/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/models/paligemma/modeling_paligemma.py5_prepare_4d_causal_attention_mask_with_cache_positionr?3sZJ!n&8&8&:&:a&?&?$ j/=!Ai_dmsttt a   7#jqAAA 36 AAA///0u|M.:OPPPSaSiSijlnoSpSppp !$aaa"23:::q"bQQ  %%++--K(.r2K&qqq!!!QQQ  '<=qqqRVX\^_^_^_O_@`@c@cdodv@w@wwL'1,L1