gA  dZddlZddlZddlmZddlmZmZmZm Z ddl Z ddl Z ddl Z ddl mZddlmZmZmZddlmZdd lmZdd lmZdd lmZmZmZdd lmZdd lm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'ddl(m)Z)m*Z*m+Z+e#rddl,m-Z-e%j.e/Z0dZ1dZ2dZ3 dRde j4de5de5de5de5de j4f dZ6dSd!Z7d"Z8d#Z9eGd$d%e Z:eGd&d'e Z;eGd(d)e Z<Gd*d+ej=Z>Gd,d-ej=Z?Gd.d/ej=Z@Gd0d1e@ZAGd2d3e@ZBe@eAeBd4ZCGd5d6ej=ZDGd7d8ej=ZEGd9d:eZFd;ZGdZJGd?d@ej=ZKGdAdBej=ZLe!dCeGGdDdEeFZMGdFdGej=ZNGdHdIej=ZOe!dJeGGdKdLeFZPe!eGGdMdNeFZQe!dOeGGdPdQeFZRdS)TzPyTorch Siglip model.N) dataclass)AnyOptionalTupleUnion)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)_calculate_fan_in_and_fan_out)ACT2FN)_prepare_4d_attention_mask)BaseModelOutputBaseModelOutputWithPoolingImageClassifierOutput)PreTrainedModel) ModelOutputadd_start_docstrings%add_start_docstrings_to_model_forwardis_flash_attn_2_available#is_flash_attn_greater_or_equal_2_10loggingreplace_return_docstrings torch_int) SiglipConfigSiglipTextConfigSiglipVisionConfig)_flash_attention_forwardrzgoogle/siglip-base-patch16-224cd}||d|zz ks ||d|zzkrtjdd|||z |z }|||z |z }|d|zdz d|zdz |||t jdz|||||dS)Nc`dtj|tjdz zdz S)N?@)matherfsqrt)xs f/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/models/siglip/modeling_siglip.pynorm_cdfz _trunc_normal_..norm_cdf<s)dhq49S>>1222c99zjmean is more than 2 std from [a, b] in nn.init.trunc_normal_. The distribution of values may be incorrect.) stacklevelrr$)minmax) warningswarnuniform_erfinv_mul_r%r'add_clamp_)tensormeanstdabr*lus r)_trunc_normal_r>9s::: q1s7{q1s7{ 2 2  ;     !d(c!""A!d(c!""A OOAEAIq1uqy))) NN KKdinn$%%% KK MMaQMr+r#r$r7r8r9r:r;returnctj5t|dd|||||ddddS#1swxYwYdS)anFills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distribution :math:`\mathcal{N}( ext{mean}, ext{std}^2)` with values outside :math:`[a, b]` redrawn until they are within the bounds. The method used for generating the random values works best when :math:`a \leq ext{mean} \leq b`. NOTE: this 'tf' variant behaves closer to Tensorflow / JAX impl where the bounds [a, b] are applied when sampling the normal distribution with mean=0, std=1.0 and the result is subsequently scaled and shifted by the mean and std args. Args: tensor: an n-dimensional `torch.Tensor` mean: the mean of the normal distribution std: the standard deviation of the normal distribution a: the minimum cutoff value b: the maximum cutoff value rr#N)torchno_gradr>r4r5)r7r8r9r:r;s r)trunc_normal_tf_rE]s* $$vq#q!,,, Cd###$$$$$$$$$$$$$$$$$$s