Ng? ddlZddlZddlZddlmZddlmZdZdd Zdd Zdd Z dZ ddej de de de de f dZ ddej de de de fdZdS)N)nn)_calculate_fan_in_and_fan_outcd}||d|zz ks ||d|zzkrtjdd|||z |z }|||z |z }|d|zdz d|zdz |||t jdz||||||S)Nc`dtj|tjdz zdz S)N?@)matherfsqrt)xs S/var/www/html/ai-engine/env/lib/python3.11/site-packages/timm/layers/weight_init.pynorm_cdfz _trunc_normal_..norm_cdf s)TXa$)B--/000B66zjmean is more than 2 std from [a, b] in nn.init.trunc_normal_. The distribution of values may be incorrect.) stacklevelr)minmax) warningswarnuniform_erfinv_mul_r r add_clamp_)tensormeanstdabrlus r _trunc_normal_r#s777 q1s7{q1s7{ 2 2 E!" $ $ $ $ !d(c!""A!d(c!""A OOAEAIq1uqy))) NN KKdimm#$$$ KK MMaQM Mrrrctj5t|||||cdddS#1swxYwYdS)aFills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distribution :math:`\mathcal{N}(\text{mean}, \text{std}^2)` with values outside :math:`[a, b]` redrawn until they are within the bounds. The method used for generating the random values works best when :math:`a \leq \text{mean} \leq b`. NOTE: this impl is similar to the PyTorch trunc_normal_, the bounds [a, b] are applied while sampling the normal with mean/std applied, therefore a, b args should be adjusted to match the range of mean, std args. Args: tensor: an n-dimensional `torch.Tensor` mean: the mean of the normal distribution std: the standard deviation of the normal distribution a: the minimum cutoff value b: the maximum cutoff value Examples: >>> w = torch.empty(3, 5) >>> nn.init.trunc_normal_(w) N)torchno_gradr#rrrrr s r trunc_normal_r*+s. 77fdCA66777777777777777777s 488ctj5t|dd|||||dddn #1swxYwY|S)aFills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distribution :math:`\mathcal{N}(\text{mean}, \text{std}^2)` with values outside :math:`[a, b]` redrawn until they are within the bounds. The method used for generating the random values works best when :math:`a \leq \text{mean} \leq b`. NOTE: this 'tf' variant behaves closer to Tensorflow / JAX impl where the bounds [a, b] are applied when sampling the normal distribution with mean=0, std=1.0 and the result is subsquently scaled and shifted by the mean and std args. Args: tensor: an n-dimensional `torch.Tensor` mean: the mean of the normal distribution std: the standard deviation of the normal distribution a: the minimum cutoff value b: the maximum cutoff value Examples: >>> w = torch.empty(3, 5) >>> nn.init.trunc_normal_(w) rrN)r'r(r#rrr)s r trunc_normal_tf_r,Fs. $$vq#q!,,, Cd###$$$$$$$$$$$$$$$ Mscs3F;;OFG x      '!Q&u}H)))TYx%8%8;M%MNNNNNN  ! ! ]__ 4 4 NNty22N 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4  " " !h,'' ]__ + + OOUFE * * * + + + + + + + + + + + + + + + + + +???@@@s$?)B55B9<B92DDDc*t|dddS)Nr-r2)r9r:)r>)rs r lecun_normal_r@|sf8:LMMMMMMr{Gz?headmodulename init_bias head_biasclassifier_namecnt|tjtjtjtjfr||rKtj|j tj |j |dStj |j dt|tjr.|j )tj |j |dSdSdSt|dr|dSdS)NrAr3 init_weights) isinstancerLinearConv1dConv2dConv3d startswithinitzeros_weight constant_biasr*hasattrrI)rCrDrErFrGs r init_weight_vitrVs &29biBIFGG  ??? + + : GNN6= ) ) ) G  fk9 5 5 5 5 5 G ! !&-T ! : : :&"),, :1H!!&+y99999 : :1H1H  ( (rct|tjr||rKtj|jtj|j|dStj |j|jPd|vr&tj |jdn#tj|jdSdSt|tj rCt|j|j&tj|jdSdSt|dr|dSdS)Nmlpgư>r3rI)rJrrKrOrPrQrRrSrTxavier_uniform_r6rMr@rUrI)rCrDrFrGs r init_weight_jaxrZsJ &")$$  ??? + + i GNN6= ) ) ) G  fk9 5 5 5 5 5 G # #FM 2 2 2{&:?4-- 666RW^^\b\gMhMh'& FBI & &fm$$$ ; " GNN6; ' ' ' ' ' # "  ( (r)r$rr%r)rr-r.)rAr$rB)r$rB)r'r rr torch.nn.initrr#r*r,r>r@ModulestrfloatrVrZrr r`sK 777777   F77776:AAAA2NNN %     ,%   r