Ng= dZddlZddlmZddlmcmZddlmZ dd e d e d e d e d e de f dZ ddej d e d e d e d e d e f dZGddejZdd e de de fdZGddejZdS)a> DropBlock, DropPath PyTorch implementations of DropBlock and DropPath (Stochastic Depth) regularization layers. Papers: DropBlock: A regularization method for convolutional networks (https://arxiv.org/abs/1810.12890) Deep Networks with Stochastic Depth (https://arxiv.org/abs/1603.09382) Code: DropBlock impl inspired by two Tensorflow impl that I liked: - https://github.com/tensorflow/tpu/blob/master/models/official/resnet/resnet_model.py#L74 - https://github.com/clovaai/assembled-cnn/blob/master/nets/blocks.py Hacked together by / Copyright 2020 Ross Wightman N)ndgrid皙??F drop_prob block_size gamma_scale with_noiseinplace batchwisec|j\}}} } | | z} t|t| | } ||z| z| dzz | |z dz| |z dzzz } ttj| |jtj| |j\}}|| dzk|| | dz dzz kz|| dzk|| | dz dzz kzz}tj|dd| | f|j}|r&tj d|| | f|j|j}ntj |}d| z |z |zdk|j}tj | | d| dz }|r{|r%tj d|| | f|j|jntj|}|r/|||d|z zn||z|d|z zz}n||tjdz |j}|r|||zn||z|z}|S)a  DropBlock. See https://arxiv.org/pdf/1810.12890.pdf DropBlock with an experimental gaussian noise option. This layer has been tested on a few training runs with success, but needs further validation and possibly optimization for lower runtime impact. r)devicedtype)rr kernel_sizestridepaddinggHz>)shapeminrtorcharangerreshapetorrand rand_likeF max_pool2drandn randn_likemul_add_numelfloat32sumadd)xrr r r r r BCHW total_sizeclipped_block_sizegammaw_ih_i valid_block uniform_noise block_mask normal_noisenormalize_scales L/var/www/html/ai-engine/env/lib/python3.11/site-packages/timm/layers/drop.py drop_block_2dr9sJAq!QQJZQ33 ) #j 03E3J J ^a A NQ$6 7 9Eel1QX666 Qqx8X8X8XYYHC-22sQBTWXBX]^A^=^7^_-22sQBTWXBX]^A^=^7^_aK- aAq\::==AG=LLK+ Aq!Q)rrr empty_like bernoulli_rr rrnormal_r#r$r%r&r'r()r)rr r r r r*r+r,r-r.r/r0r5r6r7s r8drop_block_fast_2dr?NsJAq!QQJZQ33 ) #j 03E3J J ^a A NQ$6 7 9E!!$$//66J ag,>qRdhiRikkkJ 1'**2244  B FF2 ? # # ( ( )B C C C CR*_% z(AAAA^ %++-- EM 0R0R0V0V0X0X0\0\]a0b0bbffmnmtfuu  1 FF:/ 0 0 0 0J0A Hr:cVeZdZdZ ddeded ed ed ed ed effd ZdZxZ S) DropBlock2dz9 DropBlock. See https://arxiv.org/pdf/1810.12890.pdf rrrFTrr r r r r fastctt|||_||_||_||_||_||_||_ dSN) superrA__init__rr r r r r rB) selfrr r r r r rB __class__s r8rFzDropBlock2d.__init__ysW k4  ))+++"&$$ " r:c |jr|js|S|jr-t||j|j|j|j|jSt||j|j|j|j|j|j SrD) trainingrrBr?r r r r r9r rGr)s r8forwardzDropBlock2d.forwards} DN H 9 u%4>4?D4DdoW[Wcee e!4>4?D4DdoW[Wceiesuu ur:)rrrFFFT) __name__ __module__ __qualname____doc__floatintboolrFrL __classcell__rHs@r8rArAus #!$$!#     $uuuuuuur:rATrJ scale_by_keepc|dks|s|Sd|z }|jdfd|jdz zz}|||}|dkr|r||||zS)a(Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use 'survival rate' as the argument. rVrr)r)rndim new_emptyr=div_)r)rrJrW keep_probr random_tensors r8 drop_pathr^sBhI I WQZMDAFQJ/ /EKK&&11)<ros     3 3 3 3  3  3  3 3 3 3 3 p   $ $ <$ $ $  $  $  $ $ $ $ Nuuuuu")uuuBE$t( ; ; ; ; ;ry ; ; ; ; ;r: