g (dZddlZddlZddlmZmZmZmZm Z ddl Z ddl Z ddl m Z ddl mZddlmZmZddlmZdd lmZmZmZmZdd lmZd d lmZejeZd Z Gdde j!Z"e j#j$dZ%dZ&Gdde j!Z'd5de j(de)de*de j(fdZ+Gdde j!Z,Gdde j!Z-Gdd e j!Z.Gd!d"e j!Z/d#Z0d$Z1Gd%d&e j!Z2Gd'd(e j!Z3d)e j!ddfd*Z4Gd+d,eZ5d-Z6d.Z7ed/e6Gd0d1e5Z8ed2e6Gd3d4e5eZ9dS)6zPyTorch ViTDet backbone.N)DictListOptionalTupleUnion)nn)ACT2FN)BackboneOutputBaseModelOutput)PreTrainedModel)add_start_docstrings%add_start_docstrings_to_model_forwardloggingreplace_return_docstrings) BackboneMixin) VitDetConfigrcLeZdZdZfdZdZdejdejfdZxZ S)VitDetEmbeddingsz This class turns `pixel_values` of shape `(batch_size, num_channels, height, width)` into the initial `hidden_states` (patch embeddings) to be consumed by a Transformer. cXt|j|j}}|j|j}}t |tjj r|n||f}t |tjj r|n||f}|d|dz|d|dzz}||_ ||_||_||_ |j r8|dz}tjtjd||j|_nd|_tj|||||_dS)Nrr) kernel_sizestride)super__init__pretrain_image_size patch_size num_channels hidden_size isinstance collectionsabcIterable image_size num_patches use_absolute_position_embeddingsr Parametertorchzerosposition_embeddingsConv2d projection) selfconfigr$rrrr% num_positions __class__s f/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/models/vitdet/modeling_vitdet.pyrzVitDetEmbeddings.__init__2s$ !'!;V=NJ $*$79Kk #-j+/:R#S#SqZZZdfpYq #-j+/:R#S#SqZZZdfpYq !!} 1 5*Q-:VW=:XY $$(&  2 ,'!OM')|EK=RXRd4e4e'f'fD $ $'+D $)L+:^hiiic|r|ddddf}|jd}ttj|}||z|krt dt js ||ks||kritj | d||d dddd||fdd }| ddddS| d||dS) a Calculate absolute positional embeddings. If needed, resize embeddings and remove cls_token dimension for the original embeddings. Args: abs_pos_embeddings (`torch.Tensor`): Absolute positional embeddings with (1, num_position, num_channels). has_cls_token (`bool`): If true, has 1 embedding in abs_pos_embeddings for cls token. height (`int`): Height of input image tokens. width (`int`): Width of input image tokens. Returns: Absolute positional embeddings after processing with shape (1, height, width, num_channels) Nrz5Absolute position embeddings must be a square number.rr bicubicF)sizemode align_corners) shapeintmathsqrt ValueErrorr(jit is_tracingr functional interpolatereshapepermute)r-abs_pos_embeddings has_cls_tokenheightwidth num_positionr7new_abs_pos_embeddingss r1get_absolute_positionsz'VitDetEmbeddings.get_absolute_positionsHs$  ;!3AAAqrrE!: )/2 49\**++ $;, & &TUU U 9   ! ! Ddfnn %']%>%>"**1dD"==EEaAqQQe_# &?&& "*11!Q1== =%--aCC Cr2 pixel_valuesreturncp|jd}||jkrtd|jd|d||}|jf|dddd}|||jd|jd|jdz}|dddd}|S) NrzoMake sure that the channel dimension of the pixel values match with the one set in the configuration. Expected z but got .rr5r T)r:rr>r,r*rDrK)r-rLr embeddingss r1forwardzVitDetEmbeddings.forwardns#)!, 4, , ,I!.II9EIII __\22  # /#++Aq!Q77J#d&A&A($ 0@0CZEUVWEX''J$++Aq!Q77Jr2) __name__ __module__ __qualname____doc__rrKr(TensorrQ __classcell__r0s@r1rr,s jjjjj,$D$D$DLELU\r2rctdt||zdz }|jd|krtj|d|jddddd|d}|d|dd}n|}tj |dddft||z dz}tj |dddft||z dz}||z |dz t||z dzz}|| S) a Get relative positional embeddings according to the relative positions of query and key sizes. Args: q_size (`int`): Size of query q. k_size (`int`): Size of key k. rel_pos (`torch.Tensor`): Relative position embeddings (num_embeddings, num_channels). Returns: Extracted positional embeddings according to relative positions. r5rrr4linear)r7r8N?) r;maxr:rrArBrCrDr(arangelong)q_sizek_sizerel_pos max_rel_distrel_pos_resizedq_coordsk_coordsrelative_coordss r1 get_rel_posrgsQ q3vv...233L}Q<''-33 OOAw}Q/ 4 4 < bhwkzbhwc,wkc->bhwkN)rgr:rCr(einsumview)attnqueries rel_pos_h rel_pos_wr_r`queries_height queries_width keys_height keys_widthrelative_heightrelative_width batch_size_dimr_qrelative_weights r1!add_decomposed_relative_positionsrzs,%+!NM$K!.+yIIO  IFFN J3 //*nmS I ICl#3S/JJOl#3S.IIO *nm[*UU !!!QQQ111d* + , !!!QQQ4* + , d:~ 5{Z7OPP  Kr2c,eZdZdZdfd ZddZxZS)VitDetAttentionz=Multi-head Attention block with relative position embeddings.Nct|j}|j}||_||z}|dz|_t j||dz|j|_ t j|||_ |j |_ |j rrt j tjd|dzdz ||_t j tjd|dzdz ||_dSdS)z Args: config (`VitDetConfig`): Model configuration. input_size (`Tuple[int]`, *optional*): Input resolution, only required in case relative position embeddings are added. gr biasr5rrN)rrrnum_attention_heads num_headsscalerLinearqkv_biasqkvproj use_relative_position_embeddingsr'r(r)rmrn)r-r. input_sizerwrhead_dimr0s r1rzVitDetAttention.__init__s  . ")#t^ 9S#'@@@Ic3'' 060W-  0 X\%+a*Q-6G!6KX*V*VWWDN\%+a*Q-6G!6KX*V*VWWDNNN X Xr2Fc 4|j\}}}}|||||zd|jdddddd}|d||jz||zdd\}} } ||jz| ddz} |jr"t| ||j |j ||f||f} | d} | | z}| ||j||d}|ddddd}||||d}||}|r8| ||j| jd| jd} || f} n|f} | S) Nr r4r5rr)rw)r:rrCrrDunbindr transposerrzrmrnsoftmaxrjr)r- hidden_stateoutput_attentionsrurGrHrvrrlkeysvaluesattention_scoresattention_probsoutputss r1rQzVitDetAttention.forwards'3'9$ FE1hh|$$,,Z%DN\^__gghiklnoqrtuvv # AzDN/JFUZN\^ _ _ f fgh i iv#dj0DNN2r4J4JJ  0 @ '4>4>FTY?]cej\k   +22r2::&/ #((T^VUTVWW #++Aq!Q:: #++JrJJ yy..  &-55DNO,A",EG\]_G`O$_5GG#oGr2N)FrRrSrTrUrrQrWrXs@r1r|r|s]GGXXXXXX4r2r|Finput drop_probtrainingrMc|dks|s|Sd|z }|jdfd|jdz zz}|tj||j|jz}||||z}|S)aF Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). Comment by Ross Wightman: This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use 'survival rate' as the argument. rrr)r)dtypedevice)r:ndimr(randrrfloor_div)rrr keep_probr: random_tensoroutputs r1 drop_pathrsCx I I [^  Q 7 7E 5 EL Y Y YYM YYy ! !M 1F Mr2cjeZdZdZd deeddffd ZdejdejfdZ de fdZ xZ S) VitDetDropPathzXDrop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).NrrMcVt||_dSr)rrr)r-rr0s r1rzVitDetDropPath.__init__)s$ "r2 hidden_statesc8t||j|jSr)rrr)r-rs r1rQzVitDetDropPath.forward-s FFFr2c6d|jS)Nzp={})formatrr-s r1 extra_reprzVitDetDropPath.extra_repr0s}}T^,,,r2r) rRrSrTrUrfloatrr(rVrQstrrrWrXs@r1rr&sbb##(5/#T######GU\GelGGGG-C--------r2rc*eZdZdZdfd ZdZxZS)VitDetLayerNormaL A LayerNorm variant, popularized by Transformers, that performs point-wise mean and variance normalization over the channel dimension for inputs that have shape (batch_size, channels, height, width). https://github.com/facebookresearch/ConvNeXt/blob/d1fa8f6fef0a165b27399986cc2bdacc92777e40/models/convnext.py#L119 ư>cttjt j||_tjt j||_||_ |f|_ dSr) rrrr'r(onesweightr)repsnormalized_shape)r-rrr0s r1rzVitDetLayerNorm.__init__;si l5:.>#?#?@@ L-=!>!>?? !1 3r2c"|dd}||z ddd}||z tj||jzz }|jddddf|z|jddddfz}|S)NrT)keepdimr5)meanpowr(r=rrr)r-xuss r1rQzVitDetLayerNorm.forwardBs FF1dF # # UKKNN  4  0 0 UejTX.. . K4 & *TYqqq$}-E Er2)rrrXs@r1rr4sV 444444r2rc(eZdZdZfdZdZxZS)VitDetResBottleneckBlockz The standard bottleneck residual block without the last activation layer. It contains 3 conv layers with kernels 1x1, 3x3, 1x1. cttj||dd|_t ||_t|j|_ tj||ddd|_ t ||_ t|j|_ tj||dd|_ t ||_dS)ar Args: config (`VitDetConfig`): Model configuration. in_channels (`int`): Number of input channels. out_channels (`int`): Number of output channels. bottleneck_channels (`int`): Number of output channels for the 3x3 "bottleneck" conv layers. rFr~r )paddingrN)rrrr+conv1rnorm1r hidden_actact1conv2norm2act2conv3norm3)r-r. in_channels out_channelsbottleneck_channelsr0s r1rz!VitDetResBottleneckBlock.__init__Ps Y{,?OOO $%899 6,- Y24GTU\abbb $%899 6,- Y2L!%PPP $\22 r2cX|}|D] }||}||z}|Sr)children)r-routlayers r1rQz VitDetResBottleneckBlock.forwardhs;]]__  E%**CC#g r2rrXs@r1rrJsQ 333330r2rcPeZdZdededdffd ZdejdejfdZxZS) VitDetMlp in_featureshidden_featuresrMNcttj|||_t |j|_tj|||_tj |j |_ dSr) rrrrfc1r ractfc2Dropout dropout_probdrop)r-r.rrr0s r1rzVitDetMlp.__init__rsf 9[/::&+,9_k::Jv233 r2rc||}||}||}||}||}|Sr)rrrr)r-rs r1rQzVitDetMlp.forwardysR HHQKK HHQKK IIaLL HHQKK IIaLLr2) rRrSrTr;rr(rVrQrWrXs@r1rrqsx4C4#4$444444%,r2rc v|j\}}}}|||zz |z}|||zz |z}tj|ddd|d|f}||z||z} }||||z|| |z||}|ddddddd|||} | || ffS)a  Partition into non-overlapping windows with padding if needed. Args: hidden_state (`torch.Tensor`): Input tokens with [batch_size, height, width, num_channels]. window_size (`int`): Window size. Returns: `tuple(torch.FloatTensor)` comprising various elements: - windows: windows after partition with [batch_size * num_windows, window_size, window_size, num_channels]. - (padded_height, padded_width): padded height and width before partition rrr r5rr4)r:rrApadrjrD contiguous) r window_sizerurGrHr pad_height pad_width padded_height padded_widthwindowss r1window_partitionrs/;.@+J| 44 CJu{22kAI=$$\Aq!Y:3VWWL"(:"5uy7HOSYSd>de \#6+@AAA ( [A-=-=zzKQ\C]   }dN^._._ +M+!% /"0" " /q1 (,  a  .}d>NP`cikpbqrrM!4>>-#@#@@ %txx =@Y@Y7Z7Z([([[ %--aAq99  " 9 MM-88M "W,r2)rrF)NF)rRrSrTrUrrr;boolrr(rVrrrrQrWrXs@r1rrsMMqv"49LOim @-1"' ((|(EL)( ( uU\5</0% 2EE F ((((((((r2rceZdZdeddffd Z d dejdeejd ed ed ede e e ff d Z xZ S) VitDetEncoderr.rMNc t||_|j}dt jd|j|D}g}t|D]E}|t|||||j vr|j nd||j vFtj||_d|_dS)Nc6g|]}|S)item).0rs r1 z*VitDetEncoder.__init__..s \\\q!&&((\\\r2r)rrrF)rrr.num_hidden_layersr(linspacerrangeappendrwindow_block_indicesrresidual_block_indicesr ModuleListrgradient_checkpointing)r-r.depthrlayersir0s r1rzVitDetEncoder.__init__s  (]\EN1f>SUZ,[,[\\\u  A MM#1!#4676;V6V6V 2 2\]'(F,I'I     ]6** &+###r2FTrrroutput_hidden_states return_dictc|rdnd}|rdnd}t|jD]h\}} |r||fz}|||nd} |jr%|jr|| j|| |} n | || |} | d}|r || dfz}i|r||fz}|st d|||fDSt|||S)Nrrrc3K|]}||V dSrr)rvs r1 z(VitDetEncoder.forward..Hs(mmq_`_l_l_l_l_lmmr2last_hidden_stater attentions) enumeraterrr_gradient_checkpointing_func__call__tupler ) r-rrrrrall_hidden_statesall_self_attentionsr layer_modulelayer_head_mask layer_outputss r1rQzVitDetEncoder.forward$sI#7@BBD$5?bb4(44 P POA|# I$58H$H!.7.CillO* `t} ` $ A A )!#% !! !- ]OM^ _ _ )!,M  P&9]1=M