Ng, dZddlZddlZddlmZmZmZmZmZddl Z ddl m Z ddl m cm Z ddlmZmZddlmZddlmZejeZGd d e jZGd d eZ ddeedededefdZdS)a? Image to Patch Embedding using Conv2d A convolution based approach to patchifying a 2D image w/ embedding projection. Based on code in: * https://github.com/google-research/vision_transformer * https://github.com/google-research/big_vision/tree/main/big_vision Hacked together by / Copyright 2020 Ross Wightman N)CallableListOptionalTupleUnion)nn)Formatnchw_to) to_2tuple)_assertceZdZUdZeed<ejje ed< dd e e d e d e de de e de de e de de de ffd Zd ee ee e fffdZ dd e ee ee e ffd e ee ee e fffdZddeee e fe ffdZd ee e fdee e ffdZdZxZS) PatchEmbed! 2D Image to Patch Embedding output_fmtdynamic_img_padNTFimg_size patch_sizein_chans embed_dim norm_layerflattenbiasstrict_img_sizec tt||_||\|_|_|_|d|_t||_ n||_tj |_ | |_ | |_ tj||||||_|r ||ntj|_dS)NF kernel_sizestrider)super__init__r r_init_img_sizer grid_size num_patchesrr rNCHWrrrConv2dprojIdentitynorm) selfrrrrrrrrrr __class__s S/var/www/html/ai-engine/env/lib/python3.11/site-packages/timm/layers/patch_embed.pyr$zPatchEmbed.__init__ s #J//:>:M:Mh:W:W7 t~t'7  ! DL$Z00DOO#DL$kDO..Ih zR\cghhh -7JJJy)))R[]] c|jsJ|dSt|}tdt||jD}|d|dz}|||fS)N)NNNcg|] \}}||z Sr3).0sps r/ z-PatchEmbed._init_img_size..Cs MMMda16MMMr0rr )rr tuplezip)r-rr&r's r/r%zPatchEmbed._init_img_size>sp  ##X&&MMc(DO.L.LMMMNN lYq\1 K//r0c rd}|t|}|||jkrtj5t j|jj|jj|||jj du}|j t|jj |d|jj $|j |jj ||_dddn #1swxYwY||_|p|j }||j ks|*||\|_ |_|_dSdS)Nr T)verbose)r rtorchno_gradrr)r* in_channels out_channelsrweightcopy_resample_patch_embedrr%r&r')r-rrnew_patch_sizenew_projs r/set_input_sizezPatchEmbed.set_input_sizeGsx   !&z22N  %.DO*K*K % %9I)I* .)t3 %%&:49;K^ei&j&j&jkkk9>-M'' 777$  % % % % % % % % % % % % % % %-DO,t} t} $ $(B>B>Q>QRZ>[>[ ;DM4>4+;+;+;)C(BsB%C%%C),C)returnc<|rt|jS|jS)N)maxr)r- as_scalars r/ feat_ratiozPatchEmbed.feat_ratioas"  #t'' '? "r0c|jrPtj|d|jdz tj|d|jdz fS|d|jdz|d|jdzfS)z Get grid (feature) size for given image size taking account of dynamic padding. NOTE: must be torchscript compatible so using fixed tuple indexing rr )rmathceilr)r-rs r/dynamic_feat_sizezPatchEmbed.dynamic_feat_sizegs   X9Xa[4?1+==>> (ST+X\XghiXjJj@k@kk kA;$/!"44hqkT_UVEW6WW Wr0c|j\}}}}|j|jrct||jdkd|d|jddt||jdkd|d|jddno|jsht||jdzdkd|d|jddt||jdzdkd|d|jdd|jrq|jd||jdzz |jdz}|jd||jdzz |jdz}t j|d|d|f}||}|j r*| d dd}n*|j tj krt||j }||}|S) NrzInput height (z) doesn't match model ().r z Input width (z%) should be divisible by patch size ()shaperrr rrFpadr*r transposerr r(r r,)r-xBCHWpad_hpad_ws r/forwardzPatchEmbed.forwardpsW 1a = $# T]1--/n/n/nZ^ZghiZj/n/n/noooT]1--/mq/m/mY]YfghYi/m/m/mnnnn) **a/cQccT_]^M_ccc**a/bAbbDO\]L^bbb   /_Q'!doa.@*@@DOTUDVVE_Q'!doa.@*@@DOTUDVVEa!UAu-..A IIaLL < , ! &&q!,,AA _ + +4?++A IIaLLr0) rrrrNTNTTF)NN)T)__name__ __module__ __qualname____doc__r __annotations__r<jitFinalboolrintrstrr$rrr%rErJrNr] __classcell__r.s@r/rrsY_T****'*  -1 (,$($)KKsmKK K  K !* KK! KK"K"KKKKKK<0uS%S/-A'B0000?C@D\\uS%S/%9:;\!sE#s(O';!<=\\\\4##E%S/32F,G#### X%S/XeCHoXXXXr0rceZdZUdZeed< dd eed ed ed ed eede dee de ffd Z de e jeeffdZxZS)PatchEmbedWithSizerrrrrrNTrrrrrrrc Zt||||||||dS)N)rrrrrrrr)r#r$) r-rrrrrrrrr.s r/r$zPatchEmbedWithSize.__init__sH !!!  r0rFc(|j\}}}}|jht||jdzdkd|d|jddt||jdzdkd|d|jdd||}|jdd}|jr*|ddd}n*|jtj krt||j}| |}||fS) NrzInput image height (z#) must be divisible by patch size (rPr zInput image width (rQ) rRrr rr*rrUrr r(r r,)r-rVrWrXrYrZ feat_sizes r/r]zPatchEmbedWithSize.forwardsSW 1a = $ A**a/2E2E2Enrn}~oA2E2E2E F F F A**a/2Dq2D2Dmqm|}~m2D2D2D E E E IIaLLGBCCL < , ! &&q!,,AA _ + +4?++A IIaLL)|r0)rrrrNTNT)r^r_r`rar rbrrfrrergr$rr<Tensorrr]rhris@r/rkrks'*  -1 (,  sm      !*   !        , E%,S "9:        r0rkbicubicTFnew_size interpolation antialiasr;c  ddl ddlm}n#t$r ddlm}YnwxYwt |jdks Jdt dks Jd|jdd}t|tkr|S|r)t d |jd d d fd  fd}||}tj j |j |jfd} ||| dddd} |j} |}| |}|| }|S)a/Resample the weights of the patch embedding kernel to target resolution. We resample the patch embedding kernel by approximately inverting the effect of patch resizing. Code based on: https://github.com/google-research/big_vision/blob/b00544b81f8694488d5f36295aeb7972f3755ffe/big_vision/models/proj/flexi/vit.py With this resizing, we can for example load a B/8 filter into a B/16 model and, on 2x larger input image, the result will match. Args: patch_embed: original parameter to be resized. new_size (tuple(int, int): target shape (height, width)-only. interpolation (str): interpolation for resize antialias (bool): use anti-aliasing filter in resize verbose (bool): log operation Returns: Resized patch embedding kernel. rN)vmapzFour dimensions expectedrQzNew shape should only be hwrnzResize patch embedding z to z, w/ z interpolation.ctj|d}tj||d}|S)N)NN.)sizemodert)rr.)r<rprS interpolatenumpy)x_np _new_sizex_tf x_upsampledrtrss r/resizez$resample_patch_embed..resizesW|D!!/2m y} KKKKTVV[V[V]V] r0cFg}t|D]b}|}d|||<|||dc|jS)Ng?)rangeprodzeros unravel_indexappendreshapestackT) _old_sizer~mati basis_vecnprs r/get_resize_matz,resample_patch_embed..get_resize_matsrwwy))** A AA++I8:Ib&&q)44 5 JJvvi33;;B?? @ @ @ @xx}}r0)devicec^|dz}|S)Nr)r)kernelresampled_kernelrrresize_mat_pinvs r/resample_kernelz-resample_patch_embed..resample_kernels.*V^^B-?-??''111r0r )r|r<rv ImportError functorchlenrRr8_loggerinfotensorlinalgpinvrrdtypefloatto) patch_embedrrrsrtr;rvold_sizer resize_matrv_resample_kernel orig_dtyperrrs ``` @@@r/rBrBs 4# ###""""""""# { ! !Q & & &(B & & & x==A   <    %H X%//))u s{/@sshssUbsssttt  (33Jl29>>*,#?#? HZ[[[O222222TT/1a88!Q??"J##%%K##K00K..,,K s &&)rqTF)raloggingrLtypingrrrrrr<rtorch.nn.functional functionalrSformatr r helpersr trace_utilsr getLoggerr^rModulerrkrfrgrerBr3r0r/rs   99999999999999 ######## ' H % %ooooooood((((((((\' CCs)CC C  CCCCCCr0