gBdZddlmZmZmZmZddlZddlmZddlm Z ddl m Z ddl m Z dd lmZmZmZdd lmZd d lmZd ZGddejZGddejZGddejZGddejZGddejZGdde ZdZdZedeGddeZ dS)zrPyTorch UperNet model. Based on OpenMMLab's implementation, found in https://github.com/open-mmlab/mmsegmentation.)ListOptionalTupleUnionN)nn)CrossEntropyLoss)SemanticSegmenterOutput)PreTrainedModel)add_start_docstrings%add_start_docstrings_to_model_forwardreplace_return_docstrings) load_backbone) UperNetConfigrceZdZdZ ddededeeeeeffdeeeeefefd ed eeeeeffd d ffd Z de j d e j fdZ xZ S)UperNetConvModulez A convolutional block that bundles conv/norm/activation layers. This block simplifies the usage of convolution layers, which are commonly used with a norm layer (e.g., BatchNorm) and activation layer (e.g., ReLU). rFr in_channels out_channels kernel_sizepaddingbiasdilationreturnNcttj|||||||_tj||_tj|_dS)N)rrrrrr) super__init__rConv2dconv BatchNorm2d batch_normReLU activation)selfrrrrrr __class__s h/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/models/upernet/modeling_upernet.pyrzUperNetConvModule.__init__(si I#%#     .66'))inputc||}||}||}|SN)rr!r#)r$r(outputs r&forwardzUperNetConvModule.forward=s:5!!(((( r')rFr)__name__ __module__ __qualname____doc__intrrstrboolrtorchTensorr, __classcell__r%s@r&rr"s5601$$$$3c3h/0 $ sE#s(OS01 $  $U38_,-$ $$$$$$*U\elr'rcTeZdZdedededdffd ZdejdejfdZxZS) UperNetPyramidPoolingBlock pool_scalerchannelsrNcttj|t ||dg|_t |jD](\}}|t||)dS)Nrr) rrrAdaptiveAvgPool2drlayers enumerate add_moduler2)r$r:rr;ilayerr%s r&rz#UperNetPyramidPoolingBlock.__init__Fs    , , k8 C C C  "$+.. + +HAu OOCFFE * * * * + +r'r(c4|}|jD] }||}|Sr*)r?)r$r( hidden_staterCs r&r,z"UperNetPyramidPoolingBlock.forwardOs/ [ / /E 5..LLr') r-r.r/r1rr4r5r,r6r7s@r&r9r9Es+3+S+C+D++++++U\elr'r9c xeZdZdZdeedfdedededdf fd Zd ej de ej fd Z xZ S) UperNetPyramidPoolingModulea} Pyramid Pooling Module (PPM) used in PSPNet. Args: pool_scales (`Tuple[int]`): Pooling scales used in Pooling Pyramid Module. in_channels (`int`): Input channels. channels (`int`): Channels after modules, before conv_seg. align_corners (`bool`): align_corners argument of F.interpolate. pool_scales.rr; align_cornersrNcVt||_||_||_||_g|_t|D]T\}}t|||}|j || t||UdS)N)r:rr;) rrrHrIrr;blocksr@r9appendrAr2) r$rHrr;rIrBr:blockr%s r&rz$UperNetPyramidPoolingModule.__init__es &*&   &{33 + +MAz.*R]hpqqqE K  u % % % OOCFFE * * * * + +r'xcg}|jD]d}||}tj||ddd|j}||e|S)NbilinearsizemoderI)rKr functional interpolaterSrIrL)r$rNppm_outsppmppm_outupsampled_ppm_outs r&r,z#UperNetPyramidPoolingModule.forwardqs{; / /Cc!ffG " 9 9affhhqrrl4K]!:!!  OO- . . . .r') r-r.r/r0rr1r3rr4r5rr,r6r7s@r&rGrGVs   +E#s(O +# +QT +ei +nr + + + + + +$u|*<r'rGcXeZdZdZfdZdZdZdZdej dej fdZ xZ S) UperNetHeadz Unified Perceptual Parsing for Scene Understanding. This head is the implementation of [UPerNet](https://arxiv.org/abs/1807.10221). ct||_|j|_||_|j|_d|_tj |j|j d|_ t|j|jd|j|j|_ t|jdt|j|jzz|jdd|_tj|_tj|_|jddD]j}t||jd}t|j|jdd}|j||j|ktt|j|jz|jdd|_dS)NFrr=)rIr rr)rrconfigrHr hidden_sizer;rIrr num_labels classifierrG psp_modulesrlen bottleneck ModuleList lateral_convs fpn_convsrLfpn_bottleneck)r$r`rl_convfpn_convr%s r&rzUperNetHead.__init__s  !-&* ")DM63DRSTTT7    R M,     ,  R 3t'7#8#84=#H H M     ]__+CRC0 , ,K&{DMqQQQF( ST^_```H   % %f - - - N ! !( + + + +/  ! !DM 1 M    r'c:||jdSr*apply _init_weightsr$s r& init_weightszUperNetHead.init_weights 4%&&&&&r'ct|tjrR|jjd|jj|j"|jj dSdSdSNg)meanstd isinstancerrweightdatanormal_r`initializer_rangerzero_r$modules r&rpzUperNetHead._init_weightsn fbi ( ( ) M  & &CT[5R & S S S{&  &&((((( ) )&&r'c|d}|g}|||tj|d}||}|S)Nr^rdim)extendrdr4catrf)r$inputsrNpsp_outsr+s r& psp_forwardzUperNetHead.psp_forwards\ 2J3((++,,,9X1---** r'encoder_hidden_statesrcBfdtjDt }t |dz ddD]Z}|dz jdd}|dz tj ||dj z|dz <[fdt |dz D}|dt |dz ddD]F}tj |||djdddj ||<Gtj |d } |}|}|S) Nc8g|]\}}||Sr).0rB lateral_convrs r& z'UperNetHead.forward..s-pppq,LL!6q!9::pppr'rrr^rPrQrRcHg|]}j||Sr)ri)rrBlateralsr$s r&rz'UperNetHead.forward..s/\\\q%DN1%hqk22\\\r'r)r@rhrLrrerangeshaperrUrVrIr4rrjrc)r$rused_backbone_levelsrB prev_shapefpn_outsr+rs`` @r&r,zUperNetHead.forwardsppppR[\`\nRoRoppp(()>??@@@ #8}}+a/B77  A!!a%.qrr2J&q1uo 0I0I *:TM_1J11HQUOO ]\\\\EBVYZBZ<[<[\\\ %%%+a/B77  A-33 (1+"3ABB"7jX\Xj4HQKK9X1---$$X..(( r') r-r.r/r0rrrrprr4r5r,r6r7s@r&r\r\|s % % % % % N'''))) U\elr'r\c eZdZdZ ddededeeeeeffdd ffd Zd Zd Z d e j de j fdZ xZ S)UperNetFCNHeada Fully Convolution Networks for Semantic Segmentation. This head is the implementation of [FCNNet](https://arxiv.org/abs/1411.4038>). Args: config: Configuration. in_channels (int): Number of input channels. kernel_size (int): The kernel size for convs in the head. Default: 3. dilation (int): The dilation rate for convs in the head. Default: 1. rPr rin_indexrrrNc t||_|j|_|j|_|j|_|j |_ ||_ |dz|z}g}| t|j|j|||t|jdz D]3}| t|j|j|||4|jdkrtj|_ntj||_|j r-t|j|jz|j||dz|_tj|j|jd|_dS)NrP)rrrrrr_r=)rrr`auxiliary_in_channelsrauxiliary_channelsr;auxiliary_num_convs num_convsauxiliary_concat_input concat_inputrrLrrrIdentityconvs Sequentialconv_catrrbrc) r$r`rrr conv_paddingrrBr%s r&rzUperNetFCNHead.__init__s  !71 3"9  #q(H4   $-[R^iq      t~)**  A LL!M4=kS_jr     >Q  DJJ.DJ   - 4=0$-[bmqrbrDM)DM63DRSTTTr'c:||jdSr*rnrqs r&rrzUperNetFCNHead.init_weights rsr'ct|tjrR|jjd|jj|j"|jj dSdSdSrurxrs r&rpzUperNetFCNHead._init_weightsrr'rc||j}||}|jr+|t j||gd}||}|S)Nrr)rrrrr4rrc)r$r hidden_statesr+s r&r,zUperNetFCNHead.forwardsf-dm< M**   N]]59mV-D!#L#L#LMMF(( r')rPr r)r-r.r/r0r1rrrrrrpr4r5r,r6r7s@r&rrs   hi"U"U #"U69"UINsTYZ]_bZbTcOcId"U "U"U"U"U"U"UH'''))) U\elr'rc*eZdZdZeZdZgZdZdZ dS)UperNetPreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. pixel_valuesct|trT|j|j|j|jdSdSdSr*)ryrbackbonerr decode_headauxiliary_headrs r&rpz$UperNetPreTrainedModel._init_weights*st f4 5 5 5 O ( ( * * *   + + - - -$0%2244444  5 510r'c|j|j|j|jdSdS)zInitialize the weightsN)rrrrrrqs r&rrz#UperNetPreTrainedModel.init_weights1sW ""$$$ %%'''   *   , , . . . . . + *r'N) r-r.r/r0r config_classmain_input_name_no_split_modulesrprrrr'r&rr sN !L$O555/////r'raI Parameters: This model is a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) sub-class. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. config ([`UperNetConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. ax Args: pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Pixel values. Padding will be ignored by default should you provide it. Pixel values can be obtained using [`AutoImageProcessor`]. See [`SegformerImageProcessor.__call__`] for details. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers in case the backbone has them. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers of the backbone. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. zMUperNet framework leveraging any vision backbone e.g. for ADE20k, CityScapes.ceZdZfdZeedeee  d de e j de e de e de e j d e e d eeeff d ZxZS) UperNetForSemanticSegmentationct|t||_t ||jj|_|jrt|nd|_ | dS)N)r) rrrrr\r;ruse_auxiliary_headrr post_init)r$r`r%s r&rz'UperNetForSemanticSegmentation.__init__Xs|    %f-- 'v4=;QRRR8>8Q[nV444W[ r'zbatch_size, sequence_length) output_typerNroutput_attentionsoutput_hidden_stateslabels return_dictrc6||jjdkrtd||n |jj}||n |jj}||n |jj}|j|||}|j}| |}tj ||j dddd}d} |jE||} tj | |j dddd} d} |Ft|jj } | ||} | | | |} | |jj| zz } |s)|r|f|ddz} n|f|ddz} | | f| zn| St%| ||j|j S) a labels (`torch.LongTensor` of shape `(batch_size, height, width)`, *optional*): Ground truth semantic segmentation maps for computing the loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels > 1`, a classification loss is computed (Cross-Entropy). Returns: Examples: ```python >>> from transformers import AutoImageProcessor, UperNetForSemanticSegmentation >>> from PIL import Image >>> from huggingface_hub import hf_hub_download >>> image_processor = AutoImageProcessor.from_pretrained("openmmlab/upernet-convnext-tiny") >>> model = UperNetForSemanticSegmentation.from_pretrained("openmmlab/upernet-convnext-tiny") >>> filepath = hf_hub_download( ... repo_id="hf-internal-testing/fixtures_ade20k", filename="ADE_val_00000001.jpg", repo_type="dataset" ... ) >>> image = Image.open(filepath).convert("RGB") >>> inputs = image_processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> logits = outputs.logits # shape (batch_size, num_labels, height, width) >>> list(logits.shape) [1, 150, 512, 512] ```Nrz/The number of labels should be greater than one)rrrPrQFrR) ignore_index)losslogitsr attentions)r`rb ValueErroruse_return_dictrrrforward_with_filtered_kwargs feature_mapsrrrUrVrrrloss_ignore_indexauxiliary_loss_weightr rr)r$rrrrroutputsfeaturesrauxiliary_logitsrloss_fctauxiliary_lossr+s r&r,z&UperNetForSemanticSegmentation.forwardds N  $+"8A"="=NOO O%0%R 'tn R & R d^ R  u-- .R R R bakjR R R R R r'r)!r0typingrrrrr4rtorch.nnrmodeling_outputsr modeling_utilsr utilsr r rutils.backbone_utilsrconfiguration_upernetrrModulerr9rGr\rrUPERNET_START_DOCSTRINGrrrr'r&rsWyx//////////// %%%%%%777777------kkkkkkkkkk111111000000"        F"#####")###LZZZZZ")ZZZzDDDDDRYDDDN/////_///2  Wa a a a a %;a a a a a r'