g2dZddlmZddlmZmZddlZddlmZddlm Z ddl m Z m Z m Z mZdd lmZd d lmZd ZeGd de ZGdde ZGddejZGddejZGddejZGddejZGddejZdZdZe deGddeZdS) zPyTorch ViTMatte model.) dataclass)OptionalTupleN)nn)PreTrainedModel) ModelOutputadd_start_docstrings%add_start_docstrings_to_model_forwardreplace_return_docstrings) load_backbone)VitMatteConfigrceZdZUdZdZeejed<dZ ejed<dZ ee ejed<dZ ee ejed<dS)ImageMattingOutputa Class for outputs of image matting models. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Loss. alphas (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Estimated alpha values. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each stage) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states (also called feature maps) of the model at the output of each stage. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, patch_size, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. Nlossalphas hidden_states attentions) __name__ __module__ __qualname____doc__rrtorch FloatTensor__annotations__rrrrj/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/models/vitmatte/modeling_vitmatte.pyrr&s()-D(5$ %,,, $FE $$$8 __class__s rrAzVitMatteBasicConv3x3.__init__Yso I#%     .6;PQQQGII rc||}||}||}|SN)rBrErGr/ hidden_states rforwardzVitMatteBasicConv3x3.forwardfs;yy.. |44 yy.. r)r8rrrrrrArM __classcell__rHs@rr7r7TsV      rr7c(eZdZdZfdZdZxZS)VitMatteConvStreamzc Simple ConvStream containing a series of basic conv3x3 layers to extract detail features. ctd}|j |jj}|j}t j|_|g|z|_tt|jdz D]H}|j|}|j|dz}|j t|||IdS)Nr) r@rAbackbone_config num_channelsconvstream_hidden_sizesr ModuleListconvs conv_chansrangelenappendr7)r/r+r:r;iin_chan_ out_chan_rHs rrAzVitMatteConvStream.__init__ss   ! - 0=K5 ]__ &-,6s4?++a/00 Q QAq)HA.I J  268YOO P P P P Q Qrcd|i}|}tt|jD]2}|j||}dt|dzz}|||<3|S)Ndetailed_feature_map_0detailed_feature_map_r)r[r\rYstr)r/r"out_dict embeddingsr^name_s rrMzVitMatteConvStream.forwardsi,l;! s4:'' ) )A&Az22J+c!a%jj8E(HUOOrrNrPs@rrRrRnsVQQQQQ&rrRc(eZdZdZfdZdZxZS)VitMatteFusionBlockz\ Simple fusion block to fuse features from ConvStream and Plain Vision Transformer. cztt|||dd|_dS)Nr)r=r>)r@rAr7rB)r/r+r:r;rHs rrAzVitMatteFusionBlock.__init__s9 (lST^_``` rctj|ddd}tj||gd}||}|S)Nr8bilinearF) scale_factormode align_cornersr)dim)r functional interpolatercatrB)r/featuresdetailed_feature_mapupscaled_featuresouts rrMzVitMatteFusionBlock.forwardsRM55hQU_ot5uui-/@AqIIIiinn rrNrPs@rririsVaaaaarric(eZdZdZfdZdZxZS) VitMatteHeadzJ Simple Matting Head, containing only conv3x3 and conv1x1 layers. c Bt|jd}d}tjtj||dddtj|tjdtj|dddd|_dS)Nrr)r<r=r>Tr) r@rAfusion_hidden_sizesr Sequentialr'rCrF matting_convs)r/r+r: mid_channelsrHs rrAzVitMatteHead.__init__s 04  ] Ik  v) * *c&2P.Q.QTU.U U Uq  ,V44/4]__ & 23f6PPs4/001455  A   % %#! $ 4Q 7$/APQE(:S S!%!5a!e!<    )00rcT||}tt|jD]I}dt t|j|z dz z}|j||||}Jt j||}|S)Nrcr)rr[r\rrdrsigmoidr)r/rtr"detail_featuresr^detailed_feature_map_namers rrMz#VitMatteDetailCaptureModule.forwards//,77s4-..// c cA(?#c$J\F]F]`aFadeFeBfBf(f %,t)!,XG`7abbHHt00::;; rrNrPs@rrrsQ111112rraI Parameters: This model is a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) sub-class. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. config ([`UperNetConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. aw Args: pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Pixel values. Padding will be ignored by default should you provide it. Pixel values can be obtained using [`AutoImageProcessor`]. See [`VitMatteImageProcessor.__call__`] for details. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers in case the backbone has them. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers of the backbone. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. zNViTMatte framework leveraging any vision backbone e.g. for ADE20k, CityScapes.ceZdZfdZeedeee  d de e j de e de e de e j d e e f d ZxZS) VitMatteForImageMattingct|||_t||_t ||_|dSrJ)r@rAr+r backbonerdecoder post_init)r/r+rHs rrAz VitMatteForImageMatting.__init__sX     %f-- 26::  rzbatch_size, sequence_length) output_typer2Nr"output_attentionsoutput_hidden_stateslabels return_dictcv||n |jj}||n |jj}||n |jj}d}|t d|j|||}|jd}|||} |s| f|ddz} ||f| zn| St|| |j |j S)aJ labels (`torch.LongTensor` of shape `(batch_size, height, width)`, *optional*): Ground truth image matting for computing the loss. Returns: Examples: ```python >>> from transformers import VitMatteImageProcessor, VitMatteForImageMatting >>> import torch >>> from PIL import Image >>> from huggingface_hub import hf_hub_download >>> processor = VitMatteImageProcessor.from_pretrained("hustvl/vitmatte-small-composition-1k") >>> model = VitMatteForImageMatting.from_pretrained("hustvl/vitmatte-small-composition-1k") >>> filepath = hf_hub_download( ... repo_id="hf-internal-testing/image-matting-fixtures", filename="image.png", repo_type="dataset" ... ) >>> image = Image.open(filepath).convert("RGB") >>> filepath = hf_hub_download( ... repo_id="hf-internal-testing/image-matting-fixtures", filename="trimap.png", repo_type="dataset" ... ) >>> trimap = Image.open(filepath).convert("L") >>> # prepare image + trimap for the model >>> inputs = processor(images=image, trimaps=trimap, return_tensors="pt") >>> with torch.no_grad(): ... alphas = model(**inputs).alphas >>> print(alphas.shape) torch.Size([1, 1, 640, 960]) ```NzTraining is not yet supported)rrr{r)rrrr) r+use_return_dictrrNotImplementedErrorrforward_with_filtered_kwargs feature_mapsrrrr) r/r"rrrrroutputsrtroutputs rrMzVitMatteForImageMatting.forward s X&1%D 'tn D & D d^ D D D ]\lkD D D D D rr)r dataclassesrtypingrrrrmodeling_utilsrutilsr r r r utils.backbone_utilsr configuration_vitmatterrrr!Moduler7rRriryrVITMATTE_START_DOCSTRINGrrrrrrs|!!!!!!"""""""" ------ 211111222222# ::::::: :6)))))o)))$294        F")"290&&&&&")&&&R  XQ Q Q Q Q 5Q Q Q Q Q r