g#vdZddlZddlmZmZmZddlmZddlm Z m Z m Z m Z m Z ddlmZGdd eZdS) z! Processor class for LayoutLMv3. N)ListOptionalUnion)ProcessorMixin) BatchEncodingPaddingStrategyPreTokenizedInput TextInputTruncationStrategy) TensorTypec(ReZdZdZddgZdZdZd%fd Z d&d ee e e e e e fd e ee e e fd ee e e e e e e fde ee e e e e fdedeeeefdeeeefde e de de e de ede edededededede eeefdef&dZdZd Zd!Zed"Zed#Zed$ZxZS)'LayoutLMv3Processorav Constructs a LayoutLMv3 processor which combines a LayoutLMv3 image processor and a LayoutLMv3 tokenizer into a single processor. [`LayoutLMv3Processor`] offers all the functionalities you need to prepare data for the model. It first uses [`LayoutLMv3ImageProcessor`] to resize and normalize document images, and optionally applies OCR to get words and normalized bounding boxes. These are then provided to [`LayoutLMv3Tokenizer`] or [`LayoutLMv3TokenizerFast`], which turns the words and bounding boxes into token-level `input_ids`, `attention_mask`, `token_type_ids`, `bbox`. Optionally, one can provide integer `word_labels`, which are turned into token-level `labels` for token classification tasks (such as FUNSD, CORD). Args: image_processor (`LayoutLMv3ImageProcessor`, *optional*): An instance of [`LayoutLMv3ImageProcessor`]. The image processor is a required input. tokenizer (`LayoutLMv3Tokenizer` or `LayoutLMv3TokenizerFast`, *optional*): An instance of [`LayoutLMv3Tokenizer`] or [`LayoutLMv3TokenizerFast`]. The tokenizer is a required input. image_processor tokenizerLayoutLMv3ImageProcessor)LayoutLMv3TokenizerLayoutLMv3TokenizerFastNc d}d|vr/tjdt|d}||n|}|t d|t dt ||dS)Nfeature_extractorzhThe `feature_extractor` argument is deprecated and will be removed in v5, use `image_processor` instead.z)You need to specify an `image_processor`.z"You need to specify a `tokenizer`.)warningswarn FutureWarningpop ValueErrorsuper__init__)selfrrkwargsr __class__s p/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/models/layoutlmv3/processing_layoutlmv3.pyrzLayoutLMv3Processor.__init__3s  & ( ( M    !' +> ? ? -<-H//N_  "HII I  ABB B )44444TFrtext text_pairboxes word_labelsadd_special_tokenspadding truncation max_lengthstridepad_to_multiple_ofreturn_token_type_idsreturn_attention_maskreturn_overflowing_tokensreturn_special_tokens_maskreturn_offsets_mapping return_lengthverbosereturn_tensorsreturnc 2|jjr|td|jjr|td|||}|.|jjr"| t|tr|g}|d}|jdid||n|dd||ndd||n|dd |d |d |d |d | d| d| d| d| d|d|d|d|d|d||}|d}|dur|||d}||d<|S)aF This method first forwards the `images` argument to [`~LayoutLMv3ImageProcessor.__call__`]. In case [`LayoutLMv3ImageProcessor`] was initialized with `apply_ocr` set to `True`, it passes the obtained words and bounding boxes along with the additional arguments to [`~LayoutLMv3Tokenizer.__call__`] and returns the output, together with resized and normalized `pixel_values`. In case [`LayoutLMv3ImageProcessor`] was initialized with `apply_ocr` set to `False`, it passes the words (`text`/``text_pair`) and `boxes` specified by the user along with the additional arguments to [`~LayoutLMv3Tokenizer.__call__`] and returns the output, together with resized and normalized `pixel_values`. Please refer to the docstring of the above two methods for more information. NzdYou cannot provide bounding boxes if you initialized the image processor with apply_ocr set to True.zaYou cannot provide word labels if you initialized the image processor with apply_ocr set to True.)imagesr4wordsr#r$r%r&r'r(r)r*r+r,r-r.r/r0r1r2r3r4 pixel_valuesToverflow_to_sample_mapping)r apply_ocrr isinstancestrrrget_overflowing_images)rr7r#r$r%r&r'r(r)r*r+r,r-r.r/r0r1r2r3r4rfeaturesencoded_inputss r!__call__zLayoutLMv3Processor.__call__EsF   ) u/@v    ) {/Fs  ''vn'UU   4 > 9CT$$$ v )I'   )x/@ #,#8iid !,%%(72C $  21  G  "z "z 6  21 #8"7 #8"7 '@&? (B'A $:#9 (-! "G# $*>'  .n-- $ , ,00Hd9effF)/~&r"cg}|D]}|||t|t|kr/tdt|dt||S)Nz`Expected length of images to be the same as the length of `overflow_to_sample_mapping`, but got z and )appendlenr)rr7r:images_with_overflow sample_idxs r!r?z*LayoutLMv3Processor.get_overflowing_imagess!4 < r r r rrBr?rIrMpropertyrSrVr __classcell__)r s@r!rrs&$[1J6HO555555*_cQU?CCG#'5:;?$(,00404*/+0',#;?)RRI0$y/4HYCZZ[RE"3T:K5L"LMN R T$s)_d4S ?&;;< R eDItDI$>?@ R!RtS/12R$%778RSMRR%SMR (~R (~R$(R %)!R"!%#R$%R&'R(!sJ!78)R, -RRRRh $ $ $<<<666GGXG**X*$$X$$$$$r"r)r[rtypingrrrprocessing_utilsrtokenization_utils_baserr r r r utilsr rr;r"r!rfs((((((((((......wwwwwwwwwwwwwwl$l$l$l$l$.l$l$l$l$l$r"