g&(dZddlZddlmZmZddlmZddlmZddl m Z m Z m Z ddl mZmZmZmZdd lmZd d lmZejeZGd d e dZGdde ZdS)zq Processor class for InstructBLIP. Largely copy of Blip2Processor with addition of a tokenizer for the Q-Former. N)ListUnion) BatchFeature) ImageInput)ProcessingKwargsProcessorMixinUnpack) AddedToken BatchEncodingPreTokenizedInput TextInput)logging) AutoTokenizerc ,eZdZdddddddddd idZdS)InstructBlipProcessorKwargsTFr) add_special_tokenspaddingstridereturn_overflowing_tokensreturn_special_tokens_maskreturn_offsets_mappingreturn_token_type_ids return_lengthverbose) text_kwargs images_kwargsN)__name__ __module__ __qualname__ _defaultst/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/models/instructblip/processing_instructblip.pyrr&sF#').*/&+%*"     IIIr$rF)totalc eZdZdZgdZdgZdZdZdZdfd Z dde d e e e ee ee fd eed efd Zd ZdZedZfdZefdZxZS)InstructBlipProcessora Constructs an InstructBLIP processor which wraps a BLIP image processor and a LLaMa/T5 tokenizer into a single processor. [`InstructBlipProcessor`] offers all the functionalities of [`BlipImageProcessor`] and [`AutoTokenizer`]. See the docstring of [`~BlipProcessor.__call__`] and [`~BlipProcessor.decode`] for more information. Args: image_processor (`BlipImageProcessor`): An instance of [`BlipImageProcessor`]. The image processor is a required input. tokenizer (`AutoTokenizer`): An instance of ['PreTrainedTokenizer`]. The tokenizer is a required input. qformer_tokenizer (`AutoTokenizer`): An instance of ['PreTrainedTokenizer`]. The Q-Former tokenizer is a required input. num_query_tokens (`int`, *optional*):" Number of tokens used by the Qformer as queries, should be same as in model's config. )image_processor tokenizerqformer_tokenizernum_query_tokensBlipImageProcessorrNc tddd|_||jgd||_t |||dS)NzFT) normalizedspecial)special_tokens)r image_token add_tokensr,super__init__)selfr)r*r+r,kwargs __class__s r%r5zInstructBlipProcessor.__init__Psd%iE4PPPd./EEE 0 )5FGGGGGr$imagestextr7returnc ||td|jtfd|jji|}t }|t |tr|g}n?t |ts*t |dtstd|d dd}|j|fi|dddi} ||dd<|j p|ni} |j j |j z} || gt|zdd } | D]+} d t| | | | D| | <,n| } |td t#| | } || |j|fi|d}| d |d<| d|d<|)|j|fi|d}|||S)a This method uses [`BlipImageProcessor.__call__`] method to prepare image(s) for the model, and [`BertTokenizerFast.__call__`] to prepare text for the model. Please refer to the docstring of the above two methods for more information. Args: images (`ImageInput`): The image or batch of images to be prepared. Each image can be a PIL image, NumPy array or PyTorch tensor. Both channels-first and channels-last formats are supported. text (`TextInput`, `PreTokenizedInput`, `List[TextInput]`, `List[PreTokenizedInput]`): The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings (pretokenized string). If the sequences are provided as list of strings (pretokenized), you must set `is_split_into_words=True` (to lift the ambiguity with a batch of sequences). Nz,You have to specify at least images or text.tokenizer_init_kwargsrzAInvalid input text. Please provide a string, or a list of stringsrreturn_tensorsF)rr>cg|] \}}||z Sr#r#).0 img_encoding txt_encodings r% z2InstructBlipProcessor.__call__..s1(((6L,%|3(((r$aAExpanding inputs for image tokens in InstructBLIP should be done in processing. Please follow instruction here (https://gist.github.com/zucchini-nlp/e9f20b054fa322f84ac9311d9ab67042) to update your InstructBLIP model. Using processors without these attributes in the config is deprecated and will throw an error in v4.47.) tensor_type input_idsqformer_input_idsattention_maskqformer_attention_maskr) ValueError _merge_kwargsrr* init_kwargsr isinstancestrlistpopr,r2contentlenziplogger warning_oncer updater+r))r6r9r:audiovideosr7 output_kwargsencodingr>_text_encoding text_encoding image_tokensimage_token_encodingkqformer_text_encodingimage_encodings r%__call__zInstructBlipProcessor.__call__Vs, >dlKLL L** '  "&."<    >>  $$$ fvd++ fJtAw4L4L f !deee+=9==>NPTUUN+T^DffM-4PffaefffN=KM- ()9 :$0V5G " #/7$:OO '+~~!NSYY.5Y](6(($(A((:=>RST>UWefgWh:i:i(((M!$$ !/ %''B*-^TTTM OOM * * *$:D$:4$`$`=Q^C_$`$` !,A,E,Ek,R,RH( )1F1J1JK[1\1\H- .  1T1&[[M/>. ) ) db~bbbcc c NT2222!#n>Q!R!R ../EFFF.@  8 O " "#6 7 7 7)%'').CCFCC  5 OO 34 4OOr$c tj|fi|}t|tr|d}t j|d}||_|S)Nrr+) subfolder)r4from_pretrainedrLtuplerr+)clspretrained_model_name_or_pathr7 processorr+r8s r%rz%InstructBlipProcessor.from_pretrainedse+EGG+,ITTVTT  i ' ' %!! I)9:Wcvwww&7 #r$ri)NNNN)rr r!__doc__rw valid_kwargsimage_processor_classtokenizer_classqformer_tokenizer_classr5rrrr rr rrrarcrgpropertyrjrv classmethodr __classcell__)r8s@r%r(r(7sd$GFFJ&'L0%O-HHHHHH"^b MMMI0$y/4HYCZZ[M 45 M MMMM`<<<666XXXX &[r$r()rrqtypingrrimage_processing_utilsr image_utilsrprocessing_utilsrr r tokenization_utils_baser r r rutilsrautor get_loggerrrSrr(r#r$r%rsZ 222222%%%%%%HHHHHHHHHH    H % %"2%"bbbbbNbbbbbr$