gDbdZddlZddlZddlZddlZddlZddlmZddlm Z ddl m Z m Z ddl mZddlmZdd lmZdd lmZdd lmZmZmZmZd d lmZd dlmZmZmZm Z d dl!m"Z"d dl#m$Z$d dl%m&Z&ej'e(Z)egdZ*eee*Z+de,fdZ-GddZ.dS)zAutoProcessor class.N) OrderedDict)PretrainedConfig)get_class_from_dynamic_moduleresolve_trust_remote_code)FeatureExtractionMixin)ImageProcessingMixin)ProcessorMixin)TOKENIZER_CONFIG_FILE)FEATURE_EXTRACTOR_NAMEPROCESSOR_NAMEget_file_from_repologging)_LazyAutoMapping)CONFIG_MAPPING_NAMES AutoConfigmodel_type_to_module_name!replace_list_option_in_docstrings)AutoFeatureExtractor)AutoImageProcessor) AutoTokenizer)C)alignAlignProcessor)altclipAltCLIPProcessor)bark BarkProcessor)blip BlipProcessor)zblip-2Blip2Processor) bridgetowerBridgeTowerProcessor) chameleonChameleonProcessor) chinese_clipChineseCLIPProcessor)clap ClapProcessor)clip CLIPProcessor)clipsegCLIPSegProcessor)clvp ClvpProcessor)flavaFlavaProcessor)fuyu FuyuProcessor)git GitProcessor)zgrounding-dinoGroundingDinoProcessor)groupvitr+)hubertWav2Vec2Processor)ideficsIdeficsProcessor)idefics2Idefics2Processor)idefics3Idefics3Processor) instructblipInstructBlipProcessor)instructblipvideoInstructBlipVideoProcessor)zkosmos-2Kosmos2Processor) layoutlmv2LayoutLMv2Processor) layoutlmv3LayoutLMv3Processor)llavaLlavaProcessor) llava_nextLlavaNextProcessor)llava_next_videoLlavaNextVideoProcessor)llava_onevisionLlavaOnevisionProcessor)markuplmMarkupLMProcessor)mctctMCTCTProcessor)zmgp-strMgpstrProcessor)mllamaMllamaProcessor) oneformerOneFormerProcessor)owlv2Owlv2Processor)owlvitOwlViTProcessor) paligemmaPaliGemmaProcessor) pix2structPix2StructProcessor)pixtralPixtralProcessor) pop2pianoPop2PianoProcessor) qwen2_audioQwen2AudioProcessor)qwen2_vlQwen2VLProcessor)sam SamProcessor) seamless_m4tSeamlessM4TProcessor)sewr9)zsew-dr9)siglipSiglipProcessor)speech_to_textSpeech2TextProcessor)speech_to_text_2Speech2Text2Processor)speecht5SpeechT5Processor)trocrTrOCRProcessor)tvlt TvltProcessor)tvp TvpProcessor)udop UdopProcessor) unispeechr9)z unispeech-satr9) video_llavaVideoLlavaProcessor)vilt ViltProcessor)vipllavarJ)zvision-text-dual-encoderVisionTextDualEncoderProcessor)wav2vec2r9)z wav2vec2-bertr9)zwav2vec2-conformerr9)wavlmr9)whisperWhisperProcessor)xclipXCLIPProcessor class_namectD]S\}}||vrJt|}tjd|d} t ||cS#t $rYOwxYwTtj D]}t |dd|kr|cStjd}t||rt ||SdS)N.ztransformers.models__name__ transformers) PROCESSOR_MAPPING_NAMESitemsr importlib import_modulegetattrAttributeErrorPROCESSOR_MAPPING_extra_contentvalueshasattr)r module_name processorsmodule processor main_modules d/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/models/auto/processing_auto.pyprocessor_class_from_namerys#:#@#@#B#B Z  # #3K@@K,->->->@UVVF vz22222!     $'5<<>> 9j$ / /: = =    > ).99K{J''0{J/// 4s A A*)A*cheZdZdZdZeeedZe ddZ dS) AutoProcessora This is a generic processor class that will be instantiated as one of the processor classes of the library when created with the [`AutoProcessor.from_pretrained`] class method. This class cannot be instantiated directly using `__init__()` (throws an error). c td)Nz}AutoProcessor is designed to be instantiated using the `AutoProcessor.from_pretrained(pretrained_model_name_or_path)` method.)EnvironmentError)selfs r__init__zAutoProcessor.__init__s _   c dd}|Dtjdtddt d|d<dd}dd}dd <d}d}fd t jtj D}t|tfi|} | Qtj |fi\} } | d d}d | d ivr| d d }|t|tfi|} | Qtj|fi\} } | d d}d | d ivr| d d }| S|Qt#j|fi\} } | d d}d | d ivr| d d }|t|t&fi|} | yt)| d5}t+j|} dddn #1swxYwY| d d}d | d ivr| d d }|`t/|t0st3j|fd|i}t7|d d}t9|d rd |jvr |jd }|t=|}|du}|dupt?|t@v}tC||||}|ri|rgtE||fi}dd} tFj$%|r|&|j|fd|iS||j|fd|iSt?|t@vr%tAt?|j|fiS tOj|fd|iS#tP$rQ tSj|fd|icYS#tP$rYnwxYw tUj|fd|icYS#tP$rYnwxYwYnwxYwt d|d)a Instantiate one of the processor classes of the library from a pretrained model vocabulary. The processor class to instantiate is selected based on the `model_type` property of the config object (either passed as an argument or loaded from `pretrained_model_name_or_path` if possible): List options Params: pretrained_model_name_or_path (`str` or `os.PathLike`): This can be either: - a string, the *model id* of a pretrained feature_extractor hosted inside a model repo on huggingface.co. - a path to a *directory* containing a processor files saved using the `save_pretrained()` method, e.g., `./my_model_directory/`. cache_dir (`str` or `os.PathLike`, *optional*): Path to a directory in which a downloaded pretrained model feature extractor should be cached if the standard cache should not be used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force to (re-)download the feature extractor files and override the cached versions if they exist. resume_download: Deprecated and ignored. All downloads are now resumed by default when possible. Will be removed in v5 of Transformers. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. token (`str` or *bool*, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. return_unused_kwargs (`bool`, *optional*, defaults to `False`): If `False`, then this function returns just the final feature extractor object. If `True`, then this functions returns a `Tuple(feature_extractor, unused_kwargs)` where *unused_kwargs* is a dictionary consisting of the key/value pairs whose keys are not feature extractor attributes: i.e., the part of `kwargs` which has not been used to update `feature_extractor` and is otherwise ignored. trust_remote_code (`bool`, *optional*, defaults to `False`): Whether or not to allow for custom models defined on the Hub in their own modeling files. This option should only be set to `True` for repositories you trust and in which you have read the code, as it will execute code present on the Hub on your local machine. kwargs (`Dict[str, Any]`, *optional*): The values in kwargs of any keys which are feature extractor attributes will be used to override the loaded values. Behavior concerning key/value pairs whose keys are *not* feature extractor attributes is controlled by the `return_unused_kwargs` keyword parameter. Passing `token=True` is required when you want to use a private model. Examples: ```python >>> from transformers import AutoProcessor >>> # Download processor from huggingface.co and cache. >>> processor = AutoProcessor.from_pretrained("facebook/wav2vec2-base-960h") >>> # If processor files are in a directory (e.g. processor was saved using *save_pretrained('./test/saved_model/')*) >>> # processor = AutoProcessor.from_pretrained("./test/saved_model/") ```use_auth_tokenNzrThe `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.tokenzV`token` and `use_auth_token` are both specified. Please set only the argument `token`.configtrust_remote_codeT _from_autoc*i|]}|v||Sr).0keykwargss r z1AutoProcessor.from_pretrained..s2% % % !$cfjpcpcpCcpcpcprprocessor_classrauto_mapzutf-8)encoding code_revisionz!Unrecognized processing class in z. Can't instantiate a processor, a tokenizer, an image processor or a feature extractor for this model. Make sure the repository contains the files of at least one of those processing classes.)+popwarningswarn FutureWarningget ValueErrorinspect signaturer parameterskeysr r get_processor_dictr r get_image_processor_dictrget_feature_extractor_dictr openjsonload isinstancerrfrom_pretrainedrrrrtyperrrospathisdirregister_for_auto_classr Exceptionrr)clspretrained_model_name_or_pathrrrrrprocessor_auto_mapget_file_from_repo_kwargsprocessor_config_file config_dict_preprocessor_config_filetokenizer_config_filereaderhas_remote_codehas_local_codes ` rrzAutoProcessor.from_pretrainedsJ $4d;;  % ME   zz'4((4 l-F7OHd++"JJ':DAA#|!% % % % (/(9:L(M(M(X(](](_(_% % % ! !3 )>! ! =V! !  ! ,+>?\gg`fggNK)oo.?FFO+//*b"A"AAA%0%<_%M"  "'9-/E((Ib(( $(3!5!NOl!w!wpv!w!w Q"-//2CT"J"J"kooj"&E&EEE)4Z)@)Q&(38O!7!R1""5;"" Q#.//2CT"J"J"kooj"&E&EEE)4Z)@)Q&  "$6-/D%%Ha%% !%0/'BBB4f"&)F"3"3K444444444444444#.//2CT"J"J"kooj"&E&EEE)4Z)@)Q&  "f&677 #31EVZ` &f.?FFOvz** F&//Q/Q%+__%E"  &7HHO,D8(4YV HY8Y5 ??G G @?[G R U U U\ U U Urr)/rrrrrr collectionsrconfiguration_utilsrdynamic_module_utilsrrfeature_extraction_utilsrimage_processing_utilsr processing_utilsr tokenization_utilsr utilsr r rr auto_factoryrconfiguration_autorrrrfeature_extraction_autorimage_processing_autortokenization_autor get_loggerrloggerrrstrrrrrrrs  ######433333\\\\\\\\>>>>>>::::::......777777XXXXXXXXXXXX****** :99999555555,,,,,,  H % %%+DDDFFP%$%9;RSS#0cUcUcUcUcUcUcUcUcUcUr