g"6ddlZddlmZmZddlmZmZmZmZm Z m Z ddl m Z m Z er ddlmZddlmZerdd lmZer ddlZdd lmZe jeZee d d Gd de ZdS)N)ListUnion)add_end_docstringsis_tf_availableis_torch_availableis_vision_availableloggingrequires_backends)Pipelinebuild_pipeline_init_args)Image) load_image)'TF_MODEL_FOR_VISION_2_SEQ_MAPPING_NAMES)$MODEL_FOR_VISION_2_SEQ_MAPPING_NAMEST) has_tokenizerhas_image_processorc|eZdZdZfdZd dZd deeeededfffd Z d dZ d Z d Z xZ S)ImageToTextPipelinea Image To Text pipeline using a `AutoModelForVision2Seq`. This pipeline predicts a caption for a given image. Example: ```python >>> from transformers import pipeline >>> captioner = pipeline(model="ydshieh/vit-gpt2-coco-en") >>> captioner("https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png") [{'generated_text': 'two birds are standing next to each other '}] ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial) This image to text pipeline can currently be loaded from pipeline() using the following task identifier: "image-to-text". See the list of available models on [huggingface.co/models](https://huggingface.co/models?pipeline_tag=image-to-text). ctj|i|t|d||jdkrt nt dS)Nvisiontf)super__init__r check_model_type frameworkrr)selfargskwargs __class__s `/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/pipelines/image_to_text.pyrzImageToTextPipeline.__init__Fsg$)&)))$))) 7;~7M7M 3 3Sw     Nci}i}|||d<|tjdt||d<|||d<|*|d|vrtd||||ifS)NpromptzUThe `timeout` argument is deprecated and will be removed in version 5 of Transformerstimeoutmax_new_tokenszp`max_new_tokens` is defined both as an argument and inside `generate_kwargs` argument, please use only 1 version)warningswarn FutureWarning ValueErrorupdate)rr'generate_kwargsr%r&forward_paramspreprocess_paramss r"_sanitize_parametersz(ImageToTextPipeline._sanitize_parametersMs  *0 h '   Mgiv   ,3 i (  %/=N+ ,  &).>/.Q.Q &  ! !/ 2 2 2 ."44r#inputsz Image.Imagec d|vr|d}|tdtj|fi|S)a Assign labels to the image(s) passed as inputs. Args: inputs (`str`, `List[str]`, `PIL.Image` or `List[PIL.Image]`): The pipeline handles three types of images: - A string containing a HTTP(s) link pointing to an image - A string containing a local path to an image - An image loaded in PIL directly The pipeline accepts either a single image or a batch of images. max_new_tokens (`int`, *optional*): The amount of maximum tokens to generate. By default it will use `generate` default. generate_kwargs (`Dict`, *optional*): Pass it to send all of these arguments directly to `generate` allowing full control of this function. Return: A list or a list of list of `dict`: Each result comes as a dictionary with the following key: - **generated_text** (`str`) -- The generated text. imagesNzBCannot call the image-to-text pipeline without an inputs argument!)popr+r__call__)rr1r r!s r"r5zImageToTextPipeline.__call__esT4 v  ZZ))F >abb buww11&111r#ct||}|t|ts tdt |d|jjj}|dkr|||j }|j dkr| |j }| |dj }|j jg|z}tj|d }|d |in|d krC||||j }|j dkr| |j }n|d krs|||j }|j dkr| |j }| ||j }||nTtd|d|||j }|j dkr| |j }|jjjdkr|d|d <|S)N)r&z&Received an invalid text input, got - zy - but expected a single string. Note also that one single text can be provided for conditional image to text generation.git)r3return_tensorsptF)textadd_special_tokensr input_ids pix2struct)r3 header_textr8zvision-encoder-decoder)r8z Model type z- does not support conditional text generation)r isinstancestrr+typemodelconfig model_typeimage_processorrto torch_dtype tokenizerr< cls_token_idtorchtensor unsqueezer,)rimager%r&rD model_inputsr< text_inputss r" preprocesszImageToTextPipeline.preprocesssZ5'222  fc**  oT&\\ooo *5JU""#335QUQ_3`` >T))#/??43C#D#DL NN5NQQ[ !^89IE !L33==a@@ ##[)$<====|++#335feies3tt >T))#/??43C#D#DL777#335QUQ_3`` >T))#/??43C#D#DL"nnVDNnSS ##K0000!!hz!h!h!hiii //uT^/\\L~%%+t/?@@ :  '5 0 0V^(,L %r#c d|vr?t|dtr$td|dDrd|d<d|vr |j|d<||jj}|jj|fi||}|S)Nr<c3K|]}|duV dSN).0xs r" z/ImageToTextPipeline._forward..s&AA!AIAAAAAAr#generation_config)r?listallrXr4rBmain_input_namegenerate)rrNr-r1 model_outputss r"_forwardzImageToTextPipeline._forwards < ' '< 4d;; (AA|K'@AAAAA ()-L % o 5 5373IO/ 0 !!$*"<==+ +FVVlVoVV r#czg}|D]5}d|j|di}||6|S)Ngenerated_textT)skip_special_tokens)rHdecodeappend)rr]records output_idsrecords r" postprocesszImageToTextPipeline.postprocesss[' # #J $."7"7(,#8##F NN6 " " " "r#)NNNNrS)NN)__name__ __module__ __qualname____doc__rr0rr@rr5rPr^rg __classcell__)r!s@r"rr.s,     5555022uS$s)]DDW%WX222222@----^,       r#r)r(typingrrutilsrrrr r r baser rPILr image_utilsrmodels.auto.modeling_tf_autorrJmodels.auto.modeling_autor get_loggerrhloggerrrTr#r"rvs 54444444)((((((?WVVVVVVQLLLPPPPPP  H % %,,4UYZZZ[[eeeee(ee\[eeer#