g,ddlZddlZddlZddlmZmZddlmZddlm Z m Z m Z m Z m Z mZmZmZmZmZddlZddlmZddlmZmZmZmZdd lmZmZmZe rdd lm Z dd l!m"Z"dd l#m$Z$dd l%m&Z&erddl'm(Z(ej)e*Z+dZ,dZ-ej.GddZ/GddeZ0Gdde0eZ1Gdde1Z2dS)N)ABCabstractmethod) OrderedDict) TYPE_CHECKINGAnyCallableDictIterableListMappingOptionalTupleUnion)version) TensorTypeis_torch_availableis_vision_availablelogging)ParameterFormat compute_effective_axis_dimension"compute_serialized_parameters_size)PretrainedConfigFeatureExtractionMixinImageProcessingMixinPreTrainedTokenizerBase)Image lcfeZdZUdZeed<eed<eed<dZe eed<dZ e eed<dS) PatchingSpeca Data class that holds patching specifications. Args: o: Module / object where the op to patch is located name: Name of the op to monkey patch custom_op: Custom op that patches the original op orig_op: Original op that is being patched op_wrapper: Wrapper (optional) that wraps both the original and custom ops. It is useful for ops that are class or static methods for instance. oname custom_opNorig_op op_wrapper) __name__ __module__ __qualname____doc__r__annotations__strrr(r r)T/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/onnx/config.pyr$r$/sf   FFF III"&GXh &&&%)J")))))r1r$c,eZdZdZdZdZdZejdZ e dddd ie d ddd ie dddd ie ddd ddd ddd d e dddd ie dddd ie dd diie ddd ddd d e ddd ddd de ddddddie dddd ie dd diie dddd ie dddd ie dddd idZ dIddde de efdZedJddde ddfdZeedee eee fffdZedee eee fffdZedeee effd Zedefd!Zedefd"Zedefd#Zedefd$Zedefd%Z ede!fd&Z"e#d'ede!fd(Z$ dKd+ed,ed-ed.efd/Z% dLd+ed3ed4ed5efd6Z& dMd9e'd:d+ed;edee(d,ed.ed-ed3ed4ed5ed?d@dee effdAZ)dBee efdee effdCZ*dDZ+dEZ,edFe dGe-ede.e effdHZ/dS)N OnnxConfigzv Base class for ONNX exportable model describing metadata on how to export the model through the ONNX format. rz1.8logitsbatchsequencerrlast_hidden_state)r7 pred_boxes pred_masksr)r7r<) start_logits end_logits num_labelsheightwidth)rrrdecoder_sequence)z causal-lmdefaultzimage-classificationzimage-segmentationz masked-imz masked-lmmultiple-choicezobject-detectionzquestion-answeringzsemantic-segmentationz seq2seq-lmzsequence-classificationztoken-classificationz vision2seq-lmz speech2seq-lmrENconfigrtaskpatching_specscL||_||jvr+t|d|j||_g|_||ngD]S}|}|j.tj|t|j |j }|j |TdS)Nz+ is not a supported task, supported tasks: )r() _config_tasks_to_common_outputs ValueErrorkeysrH_patching_specsr( dataclassesreplacegetattrr%r&append)selfrGrHrIspec final_specs r2__init__zOnnxConfig.__init__os t4 4 4jjDDaDfDfDhDhjj  !&4&@NNb 4 4DJ|#(0wtvty?Y?YZZZ  ' ' 3 3 3 3  4 4r1returnc|||S)z Instantiate a OnnxConfig for a specific model Args: config: The model's configuration to use when exporting to ONNX Returns: OnnxConfig for this model )rHr0clsrGrHs r2from_model_configzOnnxConfig.from_model_configss6%%%%r1ct)z Mapping containing the axis definition of the input tensors to provide to the model Returns: For each input: its name associated to the axes symbolic name and the axis position within the tensor )NotImplementedErrorrTs r2inputszOnnxConfig.inputss"###r1cN|j|j}tj|S)z Mapping containing the axis definition of the output tensors to provide to the model Returns: For each output: its name associated to the axes symbolic name and the axis position within the tensor )rLrHcopydeepcopy)rTcommon_outputss r2outputszOnnxConfig.outputss#6tyA}^,,,r1c8t|jdrddiSdS)z Dictionary of keys to override in the model's config before exporting Returns: Dictionary with the keys (and their corresponding values) to override use_cacheFN)hasattrrKr_s r2values_overridezOnnxConfig.values_overrides( 4< - - (' 'tr1ctjS)zp The default batch size to use if no other indication Returns: Integer > 0 )r4default_fixed_batchr_s r2default_batch_sizezOnnxConfig.default_batch_sizes --r1ctjS)zu The default sequence length to use if no other indication Returns: Integer > 0 )r4default_fixed_sequencer_s r2default_sequence_lengthz"OnnxConfig.default_sequence_lengths 00r1ctjS)zw The default number of choices to use if no other indication Returns: Integer > 0 )r4default_fixed_num_choicesr_s r2default_num_choiceszOnnxConfig.default_num_choicess 33r1ctS)z{ Which onnx opset to use when exporting the model Returns: Integer ONNX Opset version )DEFAULT_ONNX_OPSETr_s r2default_onnx_opsetzOnnxConfig.default_onnx_opsets "!r1cdS)z What absolute tolerance value to use during model conversion validation. Returns: Float absolute tolerance value. gh㈵>r0r_s r2atol_for_validationzOnnxConfig.atol_for_validations tr1cxtr+ddlm}tj||jkSdS)z The minimum PyTorch version required to export the model. Returns: `bool`: Whether the installed version of PyTorch is compatible with the model. r)get_torch_versionF)rtransformers.utilsryrparsetorch_onnx_minimum_version)rTrys r2is_torch_support_availablez%OnnxConfig.is_torch_support_availablesK     < < < < < <=!2!2!4!4559XX X5r1num_parameterscHt|tjtkS)a Flag indicating if the model requires using external data format Args: num_parameters: Number of parameter on the model Returns: True if model.num_parameters() * size_of(float32) >= 2Gb False otherwise )rrFloatEXTERNAL_DATA_FORMAT_SIZE_LIMIT)r~s r2use_external_data_formatz#OnnxConfig.use_external_data_formats! /~?T U U. / r1rC( batch_size num_channels image_height image_widthcg}t|D]s}tj|||dz}|t j|ddt|S)Nuint8RGB) rangenprandomrandrSr! fromarrayastypeconvert)rTrrrrimages_datas r2_generate_dummy_imagesz!OnnxConfig._generate_dummy_imagess|z"" P PA9>>, \JJSPD MM%/$++g*>*>??GGNN O O O O r1"V@ sampling_rate time_duration frequencyc g}t|D]g}tjd|t||zd}|dtjdtjz|z|zzh|S)NrF)endpointg?r)rrlinspaceintrSsinpi)rTrrrr audio_datarts r2_generate_dummy_audioz OnnxConfig._generate_dummy_audio s z"" G GA A}c--2O.P.P[`aaaA   cBF1ru9y+@1+D$E$EE F F F Fr1F preprocessor)r rr seq_length num_choicesis_pair framework tokenizerr cddlm}ddlm}ddlm}t ||r| td| 6tj dtt d| }t ||rPt|tjd }||}t|tj| }|jt'|jd kr|jnd }d |g|zg|z}|jd krttjd |z}|||}|D]3\}fdt1d t'D||<4t3||St3|||St ||r|jd dkr*td|jjd|jd t|tj}|||| |}t3|||St ||r^|jd dkrMt|tj}|||| |}t3|||St ||r^|jd dkrMt|tj}||| | | }t3|||Std)am Generate inputs to provide to the ONNX exporter for the specific framework Args: preprocessor: ([`PreTrainedTokenizerBase`], [`FeatureExtractionMixin`], or [`ImageProcessingMixin`]): The preprocessor associated with this model configuration. batch_size (`int`, *optional*, defaults to -1): The batch size to export the model for (-1 means dynamic axis). num_choices (`int`, *optional*, defaults to -1): The number of candidate answers provided for multiple choice task (-1 means dynamic axis). seq_length (`int`, *optional*, defaults to -1): The sequence length to export the model for (-1 means dynamic axis). is_pair (`bool`, *optional*, defaults to `False`): Indicate if the input is a pair (sentence 1, sentence 2) framework (`TensorType`, *optional*, defaults to `None`): The framework (PyTorch or TensorFlow) that the tokenizer will generate tensors for. num_channels (`int`, *optional*, defaults to 3): The number of channels of the generated images. image_width (`int`, *optional*, defaults to 40): The width of the generated images. image_height (`int`, *optional*, defaults to 40): The height of the generated images. sampling_rate (`int`, *optional* defaults to 22050) The sampling rate for audio data generation. time_duration (`float`, *optional* defaults to 5.0) Total seconds of sampling for audio data generation. frequency (`int`, *optional* defaults to 220) The desired natural frequency of generated audio. Returns: Mapping[str, Tensor] holding the kwargs to provide to the model's forward function rrrrNzPYou cannot provide both a tokenizer and a preprocessor to generate dummy inputs.ztThe `tokenizer` argument is deprecated and will be removed in version 5 of Transformers. Use `preprocessor` instead.zSOverwriting the `preprocessor` argument with `tokenizer` to generate dummmy inputs.r)fixed_dimensionnum_token_to_add0 rF) text_pairc*g|]}||zSr0r0).0irvs r2 z4OnnxConfig.generate_dummy_inputs..rs')h)h)hQ!AK,?*@)h)h)hr1) tensor_type)return_tensors pixel_valuesz*The `preprocessor` is an image processor (zC) and expects `model_input_names[0]` to be "pixel_values", but got )r)rrinput_featuresz\Unable to generate dummy inputs for the model. Please provide a tokenizer or a preprocessor.) feature_extraction_utilsrimage_processing_utilsrtokenization_utils_baser isinstancerMwarningswarn FutureWarningloggerwarningrr4rknum_special_tokens_to_addrn unk_tokenlenjoinrHrqitemsrdictconvert_to_tensorsmodel_input_names __class__r*rr)rTrrrrrrrrrrrrrrrr token_to_add input_token dummy_inputtokenized_inputkrs ` @r2generate_dummy_inputsz OnnxConfig.generate_dummy_inputss` FEEEEEAAAAAAEEEEEE l$; < < qAVopp p  M+    NNp q q q$L l$; < <8 9J,J]^J(AA'JJL9J,M`lJ !*63|?U;V;VYZ;Z;Z&&  88[M22Z?@:MKy---?1Uhi *K7 ".,{k"R"R"R+1133iiDAq)h)h)h)h)h%PQSVWXSYSY[fJgJg)h)h)hOA&&O>>9>UUVVV [KKKLL L  &: ; ; -a0NBB qAWA`qqMYMklmMnqq :*V`VtuuuJ55j,P\^ijjK K RRRSS S  &< = = ,B`abBcguBuBu9*V`VtuuuJ55j,P\^ijjK K RRRSS S |%; < < AMA_`aAbfvAvAv:*V`VtuuuJ44ZP]_hiiK [KKKLL Ln r1reference_model_inputsc|S)a Generate inputs for ONNX Runtime using the reference model inputs. Override this to run inference with seq2seq models which have the encoder and decoder exported as separate ONNX files. Args: reference_model_inputs ([`Mapping[str, Tensor]`): Reference inputs for the model. Returns: `Mapping[str, Tensor]`: The mapping holding the kwargs to provide to the model's forward function r0)rTrs r2!generate_dummy_inputs_onnxruntimez,OnnxConfig.generate_dummy_inputs_onnxruntimes &%r1c|jD]E}|j|jn||j}t|j|j|FdSN)rOr)r'setattrr%r&)rTrUr's r2 patch_opszOnnxConfig.patch_opssY( 2 2D*./*AtW[WeGfGfI DFDIy 1 1 1 1 2 2r1c|jD]E}|j|jn||j}t|j|j|FdSr)rOr)r(rr%r&)rTrUr(s r2 restore_opszOnnxConfig.restore_opssY( 0 0D&*o&=dll4??SWS_C`C`G DFDIw / / / / 0 0r1r&fieldclddlm}fdt||DS)a Flatten any potential nested structure expanding the name of the field with the index of the element within the structure. Args: name: The name of the nested structure field: The structure to, potentially, be flattened Returns: (Dict[str, Any]): Outputs with flattened structure and key mapping this new structure. r)chainc&i|] \}}d||S).r0)ridxitemr&s r2 zAOnnxConfig.flatten_output_collection_property..s)]]])#t4#]]]r1) itertoolsr enumerate from_iterable)r[r&rrs ` r2"flatten_output_collection_propertyz-OnnxConfig.flatten_output_collection_propertysH $#####]]]]yATATUZA[A[7\7\]]]]r1)rENrE)rrCrr)rrrr) rrrFNrCrrrrrN)0r*r+r,r-rkrnrqrr{r|rrLr/r r$rW classmethodr\propertyrr rr`rer rrirlrorrrufloatrwboolr} staticmethodrrrrrrrrrr r rr0r1r2r4r4Ds~ !!.u!5!5 [(J,G,G!HII; 3Z5P5PQRR + X7z7R7R,S T T)k%*55")j99")j99    ![(J,G,G!HII [(J,G,G!HII&;1g,'?@@'K%*55")j99    *k$+ ; ;")j99    "-hG Yafm8n8n-o!p!p!k8=O-P-P"QRR#.;1g,/G#H#H + X7z7R7R,S T T$hG 0K0K%LMM$hG 0K0K%LMM?  D44144Z^_kZl4444  & &'9 & &Ua & & &[ &$WS#X%6 67$$$^X$-gc3h&7!78---X- '#s(*;!<   X .C...X.1111X14S444X4"C"""X"UX D   X      \ "fh14HK_bmp   25 NS fi     *.""/3vvghvv v  v  vJ'vvvvvvv-v c vvvvp &PSUXPXHY &^efiknfn^o & & & &222 000 ^c^(3-^TXY\^aYaTb^^^[^^^r1r4ceZdZ d dddedeedeffd Zed!ddded dfd Z e d e ee e effffd Z e d ee eeffd Ze d e fdZe d e fdZ d"ddde de dedeed e eeff fd Z d#de ee e effdedefdZdZdedeed eeefffd ZxZS)$OnnxConfigWithPastrENFrGrrHrIuse_pastc^t|||||_dS)N)rHrI)superrWr)rTrGrHrIrrs r2rWzOnnxConfigWithPast.__init__s/ d>JJJ  r1rXc|||dS)z Instantiate a OnnxConfig with `use_past` attribute set to True Args: config: The underlying model's config to use when exporting to ONNX Returns: OnnxConfig with `.use_past = True` T)rHrr0rZs r2 with_pastzOnnxConfigWithPast.with_pastss6t4444r1cjtj}|jr||d|S)Nre direction)rrerfill_with_past_key_values_)rTrdrs r2rezOnnxConfigWithPast.outputss8 = Q  + +Ni + P P Pr1cBt|jdr d|jiSdS)Nrg)rhrKrr_s r2riz"OnnxConfigWithPast.values_overrides( 4< - - 0/ /tr1cbt|jdstd|jjS)z The number of layers attribute retrieved from the model config. Override this for model configs where the number of layers attribute is not called `num_layers`. num_layerszcould not find the number of layers attribute in the model configuration, override the num_layers property of the model OnnxConfig to solve this)rhrKAttributeErrorrr_s r2rzOnnxConfigWithPast.num_layerss= t|\22  B |&&r1cbt|jdstd|jjS)z The number of attention heads attribute retrieved from the model config. Override this for model configs where the number of attention heads attribute is not called `num_attention_heads`. num_attention_headszcould not find the number of attention heads attribute in the model configuration, override the num_attention_heads property of the model OnnxConfig to solve this)rhrKrrr_s r2rz&OnnxConfigWithPast.num_attention_headss> t|%:;;  V |//r1rrr rrrrcXt|||||}|jrtst dddl}|dj\}} | dz} ||j| |jj |jzf} d|vrE|dj } | |d| || | gd |d<g|d <t|jD]E} |d || || fF|S) NrrrrACannot generate dummy past_keys inputs without PyTorch installed.r input_idsrattention_mask)dtyper)dimpast_key_values)rrrrrMtorchshaperrK hidden_sizercatonesrrrSzeros)rTrrrrr common_inputsrr8seqlenpast_key_values_lengthr  mask_dtyperrs r2rz(OnnxConfigWithPast.generate_dummy_inputssg55 *W`i6   = b%''  !deee )+6C]3^3^ ///// 0$ e=DIc9d9d!T"5"5A"5"5"566=DIc9d9d!T"5"5A"5"5"566  e er1cJ|d||d|d<|d||d|d<dS)Nrrrrrr0rTflattened_outputr&rrs r2_flatten_past_key_values_z,OnnxConfigWithPast._flatten_past_key_values_?sH01!D,,3,,,-23A$D..3...///r1r&rci}|dvr.t|D]\}}|||||n"t||}|S)N)rr)rrrr)rTr&rrrrrs r2rz5OnnxConfigWithPast.flatten_output_collection_propertyCsy 1 1 1#E** O OQ../?sANNNN O %wwII$PUVV r1)rENFrrrFN)F)r*r+r,r/r r$rrWrrrr rrer rrirrrrrrr r r __classcell__rs@r2rrs-1 !!"!!\* !  !!!!!! 5 51 5 5Ma 5 5 5[ 5gc3h&7!78X'#s(*;!<X  'C ' ' 'X ' 0S 0 0 0X 0*. ((,(( (  ( J' ( c ((((((Vqvee!(gc3h.?)?!@eMPeimeeee2777 s 8C= UYZ]_bZbUc          r1rc,eZdZedeeeeeffffd Zedeeffd Z edeeffd Z ddd d ed ed e d e e deeeff fd ZdeeeeeffdefdZdZxZS)OnnxSeq2SeqConfigWithPastrXctt|j}|D]6\}}d|vrdnd}|D]\}}d|vr|||<|||<7|jr||d|S)Nencoderencoder_sequencerDr9rer)rrrerrr)rTrdr& axes_names sequence_nameaxis_idxrs r2rez!OnnxSeq2SeqConfigWithPast.outputsOs1488@ . 4 4 6 6 0 0 D*2;t2C2C..I[M","2"2"4"4 0 0$%%+8Jx((,0Jx((  0 = Q  + +Ni + P P Pr1c tj}||f}nb#t$rUt|jdr.t|jdr|jj|jjf}ntdYnwxYw|S)Nencoder_layersdecoder_layerszcould not find the number of encoder and decoder layers attributes in the model configuration, override the num_layers property of the model OnnxConfig to solve this)rrrrhrKr-r.)rTrrs r2rz$OnnxSeq2SeqConfigWithPast.num_layers`s +J$j1JJ   t|%566 74?? GDLZsDtDt '+|'KT\Mq&r##$$# #"r/rFNrr rrrrc tt||||||}|js|nd}tt||||||}d|D}t d i||} |jrt stdddl} | dj d} | dj d} | dj d}|j \} }| | | |j j | zf}| ||dz|j j |zf}g| d <|j \}}t||}t|||z }||krd nd }t!|D]m}| d | || || || |fn|d kr|n|}t!||D]E}| d | || |fF| S) Nrrc i|] \}}d|| S)decoder_r0)rr&tensors r2rzCOnnxSeq2SeqConfigWithPast.generate_dummy_inputs..s'___ f+T++V___r1rrrdecoder_input_idsrCrr'decoderr0)rrrrrrrrMrr rrKr rminmaxrrSr )rTrrrrrencoder_inputsdecoder_seq_lengthdecoder_inputsrrr8encoder_seq_lengthnum_encoder_attention_headsnum_decoder_attention_heads encoder_shape decoder_shapenum_encoder_layersnum_decoder_layersmin_num_layersmax_num_layersremaining_side_namerr rs r2rz/OnnxSeq2SeqConfigWithPast.generate_dummy_inputss1488NN *W`iO   04}CZZ!1488NN *9KU\hqO  `_H\H\H^H^___@@~@@@ =- b%''  !deee !+.4Q7E!.{!;!A!!D !./B!C!I!!L GKG_ D ')D+" (,GG M+"Q& (,GG M02M+ ,59_ 2  2 !35GHHN !35GHH>YN/ADV/V/V))\e >**  /077 M22 M22 M22 M22 &9I%E%EMM=E>>:: b b/077U9K9KU[[Y^M_M_8`aaaar1rrc |dvrtd|d|dkrdnd}|j\}}t||}t|||z }||krdnd}d } |dkrd nd } t |D]:} d | d ||d| d<d | d ||d| d<d | d ||d| d<d | d ||d| d<;t ||D]!} |dkrd | d } nd | d } | ||d| d|d<"dS)Nrrrr`rrr'r8past_encoder_sequencepast_decoder_sequencez past_decoder_sequence + sequencer8rr .decoder.key.decoder.value .encoder.key.encoder.valuer)rMrr9r:r) rTrrr&rCrDrErFrGr(rDr axes_infos r2rz4OnnxSeq2SeqConfigWithPast.fill_with_past_key_values_s 1 1 1iT]iiijj j$-$9$9  y26../1CDD/1CDD~U+=@R+R+RiiXa26?86K6K22Qs~&& _ _A?FK[;\;\ 77777 8AHM]=^=^ 99999 :?FK[;\;\ 77777 8AHM]=^=^ 99999 : :~~66 S SA"i// ',<== ',<== IR EEEE,?EEE F F  S Sr1c|d||d|d<|d||d|d<|d||d|d<|d||d|d <dS) NrrrKrrLrrMrCrNr0rs r2rz3OnnxSeq2SeqConfigWithPast._flatten_past_key_values_s89!D4434445:;A$D663666789!D4434445:;A$D663666777r1r!)r*r+r,rr r/rrerrrrr rrrrrr"r#s@r2r%r%Ns gc3h&7!78X  E#J     X  #U3Z # # # # #X #$*. CC,CC C  C J' C c CCCCCCJSGCQTVYQYIZDZ<[ShkSSSS8???????r1r%)3rbrPrabcrr collectionsrtypingrrrr r r r r rrnumpyr packagingrutilsrrrrrrrconfiguration_utilsrrrrrrr PILr! get_loggerr*rrtr dataclassr$r4rr%r0r1r2r[sx ##############ffffffffffffffffffffffffPPPPPPPPPPPPhhhhhhhhhhB666666AAAAAA======AAAAAA  H % %#9 ********(t^t^t^t^t^t^t^t^n P P P P P SP P P fW?W?W?W?W? 2W?W?W?W?W?r1