Ng~^ddlZddlZddlZddlZddlmZddlZddlZddlm Z m Z m Z m Z m Z mZmZddlmZmZddlmZejeZdZdd Zdd Zd Zgd Zd gdd gdd gdZGddZ ddZddZ dS)N)Dict)AttributeProto GraphProto ModelProto NodeProto TensorProtohelper numpy_helper) infer_shapesinfer_shapes_path)versioncd|DS)z| Convert numpy float16 to python int. :param np_list: numpy float16 list :return int_list: python int list c g|]M}tt|dddddNS)HN)intbinviewzfill).0_s \/var/www/html/ai-engine/env/lib/python3.11/site-packages/onnxruntime/transformers/float16.py z%_npfloat16_to_int..%sK D D DqCAFF3KK  $**2.. 2 2 D D D)np_lists r_npfloat16_to_intrs E DG D D DDr"\o>@c d}|tj|dkjddkr|tj|dk}|tj|dk}||kr t d|d|||kr t d|d||tj|dkjddkr|tj|dk}|tj|dk}|| kr!t d|d| || kr!t d|d| tj|d||||}tj|| |d| |}tj|||td||}tj|td|| | |}tj|S)a? Convert float32 numpy array to float16 without changing sign or finiteness. Positive values less than min_positive_val are mapped to min_positive_val. Positive finite values greater than max_finite_val are mapped to max_finite_val. Similar for negative values. NaN, 0, inf, and -inf are unchanged. c<tj||k||kSN)np logical_and)abcs rbetweenz&convert_np_to_float16..between0s~a!eQU+++rrzthe float32 number z will be truncated to infz-inf) r$whereshapemaxminloggerdebugfloatfloat16)np_arraymin_positive_valmax_finite_valr) positive_max positive_min negative_max negative_mins rconvert_np_to_float16r:(sj,,,A&&'-a0144A 6 67;;== A 6 67;;== > ) ) LLc|ccSacc d d d + + + LLe|eeScee f f fA&&'-a0144A 6 67;;== A 6 67;;== N? * * LLd|ddTbSbdd e e e ,, , , LLf|ffTdSdff g g gx8-=>>@PRZ[[Hx!1 18Q??BRART\]]Hx%,,GGYabbHxf x.IIN?\deeH :h  rct|tstdt||jtjkrtj|_|jrOttj |j||}t|}||j dd<g|jdd<|j rEtj|j d}t|||}||_ |S)aConvert tensor float to float16. Args: tensor (TensorProto): the tensor to convert. min_positive_val (float, optional): minimal positive value. Defaults to 1e-7. max_finite_val (float, optional): maximal finite value. Defaults to 1e4. Raises: ValueError: input type is not TensorProto. Returns: TensorProto: the converted tensor. 3Expected input type is an ONNX TensorProto but got Nfloat32dtype) isinstancer ValueErrortype data_typeFLOATFLOAT16 float_datar:r$arrayr int32_dataraw_data frombuffertobytes)tensorr4r5 float16_dataint_list float32_list float16_lists rconvert_tensor_float_to_float16rQJs fk * *_]tTZ||]]^^^ ;,,,&.   &0&:K1L1LN^`nooL(66H#+F aaa #%F aaa ? 5= JJJL0?OQ_``L*2244FO Mrcttj|j}tj|j|j|Sr#)r to_arrayr,r make_tensor_value_infonamerC)rLr,s rmake_value_info_from_tensorrVos/  !& ) ) /E  (f6F N NNr)ArrayFeatureExtractor BinarizerCastMapCategoryMapperDictVectorizerFeatureVectorizerImputer LabelEncoderLinearClassifierLinearRegressor Normalizer OneHotEncoderRandomUniformLike SVMClassifier SVMRegressorScalerTreeEnsembleClassifierTreeEnsembleRegressorZipMapNonMaxSuppressionTopKRoiAlignRangeCumSumMinMaxUpsampler)Resize GroupNorm SkipGroupNormc*eZdZdZdefdZdefdZdS)InitializerTrackerz'Class for keeping track of initializer. initializerc0||_g|_g|_dSr#)rx fp32_nodes fp16_nodes)selfrxs r__init__zInitializerTracker.__init__s&rnodecv|r|j|dS|j|dSr#)rzappendr{)r|r~is_node_blockeds radd_nodezInitializerTracker.add_nodesB  ) O " "4 ( ( ( ( ( O " "4 ( ( ( ( (rN)__name__ __module__ __qualname____doc__rr}rrrrrrwrwsP11K )Y))))))rrwFc |dks Jd|ttjtjjks Jd|in|} t |t r|} tjtj tjdkrz|sxtj tj| 5} | j} t#| | tj| }d}dddn #1swxYwYntj| }t |t&st)dt+|d}|sBtjtj tjd kr t,}n#wxYw|t.}|g}t1|}t1|}t2d |d |d d |d|d|d|g}g}g}g}| ||}||i}t1}t1}d|jjD}d|jjD}t t>rfd|D}fd|D}nsg}g}tA|jjD]!\}}|j|vrdt |z}|||j<|!|jdt |z}|jj"!}|#|||_tHj%|jj&_'tQj)d|jg|gtHj%|g}|jj*+||||!|#tA|jjD]\}}|j|vrdt |z}|||j<|!|jdt |z}|jj"!}|#|||_tHj%|jj&_'tQj)d|g|jgd|g}|jj*+||||!|i}|rg} |D]}!t |!t&r| |!jt |!tXr|!j-D]9}|j.tHj/kr"|j|vsJta|||j<:|!j*D]o}|j|vr tcte|jD],}|j||vr||j||j|<-tcte|jD],}|j||vr||j||j|<-|j3|vp|j|v}"tA|jD]f\}}||vr]|"p>|th5|j3gvo|| 5|j3gv}#||6||#g|"r||O|j3dkr=|j7D]5}$|$jdkr(|$j8tHj/krtHj%|$_8n6|j3dvrd}%|j7D]5}$|$jdkr(d}%|$j8tHj/krtHj%|$_86|j3d vr:|%s8|j7+tQj9dtHj%g|j3thvs |j3| vr!|j7D]}$| |$Z||qt |!ttr| |!j;|!j<D]}| ||!j=#t}|!j=|||!j?D]}t}|||}t |!tXrtjA|!j|!j|!j"D]}|jj&j'tHj/kr9|j|vr0tHj%|jj&_'|||jBd!rl|jjCj'j&j'tHj/krC|j|vr:tHj%|jjCj'j&_'||| }||DD]x}&|s|&jErmt}|&j-|||&_-|t|&j-|&jGr$|s"t2Hd"|&jEy|D]1}'tA|'jD]\}}|th|'j3vs|| 5|'j3gvr8|D]}(||(jkr|jj"!}|#|(|'jd#zt |z}||_tHj/|jj&_'|'jd$zt |z}tQj)d|g|gd|g}|jj*+|||'j|<nސ3| r tHjIn tHj/})|D]}'tcte|'jD]}|'j|}|D]}(||(jkr|jj"!}|#|(|'jd#zt |z}||_|)|jj&_'|'jd$zt |z}tQj)d|g|g|)|g}|jj*+|||'j|<nԌtcte|'jD]}|'j|}*|D]}(|*|(jkr|jj"!}|#|(|'jd%zt |z}||_|)|jj&_'|'jd&zt |z}tQj)d|g|*gd'|g}|jj*+|||'j|<nԌ|S)(aConvert tensor float type in the input ONNX model to tensor float16. Args: model (ModelProto or str): The ONNX model or path of the model to convert. min_positive_val (float, optional): minimal positive value. Defaults to 5.96e-08. max_finite_val (float, optional): maximal finite value of float16. Defaults to 65504. keep_io_types (Union[bool, List[str]], optional): It could be boolean or a list of float32 input/output names. If True, model inputs/outputs should be left as float32. Defaults to False. disable_shape_infer (bool, optional): Skips running onnx shape/type inference. Useful if shape inference has been done. Defaults to False. op_block_list (List[str], optional): List of op types to leave as float32. Defaults to None, which will use `float16.DEFAULT_OP_BLOCK_LIST`. node_block_list (List[str], optional): List of node names to leave as float32. Defaults to None. force_fp16_initializers(bool): force converting all float initializers to float16. Default to false, which will convert only the one needed to avoid precision loss. force_fp16_inputs(Dict[str, List[int]]): Force the conversion of the inputs of some operators to float16, even if this script's preference it to keep them in float32. Raises: ValueError: input type is not ModelProto. Returns: ModelProto: converted model. rzginvalid min_positive_val. smallest positive float16 value: subnormal 5.96e-08, and normalized 6.104e-05z4invalid max_finite_val. largest float16 value: 65504Nz1.8.0)dirTz$Expected an ONNX ModelProto but got z1.2.0z"fp16 parameters: min_positive_val=z max_finite_val=z keep_io_types=z disable_shape_infer=z op_block_list=z node_block_list=z force_fp16_initializers=cZg|](}|jjjtjk!|j)SrrB tensor_type elem_typerrDrUrns rrz,convert_float_to_float16..s0jjja8J8TXcXi8i8i168i8i8ircZg|](}|jjjtjk!|j)Srrrs rrz,convert_float_to_float16..s0lllq!&:L:VZeZk:k:kAF:k:k:krcg|]}|v| Srrrr keep_io_typess rrz,convert_float_to_float16.. s#DDDQm1C1Cq1C1C1Crcg|]}|v| Srrrs rrz,convert_float_to_float16.. s#FFFa1 3E3E3E3E3Ergraph_input_cast_graph_input_castCast)torUgraph_output_cast_graph_output_castrrr)EyeLike Multinomial RandomNormalRandomNormalLike RandomUniformrc SequenceEmpty BernoulliFr?)rrr sequence_typezXinitializer is used by both fp32 and fp16 nodes. Consider add these nodes to block list: _input_cast_ _input_cast _output_cast_ _output_cast )Jr1r$finfor2r-r@strr parseonnx __version__tempfileNamedTemporaryFileospathdirnamerUr loadrrArBr DEFAULT_OP_BLOCK_LISTsetr/r0rgraphinputoutputlist enumerateadd value_infoCopyFromrrErrr make_noder~extendrrxrCrDrwrangelenop_typeALWAYS_FLOAT_INPUTSgetr attributeimake_attributerggraphstrQtensors itertoolschainHasFieldrvaluesr{rVrzinfoBFLOAT16)+modelr4r5rdisable_shape_infer op_block_listnode_block_listforce_fp16_initializersforce_fp16_inputs#use_bfloat16_as_blocked_nodes_dtypeforce_fp16_inputs_dict model_pathtmpfileshape_infer_model_pathfunc_infer_shapequeuevalue_info_list node_listmixed_float_type_node_list name_mappinggraph_io_to_skipio_casts fp32_inputs fp32_outputsrr output_name node_namenew_value_infonew_node input_namefp32_initializers next_levelqruse_fp32_weightattr has_dtypevaluer~r accuracy_typers+ ` rconvert_float_to_float16rsJ H$$$p %$$ U28BJ#7#7#;<< < < <>t < < <#4# 2 2 2 LL # # #%+,--$$1 6\ ! !-A6J#-L   ( ( (+c!ff4I"[37799N  # #A & & &",N 8C8KN  + 5(*xAT]^^^_H K  # #H - - -  " "> 2 2 2 LL # # #79 b ^ :^ :A!Z(( +!!!'***!Z((@ AJJA{k&777 v->>>>>4Fq4I4I)!&1:A:AAv)) "3qw<<00BB71:55)5agaj)AAGAJ"3qx==11DD8A;,66*6qx{*CAHQK&'i=&@&]AFoD]O)217););WW :%):::.=/ !%8%<% J T / 6 6q 9 9 9E bH#))++ " e&6  ?@QSces t tE   " "#>u?P#Q#Q R R R (?  BotoBB +&tz22  MAz+DL999QBXB\B\]a]ikmBnBn=n=n-   00%*[%;%?%?%A%AN"++J777"&)n"zexternal data not loaded!)r@rrArBrCrDrFr$rGrIrJ RuntimeErrorr:amaxabsr=)rLr4r5 float32_datarMs rfloat_to_float16_max_diffrs fk * *_]tTZ||]]^^^ ;,,,>???L 3x 122  G}V_IFFF 6777(7GXXL 726,L)A)AABB C CCr)rr ) rr FFNNFNF)!rloggingrrtypingrnumpyr$rrrrrrr r onnx.shape_inferencer r packagingr getLoggerrr/rr:rQrVrrrwrrrrrrs  eeeeeeeeeeeeeeeeee@@@@@@@@  8 $ $EEE    D""""JOOO B#$Aq6QPQFSS ) ) ) ) ) ) ) )"!(-yyyyx DDDDDDr