NgR-|ddlZddlZddlZddlZddlmZddlmZmZm Z ddl m cm Z ddlmZmZmZmZGddZdeded dfd Zd eded dfd Z d.ded ede ede ede ede ed dfdZ d/d eded e ededed df dZd ed dfdZdeded dfdZded eefdZded eegeefd eefd!Zd"ed eefd#Zded eefd$Z d"ed eefd%Z!ded eefd&Z"d'ed efd(Z#ded efd)Z$ded*ed dfd+Z%d ed,ed efd-Z&dS)0N)chain)CallableIterableOptional)AttributeProto GraphProto ModelProto TensorProtoceZdZdeddfdZdS)ExternalDataInfotensorreturnNcd|_d|_d|_d|_d|_|jD]}t ||j|j|jrt|j|_|jrt|j|_dSdS)N) locationoffsetlengthchecksumbasepath external_datasetattrkeyvalueint)selfr entrys U/var/www/html/ai-engine/env/lib/python3.11/site-packages/onnx/external_data_helper.py__init__zExternalDataInfo.__init__s     ) 2 2E D%)U[ 1 1 1 1 ; +dk**DK ; +dk**DKKK + +)__name__ __module__ __qualname__r rrrr r s6+{+t++++++rr r base_dirrcxt|}tj||j|j}t |d5}|jr||j|jr | |j|_ n| |_ ddddS#1swxYwYdS)zLoads data from an external file for tensor. Ideally TensorProto should not hold any raw data but if it does it will be ignored. Arguments: tensor: a TensorProto object. base_dir: directory that contains the external data. rbN) r c_checker_resolve_external_data_locationrnameopenrseekrreadraw_data)r r$infoexternal_data_file_path data_files rload_external_data_for_tensorr1!s F # #D'G$- %t , ,/ ; ( NN4; ' ' ' ; /'nnT[99FOO'nn..FO//////////////////sA"B//B36B3modelct|D]<}t|r+t||tj|_|jdd==dS)zLoads external tensors into model Arguments: model: ModelProto to load external data to base_dir: directory that contains external data N)_get_all_tensorsuses_external_datar1r DEFAULT data_locationr)r2r$r s rload_external_data_for_modelr87s`#5))(( f % % ( )&( ; ; ;#.#6F $QQQ' ((rrrrrrc|dstd|jzdz|jdd=tj|_||t|nd|t|nd||dD];\}}|4|j }||_ t||_ = the size_threshold it will be converted to external data. To convert every tensor with raw data to external data set size_threshold=0. convert_attribute (bool): If true, convert all tensors to external data If false, convert only non-attribute tensors to external data zDlocation must be a relative path that is relative to the model path.r-N)_get_initializer_tensorsr4r?uuiduuid1ospathisabsr;r:sys getsizeofr-rBr)_is_valid_filename) r2rDrrErFtensors file_namer tensor_locations rconvert_model_to_external_datarTesQ,'u--G*"5)); %%  !w}}X&&  Z!I 5 5F ++ 5M&/22nDD!&)444  5 5 ; ;F ++ ;M&/22nDD"(+)/::8&)$*,,&7&7O!&/::: ; ;rct|D]P}t|r?|dstd|jdd=t j|_QdS)zCall to set all tensors which use external data as embedded data. save_model saves all the tensors data as embedded data after calling this function. Arguments: model (ModelProto): Model to be converted. r-raw_data field doesn't exist.N)r4r5r:r;rr r6r7)r2r s r convert_model_from_external_datarWst#5))77 f % % 7??:.. B !@AAA$QQQ'#.#6F 77r base_pathc8t|}tj||j}|dst dtj|s)t|d5 dddn #1swxYwYt|d5}| dd|j Y| }|j |kr | d|j |z z| |j | }| |j t||j|| |z ddddS#1swxYwYdS) zWrites tensor data to an external file according to information in the `external_data` field. Arguments: tensor (TensorProto): Tensor object to be serialized base_path: System path of a folder where tensor data is to be stored r-rVabNzr+br)r rKrLjoinrr:r;isfiler*r+rtellwriter-rB)r rXr.r/r0 file_sizers rsave_external_datarbs  F # #D gll9dmDD ??: & &:8999 7>>1 2 2 )4 0 0                   %u - - Tq! ; "!((I{Y&&y)@ ABBB NN4; ' ' '!!(((&$-9I9IF9RSSS T T T T T T T T T T T T T T T T T Ts%BBB1CFFFonnx_model_protocVtt|t|S)z=Scan an ONNX model for all tensors and return as an iterator.)rrH_get_attribute_tensorsrcs rr4r4s,  !122/00  r attributefuncc#K|jtjkr||jEd{V|jtjkr|jD]}||Ed{VdSdS)zICreate an iterator through processing ONNX model attributes with functor.N)typerGRAPHgGRAPHSgraphs)rgrhgraphs r_recursive_attribute_processorrps~---4 $$$$$$$$$~...% # #EtE{{ " " " " " " " "/. # #ronnx_model_proto_graphc#K|jEd{V|jD]'}|jD]}t|tEd{V(dS)z@Create an iterator of initializer tensors from ONNX model graph.N) initializernodergrp#_get_initializer_tensors_from_graphrqrtrgs rrurus&11111111&+  I5>         rc#>Kt|jEd{VdS)z:Create an iterator of initializer tensors from ONNX model.N)rurorfs rrHrHs023C3IJJJJJJJJJJJrc#K|jD]R}|jD]H}|dr |jV|jEd{Vt |t Ed{VISdS)zJCreate an iterator of tensors from node attributes of an ONNX model graph.tN)rtrgr:ryrQrp!_get_attribute_tensors_from_graphrvs rrzrzs'+  I!!#&& "k!!! ( ( ( ( ( ( ( (5<          rc#>Kt|jEd{VdS)zDCreate an iterator of tensors from node attributes of an ONNX model.N)rzrorfs rreres001A1GHHHHHHHHHHHrfilenamecrtjd}||}t|S)z8Utility to check whether the provided filename is valid.z^[^<>:;,?"*|/]+$)recompilematchbool)r|exprs rrPrPs/ *' ( (C IIh  E ;;rcV|do|jtjkS)z?Returns true if the tensor stores data in an external location.r7)r:r7r r<)r s rr5r5 s+ (( 9  K$8 8r field_keyc`t|jD]\}}|j|kr|j|=dS)aRemoves a field from a Tensor's external_data key-value store. Modifies tensor object in place. Arguments: tensor (TensorProto): Tensor object from which value will be removed field_key (string): The key of the field to be removed N) enumeraterr)r rifields rremove_external_data_fieldrsEf233((5 9 ! !$Q'((rfilepathct|D]K}t|r:|dr%t|||dL|S)aSerializes data for all the tensors which have data location set to TensorProto.External. Note: This function also strips basepath information from all tensors' external_data fields. Arguments: model (ModelProto): Model object which is the source of tensors to serialize. filepath: System path to the directory which should be treated as base path for external data. Returns: ModelProto: The modified model object. r-)r4r5r:rb ClearField)r2rr s rwrite_external_data_tensorsrsj#5))** f % % *&//**E*E * vx 0 0 0   j ) ) ) Lr)NNNN)TNrCF)'rKr~rNrI itertoolsrtypingrrronnx.onnx_cpp2py_export.checkeronnx_cpp2py_exportcheckerr' onnx.onnx_pbrrr r r r?r1r8rrBrrTrWrbr4rprurHrzrerPr5rrr#rrrs //////////333333333LLLLLLLLLLLL++++++++$/+//////, ( (c (d ( ( ( (&! "" !! !! SM! SM ! sm ! sm ! !!!!@%)"# 1;1; 1;!1;sm1; 1;  1;  1;1;1;1;h 7J 74 7 7 7 7 T{TsTtTTTTDzh{6K##%-zlH[SKKKK &  k    IZIH[