NgERdZddlZddlZddlZddlZddlZddlmZddlm Z ddl m Z m Z m Z mZmZmZmZmZddlmZddlmZddlZdejvre ejdd anHd ejvre ejd d d an&e ejd d d d aGddZdeddfdZdefdZdedeeeeffdZ dede!fdZ"d,dede!defdZ#dededdfdZ$ d-dedeedeeedeefd Z% d.deded!ee&defd"Z' d/deded!ee&d#e!d$e!deej(f d%Z) d/deded!ee&d#e!d$e!deef d&Z* d0d'ed(ed)ed*ed!ee&d#e!d$e!deej(fd+Z+dS)1zJONNX Model Hub This implements the python client for the ONNX model hub. N)BytesIO)join)IOAnyDictListOptionalSetTuplecast) HTTPError)urlopen ONNX_HOMEhubXDG_CACHE_HOMEonnx~z.cachecJeZdZdZdeeefddfdZdefdZdefdZ dS) ModelInfoaMA class to represent a model's property and metadata in the ONNX Hub. It extracts model name, path, sha, tags, etc. from the passed in raw_model_info dict. Attributes: model: The name of the model. model_path: The path to the model, relative to the model zoo (https://github.com/onnx/models/) repo root. metadata: Additional metadata of the model, such as the size of the model, IO ports, etc. model_sha: The SHA256 digest of the model file. tags: A set of tags associated with the model. opset: The opset version of the model. raw_model_inforeturnNcTtt|d|_tt|d|_tttt f|d|_d|_d|jvr%tt|jd|_t|_ d|jvr=tttt|jd|_ tt|d|_ ||_ dS)zgInitializer. Args: raw_model_info: A JSON dict containing the model info. model model_pathmetadataN model_shatags opset_version)r strrrrrrrsetrrintopsetr)selfrs D/var/www/html/ai-engine/env/lib/python3.11/site-packages/onnx/hub.py__init__zModelInfo.__init__*s #~g677 sN<$@AA(,T#s(^^J=W(X(X (, $- ' '!#t}['ABBDN!ee T] " "DcDM&,ABBCCDI#~o>?? .<c Hd|jd|jd|jd|jd S)NzModelInfo(model=z, opset=z, path=z , metadata=))rr"rrr#s r$__str__zModelInfo.__str__?s5u$*uudjuuuueieruuuur&c*|SN)r*r)s r$__repr__zModelInfo.__repr__Bs||~~r&) __name__ __module__ __qualname____doc__rrrr%r*r-r&r$rrs  =tCH~=$====*vvvvv#r&rnew_dirrc |adS)zkSets the current ONNX hub cache location. Args: new_dir: Location of new model hub cache. N _ONNX_HUB_DIR)r3s r$set_dirr7Fs MMMr&ctS)znGets the current ONNX hub cache location. Returns: The location of the ONNX hub model cache. r5r2r&r$get_dirr9Ps r&repoc |dddd}|dddd}d|vr|dd}nd}|||fS)zCGets the repo owner, name and ref from a repo specification string.:r/main)splitr: repo_owner repo_namerepo_refs r$_parse_repo_inforEYsC#))#..q1J 3"((--a0I d{{::c??1% y( **r&cLt|\}}}|dko |dko|dkS)zzVerifies whether the given model repo can be trusted. A model repo can be trusted if it matches onnx/models:main. rmodelsr?rErAs r$_verify_repo_refrIds7'7t&<&<#J 8 & VyH'< V8vCUVr&Flfsc\t|\}}}|r d|d|d|dSd|d|d|dS)aTGets the base github url from a repo specification string. Args: repo: The location of the model repo in format "user/repo[:branch]". If no branch is found will default to "main". lfs: Whether the url is for downloading lfs models. Returns: The base github url for downloading. z*https://media.githubusercontent.com/media/r=z"https://raw.githubusercontent.com/rH)r:rJrBrCrDs r$ _get_base_urlrLlsd'7t&<&<#J 8 a`J````U]```` T T TY T T T T TTr&url file_namec d}t|5}t|d5} ||}|sn||. dddn #1swxYwYddddS#1swxYwYdS)zDownloads the file with specified file_name from the url. Args: url: A url of download link. file_name: A specified file name for the downloaded file. i@wbTN)ropenreadwrite)rMrN chunk_sizeresponsefchunks r$_download_filerXsJ 4 4#8#8A MM*--E  GGENNN   s4A80A  A8 A$ $A8'A$ (A88A<?A<onnx/models:mainrrc.t|}|dz} t|5}dtjt t t |D}dddn #1swxYwYn%#t$r}td||d}~wwxYw|n fd|D}||Sd|D} g} |D]N} d| j D} t| | dkr| | O| S) aGets the list of model info consistent with a given name and tags Args: repo: The location of the model repo in format "user/repo[:branch]". If no branch is found will default to "main" model: The name of the model to search for. If `None`, will return all models with matching tags. tags: A list of tags to filter models by. If `None`, will return all models with matching name. Returns: ``ModelInfo``s. zONNX_HUB_MANIFEST.jsonc,g|]}t|Sr2)r).0infos r$ zlist_models..s+)))$( $)))r&NzCould not find manifest at crg|]3}|jk1|4Sr2)rlower)r\mrs r$r^zlist_models..s5 F F FAQW]]__ %E%Ea%E%E%Er&c6h|]}|Sr2r`r\ts r$ zlist_models..s ...Aaggii...r&c6h|]}|Sr2rcrds r$rfzlist_models..s 000Aaggii000r&r) rLrjsonloadr rrr AssertionErrorrlen intersectionappend) r:rrbase_url manifest_urlrUmanifestematching_modelscanonical_tagsmatching_info_listra model_tagss ` r$ list_modelsrvs&T""H66LR \ " " h)),0Id2c7H6M6M,N,N)))H                RRRI<IIJJPQQR =  F F F F F F F |.....N*, ))00000 ~**:66 7 7! ; ;  % %a ( ( ( s:A;=A/# A;/A33A;6A37A;; BBBr"ct||}|std|t|d}n3fd|D}|s#d|D}t|dd||d S) aGets the model info matching the given name and opset. Args: model: The name of the onnx model in the manifest. This field is case-sensitive repo: The location of the model repo in format "user/repo[:branch]". If no branch is found will default to "main" opset: The opset of the model to get. The default of `None` will return the model with largest opset. Returns: ``ModelInfo``. zNo models found with name Nc|j Sr,r")ras r$z get_model_info..s r&)keyc*g|]}|jk |Sr2ry)r\rar"s r$r^z"get_model_info..s%JJJE9I9I19I9I9Ir&cg|] }|j Sr2ry)r\ras r$r^z"get_model_info..s===AG===r&z has no version with opset z. Valid opsets: r)rvrjsorted)rr:r"rrselected_models valid_opsetss ` r$get_model_infors""$..O CA%AABBB } 6H6HIIIJJJJoJJJ ==_===L ZZUZZLZZ  1 r& force_reloadsilentc t|||}|jd}|j|jd|d|d<t t t j|}|st j |st|sX|sVd|d}t|tj tdtd krdSt jt j|d t%|d } td |d |t'| |jz|ntd|d|t)|d5} | } dddn #1swxYwY|jTt-j| } | |jks#t3d|jd| d|jdt7jt;t<t>tA| S)aDownloads a model by name from the onnx model hub. Args: model: The name of the onnx model in the manifest. This field is case-sensitive repo: The location of the model repo in format "user/repo[:branch]". If no branch is found will default to "main" opset: The opset of the model to download. The default of `None` automatically chooses the largest opset force_reload: Whether to force the model to re-download even if its already found in the cache silent: Whether to suppress the warning message if the repo is not trusted. Returns: ModelProto or None r=N_The model repo specification _ is not trusted and may contain security vulnerabilities. Only continue if you trust this repo.fileContinue?[y/n]yTexist_ok Downloading  to local path Using cached model from rbThe cached model has SHA256  while checksum should be h. The model in the hub may have been updated. Use force_reload to download the model from the model hub.)!rrr@rrr6osseppathexistsrIprintsysstderrinputr`makedirsdirnamerLrXrQrRhashlibsha256 hexdigestrjrrrir rbytesr) rr:r"rrselected_modellocal_model_path_arrlocal_model_pathmsglfs_urlrV model_bytesdownloaded_shas r$riris2$E477N)4::3??+' D D*>r*B D D   M26;;7K+L+LMME27>>*:;;E%% f X$XXXC #CJ ' ' ' ' " # # #ww}}#%%t BGOO$455EEEEd++ EUEE3CEEFFFw!::>@@!999 9N$899n99,:,D999  9T"U)W[%9%9:: ; ;;sF99F=F=c 8t|||}|jdd}|jd}||d|d|d<ttt j|}|st j|st|sX|sVd|d} t| tj td td krdSt jt j|d t#|d } td|d|t%| |jdz|ntd|d|t'|d5} | } dddn #1swxYwY|Jt+j| } | |kst1d|jd| d|dt5j|5}|dt7|dz }||dddn #1swxYwY|dzt j|dz}|S)aDownloads a model along with test data by name from the onnx model hub and returns the directory to which the files have been extracted. Args: model: The name of the onnx model in the manifest. This field is case-sensitive repo: The location of the model repo in format "user/repo[:branch]". If no branch is found will default to "main" opset: The opset of the model to download. The default of `None` automatically chooses the largest opset force_reload: Whether to force the model to re-download even if its already found in the cache silent: Whether to suppress the warning message if the repo is not trusted. Returns: str or None model_with_data_pathr=model_with_data_shaNrrrrrrrTrrrrrrrrrrr)rrr@rr6rrrrrIrrrrr`rrrLrXrQrRrrrrjrtarfilerk extractalllistdir)rr:r"rrrlocal_model_with_data_path_arrrlocal_model_with_data_pathrrrVmodel_with_data_bytesrmodel_with_data_zippedlocal_model_with_data_dir_pathrs r$download_model_with_test_datar#s2$E477N%3%<& eCjj#)12GH&# I I%CB%G I I '  "&rv{{#ABB""O27>>*DEEO%% f X$XXXC #CJ ' ' ' ' " # # #ww}}#%%t BGOO$>??$OOOOd++ OUOO3MOOPPP n-.DE E &    MeMM1KMMNNN ($ / /)1 !)))))))))))))))& (=>>HHJJ!444 9N$899n99,?999  0 1 1J5K)C .//!3 3* & ))*HIII JJJJJJJJJJJJJJJ '   *3 4 4Q 7 8 s$'GG G 30I//I36I3 network_modelpreprocessing_model network_repopreprocessing_repocjt|||||}|td|t|||||}|td|t} i} i} |jD]5} | jdkrdn| j} | | | j| | <6|jD]5} | jdkrdn| j} | | | j| | <6d}d}| D]^} | dkr| | }| | }| | vrA| | vr=| | | | kr+td|d|d | d | | d | | d _||krLtj ||}|j |_ tj |nQ||krKtj ||}|j |_ tj |d t|jj|jjD}tj|||}|S)aBuilds a composite model including data preprocessing by downloading a network and a preprocessing model and combine it into a single model Args: network_model: The name of the onnx model in the manifest. preprocessing_model: The name of the preprocessing model. network_repo: The location of the model repo in format "user/repo[:branch]". If no branch is found will default to "main" preprocessing_repo: The location of the proprocessing model repo in format "user/repo[:branch]". If no branch is found will default to "main" opset: The opset of the model to download. The default of `None` automatically chooses the largest opset force_reload: Whether to force the model to re-download even if its already found in the cache silent: Whether to suppress the warning message if the repo is not trusted. Returns: ModelProto or None Nz(Could not load the preprocessing model: z"Could not load the network model: zai.onnxrzCan not merge z and z: because they contain different opset versions for domain z (z) and z\). Only the default domain can be automatically converted to the highest version of the two.c0g|]\}}|j|jfSr2)name)r\ out_entryin_entrys r$r^z(load_composite_model..s5 Ix 'r&)io_map)ri RuntimeErrorr opset_importdomainaddversion ValueErrorrversion_converterconvert_version ir_versionchecker check_modelzipgraphoutputrcompose merge_models)rrrrr"rr preprocessingnetwork all_domainsdomains_to_version_network domains_to_version_preprocessingopset_import_entryrpreprocessing_opset_versionnetwork_opset_versionrmodel_with_preprocessings r$load_composite_modelr{s">/ fM L7J L L   =,|VLLGO OOPPPEEK1379$%2HH+2b88II>P>W  -?-G"6**+8NN+2b88II>P>W  3E3M(00"$ Y  *J6*R '$>v$F ! ! 6 6 644408/788M!4MM=MM7=MMAabhAiMM1&9MMM  #%:::(88 0  +5   )))) !< < <.>> 0  $+#5     ///#&}':'A7=CV#W#WF $|88wv 9   $#r&)F)rYNN)rYN)rYNFF)rYrYNFF),r1rrhrrrioros.pathrtypingrrrrr r r r urllib.errorr urllib.requestrrenvironr6r expanduserrrr7r9rEboolrIrLrXrvr!r ModelProtorirrr2r&r$rsr BBBBBBBBBBBBBBBBBBBB"""""""""""" "*DK0%88MM##D$45vuEEMMD++C00(FEJJM&&&&&&&&RST+3+5c3#7++++W3W4WWWWUUU$U3UUUU&$# $.. . C=. 49 . )_ ....dHL 7?}F# ><>< >< >< C=>< ><  >< do ><><><>