gy0 8ddlmZddlmZmZddlmZddlmZddl m Z m Z m Z m Z e r ddlZddlmZmZe rddlZd ded efd Zd Zd!d edededefdZdededefdZd dZd dedededefdZdZdZdZe dkr edSdS)") model_info)GatedRepoErrorRepositoryNotFoundError)init_empty_weights)CustomArgumentParser)calculate_maximum_sizes convert_bytesis_timm_availableis_transformers_availableN) AutoConfig AutoModelrepotokencn t||S#ttf$rYdSt$rYdSwxYw)zAVerifies that the model is on the hub and returns the model info.)rgatedr)rOSErrorrr)rrs X/var/www/html/ai-engine/env/lib/python3.11/site-packages/accelerate/commands/estimate.py verify_on_hubr%sZ$e,,,, ^ $ww "vvs4 44ctr&t|trd|jdvrdSt r&t|t rd|jdvrdSdS)zG Checks what library spawned `error` when a model is not found z Unknown modelrtimmz$does not appear to have a file named transformersunknown)r isinstance RuntimeErrorargsr r)errors rcheck_has_modelr/sy z%>> ?V[V`abVcCcCcv!## ug & & 3ejm C C~yF model_name library_nametrust_remote_code access_tokencBt||}|dkrtd|d|dkrtd|d|&t|dd}|st d |d |d kr#t st d |d td|d|jtd|d|j dd}tj |||}t5t}t|t rId}|D]} | dr| }n|tt&|}|||} dddn #1swxYwYn|dkrqt+st d |dtd|dt5t-j|d} dddn #1swxYwYnt d|d| S)a Creates an empty model from its parent library on the `Hub` to calculate the overall memory consumption. Args: model_name (`str`): The model name on the Hub library_name (`str`): The library the model has an integration with, such as `transformers`. Will be used if `model_name` has no metadata on the Hub to determine the library. trust_remote_code (`bool`, `optional`, defaults to `False`): Whether or not to allow for custom models defined on the Hub in their own modeling files. This option should only be set to `True` for repositories you trust and in which you have read the code, as it will execute code present on the Hub on your local machine. access_token (`str`, `optional`, defaults to `None`): The access token to use to access private or gated models on the Hub. (for use on the Gradio app) Returns: `torch.nn.Module`: The torch model that has been initialized on the `meta` device. rzRepo for model `zW` is gated. You must be authenticated to access it. Please run `huggingface-cli login`.rz` does not exist on the Hub. If you are trying to access a private repo, make sure you are authenticated via `huggingface-cli login` and have access.Nr FzModel `z{` does not have any library metadata on the Hub, please manually pass in a `--library_name` to use (such as `transformers`)rz To check `zU`, `transformers` must be installed. Please install it via `pip install transformers`zLoading pretrained config for `z` from `transformers`...Tried to load `z8` with `transformers` but it does not have any metadata.auto_map)r!r AutoModelFor)r!rzE`, `timm` must be installed. Please install it via `pip install timm`z` from `timm`...) pretrainedz Library `zM` is not supported yet, please open an issue on GitHub for us to add support.)rrrgetattr ValueErrorr ImportErrorprintconfigrgetr from_pretrainedrr rdictkeys startswithr from_configr r create_model) rr r!r"rr%r, constructorvaluekeymodels rcreate_empty_modelr8?s|*z<88JW Cz C C C    v  % \z \ \ \   z>5AA  b*bbb ~%%(** ~Z~~~  T TTTUUU   $uuuuvv v$((U;;+JJ[coppp  ! ! Y Y#K(D)) ?#==??C~~n55 #$"),">">K++FFW+XXE Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y    "" nZnnn  L LLLMMM  ! ! D D%jUCCCE D D D D D D D D D D D D D D D s s s s    Ls%A=F  F F HHHheadersrowstitlecd\}gtt|D]Hfd|D|gz}td|D}|Ifdtt|dD}||||}dfd}|ddd } t|t kr).0rowis r z&create_ascii_table..s000CQ000rc34K|]}t|VdSN)len)r?r5s r z%create_ascii_table..s(EEes5zzEEEEEErc(g|]}d|dS)%sr>)r?rA column_widthss rrBz&create_ascii_table..s*DDD1&=#&&&DDDrrc\||fdDz|S)Ncg|]}|zSr>r>r?n in_betweens rrBz8create_ascii_table..make_row..s.U.U.U!zA~.U.U.Ur)join) left_char middle_char right_charrJdiffrOs rmake_rowz$create_ascii_table..make_rowsIw[--.U.U.U.U}.U.U.UVVwXbeiXiwkuwwwru├u┼u┤u┌u┐u┬ cLg|] \}}||!Sr>center)r?rAtextrJs rrBz&create_ascii_table..s.UUUwq$T[[q!122UUUrcLg|] \}}||!Sr>rZ)r?rAtrJs rrBz&create_ascii_table..s.PPP1-"233PPPru└u┴cg|]}|zSr>r>rMs rrBz&create_ascii_table..sEEE!zA~EEEru┘) rangerEmaxappendrPsumabsr[ enumeratetuple)r9r:r;sep_char column_valuesmax_column_widthformatspatternrU separator initial_rowstable centered_linelinerJrTrArOs @@@@rcreate_ascii_tablerqs'HjM 3w<< //00004000GAJ<? EE}EEEEE-....DDDDc$q'll0C0CDDDG=8==11=8==G Dxxxxxxxu--I 5zzC &&&&3u::I.//bT!u--I E** AU\\#i..1"455AxAAu%%L IIl # #d *E"UUUU)GBTBTUUUM % ... ? ?) ? ? ??ET??774PPPP $PPP  GeM2226666 K5::EEEE}EEEFF K K KKE Lrc ||d}ntd}|dtd|dtdd d g |d td gddgd|dddd||t |S)Nzestimate-memoryz:Model size estimator for fitting a model onto CUDA memory.) descriptionrz'The model name on the Hugging Face Hub.)typehelpz--library_namezThe library the model has an integration with, such as `transformers`, needed only if this information is not stored on the Hub.rr)rtruchoicesz--dtypes+)float32float16int8int4zbThe dtypes to use for the model, must be one (or many) of `float32`, `float16`, `int8`, and `int4`)rtnargsdefaultrurvz--trust_remote_code store_trueaWhether or not to allow for custom models defined on the Hub in their own modeling files. This flag should only be used for repositories you trust and in which you have read the code, as it will execute code present on the Hub on your local machine.F)actionrur})func) add_parserr add_argumentstr set_defaultsestimate_command) subparsersparsers restimate_command_parserrs&&'899%2nooo  35^___  P(    666 q666  B !1222 Mrbytesmixed_precision msamp_configreturncddddd}|}|dz}|dkr||d<||d<|dz|d<|dz|d <n,|d vs|d kr"| ||d<||z|d<|dz|d<|d|d <|S) a Given an amount of `bytes` and `mixed_precision`, calculates how much training memory is needed for a batch size of 1. Args: bytes (`int`): The size of the model being trained. mixed_precision (`str`): The mixed precision that would be ran. msamp_config (`str`): The msamp config to estimate the training memory for if `mixed_precision` is set to `"fp8"`. rV)r7 optimizer gradientssteprWrxr7rrr)rybfloat16fp8r>)rrr memory_sizes fp32_size fp16_sizes restimate_training_usagers br2NNLI I)## ) W$- [!$-M [!(1} V 3 3 358P8PUaUi!* W$- $9 [!$-M [!+K8 V rc t|j|j|j}nK#tt f$r7}t |}|dkrt d|jd|d|d}~wwxYwt|\}}g}|jD]g}|}|d} t||} |dkr |d z}| d z} n!|d kr |d z}| d z} n|d kr |d z}| d z} | || || gh|S)z9Creates an empty model and gathers the data for the sizes)r r!rr$z` with `z=` but a possible model to load was not found inside the repo.NrryrWrzrr{) r8rr r!rrrrdtypesrrb) rr7elibrary total_size largest_layerdatadtypedtype_total_sizedtype_largest_layerdtype_training_sizes r gather_datars " O$*;tOe    ' "!!$$ i  B$/BB7BBB  !8 > >J D Y Y%+A.56FNN I    !  1 $   f__  !  1 $   f__  !  1 $  U/1ACVWXXXX Ks!$A,2A''A,ct|}|D]}t|D]\}}t|ttfrt |||<4t|t r;t|}|dkrt |nd||<gd}d|j d}t|||}t|dS)NrVzN/A)rz Largest Layerz Total SizezTraining using AdamzMemory Usage for loading ``) rrerintfloatr r/ravaluesrrqr+) rrr@rAitemtraining_usager9r;rns rrrs t  DZZ ~~ Z ZGAt$e -- Z&t,,AD$'' Z!$T[[]]!3!3:HB:N:N~666TYA  ZNMMG ; ; ; ;E we 4 4E %LLLLLrcht}|}t|dSrD)r parse_argsr)rrs rmainr.s2 $ & &F     DTr__main__rD)FN)!huggingface_hubrhuggingface_hub.utilsrr accelerateraccelerate.commands.utilsraccelerate.utilsrr r r rr r rrrrboolr8listrqrrr/rrrr__name__r>rrrs  '&&&&&IIIIIIII))))))::::::322222222KKKC    II3IcIdIjmIIIIX&&D&&&&&R    F  3  C [_    F   F"  zDFFFFFr