g*ddlZddlmZmZmZddlZddlZddlm Z ddl m Z ddl m Z mZddlmZmZmZmZmZer ddlmcmZdd ed ed efd ZddeedeejfdZddeeeefdeejfdZdS)N)ListOptionalUnion)AcceleratorState)CUDA_DISTRIBUTED_TYPES)DistributedTypeRNGType)is_mlu_availableis_musa_availableis_npu_availableis_torch_xla_availableis_xpu_availableFseeddevice_specific deterministicc|r|tjz }tj|tj|t j|tr t j |ntr t j |n{tr t j |nMtr t j |nt j |t#rt%j||rt jddSdS)a Helper function for reproducible behavior to set the seed in `random`, `numpy`, `torch`. Args: seed (`int`): The seed to set. device_specific (`bool`, *optional*, defaults to `False`): Whether to differ the seed on each device slightly with `self.process_index`. deterministic (`bool`, *optional*, defaults to `False`): Whether to use deterministic algorithms where available. Can slow down training. TN)r process_indexrandomrnptorch manual_seedrxpumanual_seed_allrnpur mlur musacudarxm set_rng_stateuse_deterministic_algorithms)rrrs S/var/www/html/ai-engine/env/lib/python3.11/site-packages/accelerate/utils/random.pyset_seedr$sO1  ""00 KINN4 d ) !!$''''   ) !!$''''   ) !!$''''   ) ""4(((( ""4((( 1 *40000011rng_type generatorc |tjkrtj}n|tjkr tj}n|tjkr?ts Jdtjtj}nM|tj kr8ts Jdtj }n|tj kr7ts Jdtj}n|tjkr7t#s Jdtj}nw|tjkr7t)s Jdtj}n0|tjkr | Jd|}t1}|jt4jkrc|tj}tj|gtj|}n|jt@vsT|jt4j!ks?|jt4j"ks*|jt4j#ks|jt4j$krO||j%}tj&'|d|}n5|jt4j(kr tj&'|d|tjkrtj)|dS|tjkr!tj)|dS|tj kr!tj )|dS|tj kr!tj)|dS|tjkr!tj)|dS|tjkr!tj)|dS|tjkr(tj)|*dS|tjkr|+|dSdS)Nz8Can't synchronize XLA seeds as torch_xla is unavailable.z;Can't synchronize NPU seeds on an environment without NPUs.z;Can't synchronize MLU seeds on an environment without MLUs.z=Can't synchronize MUSA seeds on an environment without MUSAs.z;Can't synchronize XPU seeds on an environment without XPUs.z)Need a generator to synchronize its seed.r),r TORCHr get_rng_stateCUDArXLArtensorr NPUrrMLUr rMUSAr rXPUrr GENERATOR get_staterdistributed_typer to xla_devicecollective_broadcast mark_stepcpur MULTI_MLU MULTI_MUSA MULTI_NPU MULTI_XPUdevice distributed broadcast MULTI_CPUr!item set_state)r&r' rng_statestates r#synchronize_rng_staterFBs7=  ')) W\ ! !J,,.. W[ %''cc)ccccL!1!3!344 W[ !!``#````I++-- W[ !!``#````I++-- W\ ! ! ""cc$ccccJ,,.. W[ !!``#````I++-- W& & &$$&Q$$$''))    E !444LL11   ,,, MMOO  "888  !_%> > >  !_%? ? ?  !_%> > >  !_%> > >LL..  ##Iq111MMOO  ?#< < < ##Iq1117=   I&&&&& W\ ! !   +++++ W[   ***** W[   ***** W\ ! !   +++++ W[   ***** W[  ))***** W& & &I&&&&& ' &r% rng_typescL|D] }tt||!dS)N)r')rFr )rGr'r&s r#synchronize_rng_statesrIs=FFgh//9EEEEEFFr%)FF)NN)N) rtypingrrrnumpyrrrEr constantsr dataclassesr r importsr r rrrtorch_xla.core.xla_modelcore xla_modelr intboolr$ GeneratorrFstrrIr%r#rWs (((((((((( $$$$$$------11111111tttttttttttttt*))))))))) 1 13 1 1d 1 1 1 1F='='HW$5='RWRaIb='='='='@FFd5g+>&?FHUZUdLeFFFFFFr%