gdZddlZddlZddlZddlZddlZddlZddlmZm Z m Z m Z m Z m Z mZddlmZddZdZd ed efd Zdd edefdZdefdZdS)z A collection of utilities for ensuring that training can always occur. Heavily influenced by the [toma](https://github.com/BlackHC/toma) library. N)is_cuda_availableis_ipex_availableis_mlu_availableis_mps_availableis_musa_availableis_npu_availableis_xpu_available)compare_versionsFc\|rtjtr tjdSt r tjdStr tj dStr tj dStdr tj dStr tjdSdS)z Clears the device cache by calling `torch.{backend}.empty_cache`. Can also run `gc.collect()`, but do note that this is a *considerable* slowdown and should be used sparingly. z2.0) min_versionN)gccollectr torchxpu empty_cachermlurmusar npurmpsrcudagarbage_collections S/var/www/html/ai-engine/env/lib/python3.11/site-packages/accelerate/utils/memory.pyclear_device_cacher(s   !     !    !        !  e , , ,!    !      !!ct|tst|}tt|D]}d||<t d|S)aN Releases memory from `objects` by setting them to `None` and calls `gc.collect()` and `torch.cuda.empty_cache()`. Returned objects should be reassigned to the same variables. Args: objects (`Iterable`): An iterable of objects Returns: A list of `None` objects to replace `objects` Example: ```python >>> import torch >>> from accelerate.utils import release_memory >>> a = torch.ones(1000, 1000).cuda() >>> b = torch.ones(1000, 1000).cuda() >>> a, b = release_memory(a, b) ``` NTr) isinstancelistrangelenr)objectsis rrelease_memoryr$>s_, gt $ $ w-- 3w<<  $//// Nr exceptionreturncgd}ttr3tjdkrt fd|DSdS)z Checks if `exception` relates to CUDA out-of-memory, CUDNN not supported, or CPU out-of-memory Args: exception (`Exception`): An exception )zCUDA out of memory.z(cuDNN error: CUDNN_STATUS_NOT_SUPPORTED.z*DefaultCPUAllocator: can't allocate memoryrc36K|]}|jdvVdS)rN)args).0errr%s r z+should_reduce_batch_size..js/CC3).++CCCCCCrF)r RuntimeErrorr!r)any)r% _statementss` rshould_reduce_batch_sizer0\seK )\**Ds9>/B/Ba/G/GCCCC{CCCCCC 5rfunctionstarting_batch_sizecTtjt|S|fd}|S)a A basic decorator that will try to execute `function`. If it fails from exceptions related to out-of-memory or CUDNN, the batch size is cut in half and passed to `function` `function` must take in a `batch_size` parameter as its first argument. Args: function (`callable`, *optional*): A function to wrap starting_batch_size (`int`, *optional*): The batch size to try and fit into memory Example: ```python >>> from accelerate.utils import find_executable_batch_size >>> @find_executable_batch_size(starting_batch_size=128) ... def train(batch_size, model, optimizer): ... ... >>> train(model, optimizer) ``` N)r3c \tdttjj}t |t |dzkr`ddt|dd|ddD}tdj dj d|d  d krtd  g|Ri|S#t$r0}t|rtdd znYd}~nd}~wwxYw`) NTrrz, c"g|] \}}|d| S)=)r*argvalues r zAfind_executable_batch_size..decorator..s* ^ ^ ^jc5C!1!1%!1!1 ^ ^ ^rzBatch size was passed into `zS` as the first argument when called.Remove this as the decorator already does so: `(z)`rz-No executable batch size found, reached zero.)rrinspect signature parameterskeysr!joinzip TypeError__name__r- Exceptionr0)r)kwargsparamsarg_stre batch_sizer2s r decoratorz-find_executable_batch_size..decoratorsd3333g'11<AACCDD v;;#d))a- ( (ii ^ ^Cqrr TXYZY[Y[T\D]D] ^ ^ ^__Gbx/@bbBJBSbbV]bbb  Q"#RSSS x =z2.5r) mem_get_infozThe XPU `mem_get_info` API is available in IPEX version >=2.5. The current returned available memory is incorrect. Please consider upgrading your IPEX version.) r importlibmetadataversionr intel_extension_for_pytorch.xpurSwarningswarnrrmax_memory_allocated)rP ipex_versionrSs rget_xpu_available_memoryr\s1 )112OPP L$ 6 6 1 D D D D D D< --a0 0 M j 9 ) ), 7 77r)F)Nr1)__doc__rMrrTr>rXrimportsrrrrrr r versionsr rr$rFboolr0callableintrOr\r8rrrcsM   '&&&&&!!!!,< d$777s7777t 83 8 8 8 8 8 8r