gT ZddlmZddlmZdZdZd dZdZedkr edSdS) )CustomArgumentParser)merge_fsdp_weightsa!Utility to merge the weights from multiple FSDP checkpoints into a single combined checkpoint. Should be used if `SHARDED_STATE_DICT` was used for the model. Weights will be saved to `{output_path}`. This is a CPU-bound process and requires enough RAM to load the entire model state dict.cTt|j|j|j |jdSN)rcheckpoint_directory output_pathunsafe_serializationremove_checkpoint_dir)argss U/var/www/html/ai-engine/env/lib/python3.11/site-packages/accelerate/commands/merge.py merge_commandr s7 !4#39R5RTXTnNc||dt}ntt}|dtd|dtd|dd d d |d ddd ||t |S)Nz merge-weights) descriptionrz5A directory containing sharded weights saved by FSDP.)typehelprzHThe path to save the merged weights. Defaults to the current directory. z--unsafe_serialization store_falseFzZWhether to save the merged weights as `.bin` rather than `.safetensors` (not recommended).)actiondefaultrz--remove_checkpoint_dir store_truez9Whether to remove the checkpoint directory after merging.)rrr)func) add_parserrr add_argumentstr set_defaultsr ) subparsersparsers r merge_command_parserr s&&K&PP%+>>> .S?vwww   W    i   ! H /// Mrcht}|}t|dSr)r parse_argsr )rr s r mainr!>s2 ! # #F     D$r__main__r) accelerate.commands.utilsraccelerate.utilsrrr rr!__name__rr r's ;::::://////\  <  zDFFFFFr