קg!ddlZddlZddlZddlmZddlmZddlmZ ddl m Z ddl m Z gdZe eZdZdZd Z ddededededededeefdZdZdZdejfdZdS)N)closing)Optional) get_logger)barrier)create_c10d_store get_free_portget_socket_with_portzAddress already in usezSocket Timeoutz_tcp_store/num_membersXT is_server server_addr server_port world_sizetimeoutwait_for_workers use_libuvc ,|tdtjdddk}|dkr|dkrt d|d||dkrtd||dkr|nd} |dkr|} nt} td || |||| tj || ||tj | || } |rt| || td| S#t$rc} t| tkr@||kr#td| |||dz }ntd|d| d| Yd} ~ nd} ~ wwxYw)Nzargument use_libuv is deprecated and ignored. Set USE_LIBUV environment variable to "0" to disable libuv, or "1" to enable it. If the env var is not set, libuv will be used by default. USE_LIBUV1r r zCserver_port must be specified when world_size > 1, got server_port=z , world_size=z+sever_port: %s, specified, ignoring retriesTziCreating c10d store on %s:%s world_size : %s is_server : %s timeout(sec): %s use_libuv : %s )seconds) host_nameportr is_masterrrr)rzSuccessfully created c10d storez)port: %s already in use, attempt: [%s/%s]zon z, port: z already in use)loggerwarningosenvironget ValueErrorinfordistTCPStoredatetime timedelta_check_full_rank RuntimeErrorstr_ADDRESS_IN_USE) rrrrrrretriesrattemptrstorees g/var/www/html/ai-engine/env/lib/python3.11/site-packages/torch/distributed/elastic/utils/distributed.pyrrs& 9    {C00C7IbZ!^^ xR] x xlv x x   b A;OOO%**ggG7 "  DD ??D  #       # M%%# *7;;;!1#E  E  GDDDD KK9 : : :L    1vv((W$$NNC  qLGG&HkHH4HHH GGGG G7sAD$$ F.AF  Fc t||t|dS#t$r2}t|tkrt d|d|d}~wwxYw)N) key_prefixbarrier_timeoutztimed out waiting for all z members to join)r_TCP_STORE_INITr(r)_SOCKET_TIMEOUT TimeoutError)r-rrr.s r/r'r'vs|zowWWWWWW  q66_ $ $IZIII   s A-AAct}t|5|dcdddS#1swxYwYdS)a Returns an unused port on localhost. This function finds an unused port on localhost by opening to socket to bind to a port and then closing it. Returns: int: an unused port on localhost Example: >>> # xdoctest: +SKIP("Nondeterministic") >>> get_free_port() 63976 ..note: The port returned by :func:`get_free_port` is not reserved and may be taken by another process after this function returns. r N)r r getsockname)socks r/rrs& ! !D %%!!!$%%%%%%%%%%%%%%%%%%sAA  A returnctjddtjtj}|D]}|\}}}}}tj|||} |d|d|cS#t $r:}|t d|Yd}~d}~wwxYwtd) a Returns a free port on localhost that is "reserved" by binding a temporary socket on it. Close the socket before passing the port to the entity that requires it. Usage example :: sock = _get_socket_with_port() with closing(sock): port = sock.getsockname()[1] sock.close() # there is still a race-condition that some other process # may grab this port before func() runs func(port) localhostN)hostrfamilytype)r;rrzSocket creation attempt failed.)exc_infozFailed to create a socket) socket getaddrinfo AF_UNSPEC SOCK_STREAMbindlistenOSErrorcloserrr()addrsaddrr=r>proto_sr.s r/r r s"   tF,<6CU   E J J$(!eQ M&$ . . J FF# $ $ $ HHQKKKHHH J J J GGIII NN> C0B==C)r r r Tr N)r%rr@ contextlibrtypingrtorch.distributed distributedr#'torch.distributed.elastic.utils.loggingr%torch.distributed.elastic.utils.storer__all____name__rr*r4r3boolr)intfloatrr'rr r/rZsm  >>>>>>999999 I H H H  *"* ! $UUUUU U  U  U~UUUUp   %%%04fm444444rY