g[ddlmZmZddlmZmZmZddlmZm Z m Z ddl m Z ddl mZ ddlmZmZmZdd lmZdd lmZdd lmZdd lmZd Zn#eef$r eZdZdZYnwxYwe jdZ defdZ!GddeZ"GddeZ#GddeZ$GddeZ%GddeZ&dS))ArgumentParser Namespace)AnyListOptional)Pipelineget_supported_taskspipeline)logging)BaseTransformersCLICommand)BodyFastAPI HTTPException)APIRoute) BaseModel) JSONResponse)runTcdS)N)xys Y/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/commands/serving.pyrr"s Fztransformers-cli/servingargsct|j|jr|jnd|j|j|j}t ||j|j|j S)z~ Factory function used to instantiate serving server from provided command line arguments. Returns: ServeCommand N)taskmodelconfig tokenizerdevice) r rrr r!r" ServeCommandhostportworkers)rnlps rserve_command_factoryr(+sX  Y J0djjD{.{    C TY 4< @ @@rceZdZUdZeed<dS)ServeModelInfoResultz" Expose model information infosN)__name__ __module__ __qualname____doc__dict__annotations__rrrr*r*;s% KKKKKrr*cLeZdZUdZeeed<eeeed<dS)ServeTokenizeResultz Tokenize result model tokens tokens_idsN) r,r-r.r/rstrr1rintrrrr3r3CsC Ic######rr3ceZdZUdZeed<dS)ServeDeTokenizeResultz! DeTokenize result model textN)r,r-r.r/r6r1rrrr9r9Ls% IIIIIrr9ceZdZUdZeed<dS)ServeForwardResultz Forward result model outputN)r,r-r.r/rr1rrrr<r<Ts% KKKKKrr<ceZdZedefdZdedededefdZ dZ d Z e d d e d d fdede fdZe d d e d d e d d fdeede de fdZe d d fdZd S)r#parserc\|dd}|dttd|dtdd |d td d |dtdd |dtd|dtd|dtd|dtdd |t dS)z Register this command to argparse so it's available for the transformer-cli Args: parser: Root parser to register command-specific arguments servezFCLI tool to run inference requests through REST and GraphQL endpoints.)helpz--taskzThe task to run the pipeline on)typechoicesrBz--host localhostz$Interface the server will listen on.)rCdefaultrBz--porti"z Port the serving will listen to.z --workersr zNumber of http workersz--modelz%Model's name or path to stored model.)rCrBz--configz,Model's config name or path to stored model.z --tokenizerzTokenizer name to use.z--devicezSIndicate the device to run onto, -1 indicates CPU, >= 0 indicates GPU (default: -1))funcN) add_parser add_argumentr6r r7 set_defaultsr()r? serve_parsers rregister_subcommandz ServeCommand.register_subcommand]sN(( b)   !! '))2 "   !!(kPv!www!!(dIk!lll!!+CIa!bbb!!)#!^5KKJWW **TTTT**==== X X XC"sSTvv8V8VWWW W XsAA#A## B-!BBr5skip_special_tokenscleanup_tokenization_spacesc |jj|||}td|S#t$r&}t ddt |dd}~wwxYw)a: Detokenize the provided tokens ids to readable text: - **tokens_ids**: List of tokens ids - **skip_special_tokens**: Flag indicating to not try to decode special tokens - **cleanup_tokenization_spaces**: Flag indicating to remove all leading/trailing spaces and intermediate ones. rm)rr:rlrnrpN)rXr!decoder9rtrr6)rbr5rwrx decoded_strrvs rr_zServeCommand.detokenizes~ X.299*FY[vwwK(r DDD D X X XC"sSTvv8V8VWWW W Xs14 A$!AA$cKt|dkrtggS ||}t|S#t$r$}t ddt |id}~wwxYw)zF **inputs**: **attention_mask**: **tokens_type_ids**: r)r= attention)r=rlroN)lenr<rXrtrr6)rbinputsr=rvs rr`zServeCommand.forwards v;;!  %R2>>> > 8^^F++F%V444 4 8 8 8gs1vv%677 7 8s$A A;A66A;)r,r-r. staticmethodrrMr r6r7rcrr]rboolr^rr_r`rrrr#r#\s>N>>>\>rsQ/.......&&&&&&&&&&??????????((((((*4444444444((((((""""""000000$(!!^$***I   %*!!! *  6 7 7 A A A A A 9$$$$$)$$$IH8H8H8H8H8-H8H8H8H8H8s$AA A