קgX dZddlZddlmZmZddlmZmZddlZddl m Z m Z m Z m Z mZmZmZddlmZgdZdd d efd Zdd d efd Zdd d efd Zdd d efdZGddeeZGddeZGddeZGddeZeeddejejdZ eeddej!ej"dZ# eedddejdZ$ ee Z%ee Z&e%Z'e&Z(eeddej!ej)dd Z* eeej+ejdddd!Z, eeej+dej-d"Z.eeddejejdZ/ eeddej#Z0 eeddej!ej"$Z1 eeddej!ej)$Z2 eed%dej!ej"d&'Z3 eed%dej!ej)d&'Z4 d(Z5d)Z6d*Z7d+Z8d,Z9dS)-z6Implements modules used to perform fake quantization.N)ABCabstractmethod)AnyTuple) _with_args)default_fixed_qparams_range_0to1_observer,default_fixed_qparams_range_neg1to1_observerFixedQParamsObserverHistogramObserverMovingAverageMinMaxObserver%MovingAveragePerChannelMinMaxObserver)Module)FakeQuantizeBase FakeQuantizeFixedQParamsFakeQuantizeFusedMovingAvgObsFakeQuantizedisable_fake_quantdisable_observerenable_fake_quantenable_observerdefault_fake_quantdefault_weight_fake_quantdefault_dynamic_fake_quant.default_fixed_qparams_range_neg1to1_fake_quant+default_fixed_qparams_range_0to1_fake_quant*default_symmetric_fixed_qparams_fake_quant'default_affine_fixed_qparams_fake_quant%default_per_channel_weight_fake_quantdefault_embedding_fake_quant!default_embedding_fake_quant_4bitdefault_histogram_fake_quantdefault_fused_act_fake_quantdefault_fused_wt_fake_quant'default_fused_per_channel_wt_fake_quant(fused_wt_fake_quant_range_neg_127_to_1274fused_per_channel_wt_fake_quant_range_neg_127_to_127qschemez torch.qschemereturncL|tjtjtjfvSN)torchper_channel_symmetricper_channel_affine per_channel_affine_float_qparamsr's _/var/www/html/ai-engine/env/lib/python3.11/site-packages/torch/ao/quantization/fake_quantize.py_is_per_channelr12s&  #   . c6|tjtjfvSr*)r+per_tensor_symmetricper_tensor_affiner/s r0_is_per_tensorr6:s u153JK KKr2c6|tjtjfvSr*)r+r4r,r/s r0_is_symmetric_quantr8>s u153NO OOr2c |tjfvSr*)r+r.r/s r0_is_float_qparamsr:Bs  . r2cPeZdZUdZejed<ejed<dfd ZedZ edZ ej j dd e ddfd Zej j d Zej j dd e ddfd Zej j dZedZxZS)raBase fake quantize module. Base fake quantize module Any fake quantize implementation should derive from this class. Concrete fake quantize module should follow the same API. In forward, they will update the statistics of the observed Tensor and fake quantize the input. They should also provide a `calculate_qparams` function that computes the quantization parameters given the collected statistics. fake_quant_enabledobserver_enabledr(Nct|dtjdgtj|dtjdgtjdS)z,Set fake_quant_enabled and observer_enabled.r<dtyper=N)super__init__register_bufferr+tensoruint8)self __class__s r0rCzFakeQuantizeBase.__init__Xsu  15<5;3W3W3WXXX /qc1U1U1UVVVVVr2cdSr*)rGxs r0forwardzFakeQuantizeBase.forwarda r2c dSr*rJ)rGkwargss r0calculate_qparamsz"FakeQuantizeBase.calculate_qparamserMr2Tenabledc"|rdnd|jd<dSNr?r)r<rGrQs r0rz"FakeQuantizeBase.enable_fake_quantis*1%8QQq"""r2c0|ddSNF)rrGs r0rz#FakeQuantizeBase.disable_fake_quantms u%%%%%r2c"|rdnd|jd<dSrS)r=rTs r0rz FakeQuantizeBase.enable_observerqs(/#611Qa   r2c0|ddSrV)rrWs r0rz!FakeQuantizeBase.disable_observerus U#####r2c .t|fi|}d|_|S)Nz#torch.ao.quantization.fake_quantize)r __module__)clsrOfake_quant_constructors r0 with_argszFakeQuantizeBase.with_argsys*!+C!:!:6!:!:-R)%%r2)r(N)T)__name__r[ __qualname____doc__r+Tensor__annotations__rCrrLrPjitexportboolrrrr classmethodr^ __classcell__rHs@r0rrHsr   $$$l"""WWWWWW  ^   ^  Y9999999 Y&&& Y77t7t7777 Y$$$&&[&&&&&r2rceZdZUdZejed<ejed<edddffd Zej j dZ dZ ej j d Z fd Zfd ZxZS) raKSimulate the quantize and dequantize operations in training time. The output of this module is given by:: x_out = ( clamp(round(x/scale + zero_point), quant_min, quant_max) - zero_point ) * scale * :attr:`is_dynamic` indicates whether the fake quantie is a placeholder for dynamic quantization operators (choose_qparams -> q -> dq) or static quantization operators (q -> dq) * :attr:`scale` defines the scale factor used for quantization. * :attr:`zero_point` specifies the quantized value to which 0 in floating point maps to * :attr:`fake_quant_enabled` controls the application of fake quantization on tensors, note that statistics can still be updated. * :attr:`observer_enabled` controls statistics collection on tensors * :attr:`dtype` specifies the quantized dtype that is being emulated with fake-quantization, allowable values are torch.qint8 and torch.quint8. Args: observer (module): Module for observing statistics on input tensors and calculating scale and zero-point. observer_kwargs (optional): Arguments for the observer module Attributes: activation_post_process (Module): User provided module that collects statistics on the input tensor and provides a method to calculate scale and zero-point. scale zero_pointNFc t||||ks Jd|dtj}t |dr4t t |didid|}tj|j|ks Jd|tj|j ks Jd| ||d||d<|di||_ |j j |_ |j j |_ |j j|_t|j jr tj}n tj}|d tjd gtj |d tjd g| |j j|_|j j|_t |j dr |j jnd|_t/|js3t1|jsJdt3|jzt/|j|_dS)Nz1quant_min must be less than or equal to quant_maxrApkeywordszquant_min out of boundzquant_max out of bound) quant_min quant_max is_dynamicrkg?r@rlrch_axiszYOnly per channel and per tensor quantization are supported in fake quantize got qscheme: rJ)rBrCgetr+quint8hasattrgetattriinfominmaxupdateactivation_post_processrprqrrr:r'floatintrDrErArsr1r6stris_per_channel) rGobserverrprqrrobserver_kwargsrAzero_point_dtyperHs r0rCzFakeQuantize.__init__s   Y%:Y&&&B'&&#''>>Ex%%  #r : :JKKOOU;u%%)Y6668P666 E 2 2 66668P666  " "#S#S T T T(2 %'/x'B'B/'B'B$5?5?6A T9A B B )${  $y  WelC5 &L&L&LMMM \5<CS+T+T+TUUU17 3; t3Y?? D ( 0 0 t|,, t|0L0L   $,    .dl;;r2c4|jSr*r}rPrWs r0rPzFakeQuantize.calculate_qparams+==???r2cF|jddkr|||\}}||jj||jj}}|jj|jkr>|j |j|j |j|j ||j ||j ddkrz|j r=tj||j|j|j|jj|jj}n6tj||j|j|jj|jj}|S)Nrr?)r=r}detachrPtorkdevicerlshaperesize_copy_r<rr+ fake_quantize_per_channel_affinersrprqfake_quantize_per_tensor_affine)rGX_scale _zero_points r0rLzFakeQuantize.forwardsz   #q ( (  ( ( 4 4 4"&"8"8":": FK"())DJ,=">"> &AAKFz6<// ""6<000'' (9::: J  V $ $ $ O ! !+ . . .  "1 % * *" :JOL0:0: 9JO0:0: r2cd|jd|jd|jjd|jjd|jd|jd|jd|jd |j S) Nfake_quant_enabled=, observer_enabled= , quant_min= , quant_max=, dtype= , qscheme=z , ch_axis=, scale= , zero_point=) r<r=r}rprqrAr'rsrkrlrWs r0 extra_reprzFakeQuantize.extra_reprs @$"9 @ @dNc @ @5? @ @MQMiMs @ @Z @ @+/< @ @CG< @ @Z @ @/3o @ @ r2ct||||j||dz<|j||dz<dSNrkrl)rB_save_to_state_dictrkrl)rG destinationprefix keep_varsrHs r0rz FakeQuantize._save_to_state_dict sI ##KCCC(, FW$%-1_ F\)***r2c ddg}|D]} || z} | |vr|| } | dkr |j| jn'| dksJ|j| jtjrC| dkr|j| | dksJ|j| |r|| t |||||||dSr) rkrrrlr+rd is_scriptingrappendrB_load_from_state_dict) rG state_dictrlocal_metadatastrict missing_keysunexpected_keys error_msgs local_statenamekeyvalrHs r0rz"FakeQuantize._load_from_state_dictsD -  ) )D4-Cj   o 7??J&&sy1111<////O++CI6669))++3w ((----#|3333--c222 )##C((( %%            r2)r_r[r`rar+rbrcr rCrdrerPrLrrrrhris@r0rrs!!F < - 1<1<1<1<1<1rsk=< ########    8_LOLLLLLPPTPPPP4 7&7&7&7&7&sF7&7&7&t{ { { { { #{ { { |# # # # # |# # # L@ @ @ @ @ L@ @ @ F"++ ( ,  # ,)22 ( +  & 3 *33 ( , 42J1S1S 92T22./G.P.P 6/Q//+ 3++V'(4(>(> 2 +  ' )?))% ,55 2  2 ,  6   %1$:$: 2  2 . %;%%! ,55  ,  # 6   =FF ( , G   <EE ( +  & F+H*Q*Q 2 +  ' +R++',I+R+R ( +  & ,S,,( "++6k+  ,5    ! ! !            r2