Ng[y bdZddlZddlmZddlmZddlZddlZej gdZ dZ ej dddZ eGd d ZeGd d ZGd dZedkrddlmZededgZededgZeeeD]r\ZZeeeZe\ZZedejdeeedejdeeqdSdS)zZ Implements object detection metrics: average precision, precision, recall, and f1 score. N) dataclass)Path) g?g?g333333?g?gffffff?g?g?g333333?g?gffffff?g?g)\(?g{Gz?c<eZdZUdZeed<eed<eed<eed<dS)#ObjectDetectionAggregatedEvaluationzGClass representing a gathered class-aggregated object detection metricsf1_score precisionrecallm_apN)__name__ __module__ __qualname____doc__float__annotations__a/var/www/html/ai-engine/env/lib/python3.11/site-packages/unstructured/metrics/object_detection.pyrrs=QQOOO MMM KKKKKrrceZdZUdZeeefed<eeefed<eeefed<eeefed<edZ dS)!ObjectDetectionPerClassEvaluationz@Class representing a gathered object detection metrics per-classrrr r cpfdttD}fdttDfdttDfdttD}|||S)Nc.i|]}||Srr).0i class_labelsf1s r zBObjectDetectionPerClassEvaluation.from_tensors..(s#MMMqLORUMMMrc.i|]}||Srr)rrrrs rrzBObjectDetectionPerClassEvaluation.from_tensors..)s#UUUq\!_ilUUUrc.i|]}||Srr)rrrr s rrzBObjectDetectionPerClassEvaluation.from_tensors..*s#OOO,q/6!9OOOrc.i|]}||Srr)rraprs rrzBObjectDetectionPerClassEvaluation.from_tensors..+s#III1 QAIIIr)rangelen)clsr rr rrrr s ````` r from_tensorsz.ObjectDetectionPerClassEvaluation.from_tensors&sMMMMME#l:K:K4L4LMMMUUUUUE#lBSBSz9ObjectDetectionEvalProcessor.__init__..Os#JJJ4twwvJJJrc:g|]}|Srr2)rtargetr/s rr5z9ObjectDetectionEvalProcessor.__init__..Ps% R R Rv6!2!2 R R RrN)r/r+r,r-r.r)selfr+r,r-r.rr/s `r__init__z%ObjectDetectionEvalProcessor.__init__5sc2 JJJJ>JJJ R R R RAQ R R R(&(rprediction_file_pathground_truth_file_pathreturncjt|5}tj|}dddn #1swxYwYt|5}tj|}dddn #1swxYwYt|dt|dks Jdt |dt |dks Jdt t|ddt|ddD]\}}|d |d ks-Jd |jd |d d |jd |d d |d }|d|dksZJd|jd |ddd|ddd |jd |ddd|ddd|d|d} ||| d} ||| } ||\} } || | | | | S)aI Initializes the ObjectDetection prediction and ground truth, and converts the data to the required format. Args: prediction_file_path (Path): path to json file with predictions dump from OD model ground_truth_file_path (Path): path to json file with OD ground truth data Nobject_detection_classesz5Classes in predictions and ground truth do not match.pagesz:Pages number in predictions and ground truth do not match.c|dSNnumberrps rz>ObjectDetectionEvalProcessor.from_json_files..qs AhKr)keyc|dSrArrCs rrEz>ObjectDetectionEvalProcessor.from_json_files..rs Qx[rrBzPage numbers in predictions z (z) and ground truth z) do not match.sizezPage sizes in predictions rz x z) do not match for page .T) prediction) openjsonloadsortedr"zipname _process_data_parse_page_dimensions)r#r:r;fpredictions_dataground_truth_data pred_pagegt_pagepage_numrr+r,r-r.s rfrom_json_filesz,ObjectDetectionEvalProcessor.from_json_filesUs & ' ' ,1#y||  , , , , , , , , , , , , , , , ( ) ) -Q $ !  - - - - - - - - - - - - - - -&'ABCCv 8 9H H    B   #G,-- g &2 2    G   #& #G,2G2G H H H $W-3H3H I I I# #    IwX&'(*;;;;7/C/H77h'77q-AKK$:$?KKCJ6?STCUKK6?1%KK@HKKK8777((BC **+;\VZ*[[,,-> MM$'$>$>?O$P$P! ks>#3\;P\]]]s155 A,,A03A0c|g}t|j|j|j|jD]5\}}}}|||||}||6d\}}} } t|j} tj | tj } tj | tj } tj | tj }tj | tj }t|rIdtt|D}|j |\}}}}}|||} } }|}|d}|d}|d}|d}t|D]e\}}t!||| |<t!||| |<t!||||<t!||||<ft"| | |||j}t't!| t!|t!| t!|}||fS)zGet per document OD metrics. Returns: tuple: Tuple of ObjectDetectionAggregatedEvaluation and ObjectDetectionPerClassEvaluation )predstargetsheightwidth)r`r`r`c8g|]}tj|dS)r)torchcat)rxs rr5z.s"$]$]$]UYq!__$]$]$]rrI)r rr rr)rrr r )rPr+r,r-r. _compute_page_detection_matchingappendr"rnpfullnanlist_compute_detection_metricsmean enumeraterrr$r)r8document_matchingsr\r]r^r_page_matching_tensorsmean_apmean_precision mean_recallmean_f1num_clsmean_ap_per_classmean_precision_per_classmean_recall_per_classmean_f1_per_classmatching_info_tensorsap_per_present_classesprecision_per_present_classesrecall_per_present_classesf1_per_present_classespresent_classes ap_per_classprecision_per_classrecall_per_class f1_per_classr class_indexod_per_class_evaluation od_evaluations r get_metricsz(ObjectDetectionEvalProcessor.get_metricss -0  !68I4K[. . = = )E7FE%)$I$I %J%% !  % %&; < < < <9 5gd'((GGRV44#%77BF#;#; " 8 8GGRV44 ! " "% H$]$]d3HZC[>\>\$]$]$] !0/&  &-*&.2244*//11&++--*1KN-1133G266q99L"?"D"DQ"G"G 9>>qAA 166q99L"+O"<"< H H;16|A1G1G!+.8=>QRS>T8U8U(55:;KA;N5O5O%k216|A1G1G!+.."C"P"P .( * #Q# # <7^^N++%%w    555rdatacg}g}|dD]D}||dd||ddE||fS)zX Process the page dimensions from the json file to the required format. r?rHr^r_)rf)rr-r.pages rrSz3ObjectDetectionEvalProcessor._parse_page_dimensionssi   M 6 6D   V X 6 7 7 7   tF|G4 5 5 5 5[((rFrKc `g}|dD]}g}|dD]l}|d}||}|d\} } } } |r$|d} || | | | | |gR||| | | | gmtj|}|||S)zQ Process the elements from the json file to the required format. r?elementstypebboxprob)indexrfrbtensor)rrrK pages_listr page_elementselement class_label class_idxx1y1x2y2 confidence page_tensors rrRz*ObjectDetectionEvalProcessor._process_datas  M + +DM + F F%fo (..{;; !(BBF!(J!(("b"b*i)PQQQQ!(()RR)DEEEE,}55K   k * * * *r preds_scores preds_clstop_kctj|}|ddtj|dz|jddk}|dd|z}|dd\}}|d|ddfd }||dd } | dS) a Get the indexes of all the top k predictions for every class Args: preds_scores: The confidence scores, vector of shape (n_pred) preds_cls: The predicted class, vector of shape (n_pred) top_k: Number of predictions to keep per class, ordered by confidence score Returns: top_k_idx: Indexes of the top k predictions. length <= (k * n_unique_class) rIr/rT descendingNFas_tupledim)rbmaxviewaranger/sortnonzerosplit) rrr n_unique_clsmaskpreds_scores_per_clssorted_scores_per_cls sorting_idxidx_with_satisfying_scores top_k_idxs r_get_top_k_idx_per_clsz3ObjectDetectionEvalProcessor._get_top_k_idx_per_cls s y++ ~~b!$$ 1 \%8) ) ) $q"++ ,00Q77$>-A-F-FqUY-F-Z-Z*{%:6E61119%E%M%MW\%M%]%]" : @ @ @ J JK ~~b!!!rboxes img_shapec|dddgfd|d|dddgf<|dddgfd|d|dddgf<|S)a* Clips bboxes to image boundaries. Args: bboxes: Input bounding boxes in XYXY format of [..., 4] shape img_shape: Image shape (height, width). Returns: clipped_boxes: Clipped bboxes in XYXY format of [..., 4] shape .rrI)minr)clip)rrs r"_change_bbox_bounds_for_image_sizez?ObjectDetectionEvalProcessor._change_bbox_bounds_for_image_size$su#3A;/44 ! 4MMcAq6k"3A;/44 ! 4MMcAq6k rbox1box2cxd}||j}||j}tj|dddddf|ddddftj|dddddf|ddddfz dd}||dddf|z|z z S)a Return intersection-over-union (Jaccard index) of boxes. Both sets of boxes are expected to be in (x1, y1, x2, y2) format. Args: box1: Tensor of shape [N, 4] box2: Tensor of shape [M, 4] Returns: iou: Tensor of shape [N, M]: the NxM matrix containing the pairwise IoU values for every element in boxes1 and boxes2 cH|d|dz |d|dz zS)NrrrrIr)boxs rbox_areaz7ObjectDetectionEvalProcessor._box_iou..box_areaFs%FSVOAQ8 8rNrr)Trbrrclampprod)rrrarea1area2inters r_box_iouz%ObjectDetectionEvalProcessor._box_iou6s  9 9 9    YtAAAtQRRK($qqq!""u+ 6 644QSRSQS CTVZ[\[\[\^`_`^`[`Va9b9b b U1XX T!WW  aaag.677rpreds_box_xyxytargets_box_xyxy targets_cls preds_matchedtargets_matchedpreds_idx_to_useiou_thresholdsc 8||||} ||dd|ddk} d| | <| dd\} } | |dkdD]\} }|| }| | |f}| | |f|k}t j||ddf||ddf}t j||}d|||f<d|||f<|rn|S) a Computes the matching targets based on IoU for regular scenarios. Args: preds_box_xyxy: (torch.Tensor) Predicted bounding boxes in XYXY format. preds_cls: (torch.Tensor) Predicted classes. targets_box_xyxy: (torch.Tensor) Target bounding boxes in XYXY format. targets_cls: (torch.Tensor) Target classes. preds_matched: (torch.Tensor) Tensor indicating which predictions are matched. targets_matched: (torch.Tensor) Tensor indicating which targets are matched. preds_idx_to_use: (torch.Tensor) Indices of predictions to use. Returns: targets: Computed matching targets. rrIrT)rstableFrN)rrrrrb logical_andall)r8rrrrrrrriou cls_mismatch sorted_iou target_sortedpred_selected_itarget_sorted_ipred_itarget_iis_iou_above_thresholdare_candidates_freeare_candidates_goods r_compute_targetsz-ObjectDetectionEvalProcessor._compute_targetsUs8mmN+;<>NOO !!1277A>>+BRBRSTVXBYBYY L %(HHTH$J$J! M2O1O0X0X1Y1 1    ,O_ &o6F$_o%EFH&00P%QTb%b "#("3vqqqy))OHaaaK,H+H##  #("34JL_"`"`  >BOH&99 :9=M&"55 6""$$  rdTr\r]r^r_ return_on_cpuc |j|j}t|}|t|dkrt jd|ftj|j} t jd|ftj|j} t jgtj|j} t jgtj|j} |dddf|j} | | | | | fSt jt||tj|j} t jt||tj|j}t jt||tj|j} |dddf|ddddf|dddf} }} |dddf|ddddf}} | | | |}d | ddddf<d | |<t|dkr4| |||f| || || | |||} | | | | | fS) aC Match predictions (NMS output) and the targets (ground truth) with respect to metric and confidence score for a given image. Args: preds: Tensor of shape (num_img_predictions, 6) format: (x1, y1, x2, y2, confidence, class_label) where x1,y1,x2,y2 are according to image size targets: targets for this image of shape (num_img_targets, 5) format: (label, x1, y1, x2, y2) where x1,y1,x2,y2 are according to image size height: dimensions of the image width: dimensions of the image top_k: Number of predictions to keep per class, ordered by confidence score return_on_cpu: If True, the output will be returned on "CPU", otherwise it will be returned on "device" Returns: preds_matched: Tensor of shape (num_img_predictions, n_thresholds) True when prediction (i) is matched with a target with respect to the (j)th threshold preds_to_ignore: Tensor of shape (num_img_predictions, n_thresholds) True when prediction (i) is matched with a crowd target with respect to the (j)th threshold preds_scores: Tensor of shape (num_img_predictions), confidence score for every prediction preds_cls: Tensor of shape (num_img_predictions), predicted class for every prediction targets_cls: Tensor of shape (num_img_targets), ground truth class for every target rNr)dtyper/rrITF) rr3r/r"rbzerosboolrfloat32rrr)r8r\r]r^r_rr thresholdsnum_thresholdsrpreds_to_ignorerrrr preds_box targets_boxrs rrez=ObjectDetectionEvalProcessor._compute_page_detection_matchingspR(++4;+?? Z =CJJ!OO!KN(;5:VZVabbbM#k1n*=UZX\XcdddO <%- TTTL Ru}T[QQQI!!!!Q$-**$+*>>K /<KW W JJej    + LL. 4;    + JJej   .3111b5\5AaC=%PQPQPQSTPT+l9 #*111a4='!!!QqS&/[  66|YPUVV $111,1() w<*7)9)9$+)F)FHZHZ KI I   OODK ( ( LL % % NN4; ' '"-i !255dkBB.k227799">22M4G4K [';/ D D DK+ 6t{KKK g{3DKHHH"+N";"; * * E;.7;.F{*+M77+M: / >)-8)--//"3 / 8  "BuaaaxL"/IeQQQh )F5!!!8   ]V #y6'9E'A B9fb.88rrrrc&|jd}|}tjtj|tj|} t |dkrPtj||jtj||jtj||jfS|jr|jtj ur tj n|j} tj | | d} || ddf}| | ddf} ||  }tj|dtj} tj| dtj} | |z }| | | ztjtjjzz }|ddjd}tj| | d}|dkr7tj||j}tj||j}n||d z }||d z }|d d|d }tj|j |d j}tj|tjd ||jfd }tj||d }|d}|||fS) a Compute the list of precision, recall and MaP of a given class for every recall threshold. Args: preds_matched: Tensor of shape (num_predictions, n_thresholds) True when prediction (i) is matched with a target with respect to the(j)th threshold preds_to_ignore Tensor of shape (num_predictions, n_thresholds) True when prediction (i) is matched with a crowd target with respect to the (j)th threshold preds_scores: Tensor of shape (num_predictions), confidence score for every prediction n_targets: Number of target boxes of this class recall_thresholds: Tensor of shape (max_n_rec_thresh) list of recall thresholds used to compute MaP score_threshold: Minimum confidence score to consider a prediction for the computation of precision and recall (not MaP) Returns: ap, precision, recall: Tensors of shape (nb_thrs) rrrTrN)axisr)rightrIFr)inputrr)rrbr logical_notr"rr/is_cudarruint8argsortr3 contiguouscumsumrfinfofloat64epsflipcummaxvalues searchsortedrrepeatrrcgatherrl)r8rrrrrrrtpsfpsrsort_ind rolling_tps rolling_fpsrolling_recallsrolling_precisionslowest_score_above_thresholdr rrecall_threshold_idxsampled_precision_pointsr s rrz?ObjectDetectionEvalProcessor._compute_detection_metrics_per_cls@s"@$)"-   m , ,e.?.P.P   s88q== K <<< K <<< K <<< # $(4(:ej(H(H KK#  =!7!7DIII(AAA+(AAA+#H-88:: l3QekBBB l3QekBBB % 1( + % EM(B(B(F F  044Q77>>qAAHMMaPP(-'9 MO+4( ( ( $ )A - -[T[AAAF DKII%%AA%EFF*+G!+KLI .221b99@@aPP %1   ( ( * *,=U     #Y Q DK!P!P!P QWX    $)<$,@a$ $ $  & * *1 - -9f$$rN)r*)F)rT)%r r r IOU_THRESHOLDSrSCORE_THRESHOLDrRECALL_THRESHOLDSrrjrbTensorintr&r9r'rrZtuplerrr staticmethodr%rSrrRrrgndarrayrrrrerkrrrrrr)r)0s#N%O)))U\*)u|,)3i ) #Y ) 3i )))))@2^"2^!%2^ ( 2^2^2^[2^hZ6 24UU VZ6Z6Z6Z6x )T )eD$J.? ) ) )\ )DDTRVZ\,"l"/4|"DG" """\"4z&+CHo \"8u|85<8EL888\8r3sa !!!!!! TTT ELD$//   6666666 6$I %I %I %I %I %I %I %I %X z"""""""T"<==ttD_?`?`a  ())  )** 9<699 d d446EE "8  &4%?%?%A%A"" N38NNVVG__NNOOO b'='BbbvvN_G`G`bbcccc' d dr