Ngw<ddlmZddlZddlZddlmZddlmZmZm Z m Z m Z m Z m Z ddlZddlZddlmZmZddlmZddlmZmZddlmZdd lmZdd lmZmZdd l m!Z!Gd d Z"GddZ#d)dZ$ d*d+d"Z% d,d-d(Z&dS).) annotationsN)PurePath)AnyBinaryIO CollectionListOptionalUnioncast)Image ImageSequence) TextRegion) LayoutElementLayoutElements)logger) get_model)"UnstructuredElementExtractionModel UnstructuredObjectDetectionModel) draw_bboxceZdZdZddZddZeddZedd Z e dddZ e dd dZ dS)!DocumentLayoutzClass for handling documents that are saved as .pdf files. For .pdf files, a document image analysis (DIA) model detects the layout of the page prior to extracting element.Nc||_dSN_pages)selfpagess c/var/www/html/ai-engine/env/lib/python3.11/site-packages/unstructured_inference/inference/layout.py__init__zDocumentLayout.__init__s  returnstrcJdd|jDS)N c,g|]}t|Sr").0pages r z*DocumentLayout.__str__.."s===$CII===r )joinrrs r__str__zDocumentLayout.__str__!s%{{==$*===>>>r List[PageLayout]c|jS)z1Gets all elements from pages in sequential order.rr,s rrzDocumentLayout.pages$s {r rc(|}||_|S)zCGenerates a new instance of the class from a list of `PageLayouts`sr)clsr doc_layouts r from_pageszDocumentLayout.from_pages)sSUU ! r filename fixed_layouts*Optional[List[Optional[List[TextRegion]]]] pdf_image_dpiintc vtjd|dtj5}t |||d}t t t|}t|}g} |dtd|D}tt||D]b\} \} } tj | 5} tj| f| dz|| d |}| |dddn #1swxYwYc|| cdddS#1swxYwYdS) z)Creates a DocumentLayout from a pdf file.zReading PDF for file:  ...T) output_folder path_onlyNcg|]}dSrr&r(_s rr*z,DocumentLayout.from_file..Fs I I I! I I Ir r)numberdocument_filename fixed_layout)rinfotempfileTemporaryDirectoryconvert_pdf_to_imager rr"lenrange enumeratezipr open PageLayout from_imageappendr3)r1r5r6r8kwargstemp_dir _image_paths image_pathsnumber_of_pagesri image_pathrDimager)s r from_filezDocumentLayout.from_file0s  ;X;;;<<<  ( * * )h/& L tCy,77K!+..O&(E$ I IuQ/H/H I I I 1:3{M;Z;Z1[1[ ' '--J Z ++'u%0 1u*2%1  ! DLL&&&'''''''''''''''>>%((1 ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) )s6BD./D5 D.D D.D D..D25D2detection_model*Optional[UnstructuredObjectDetectionModel]element_extraction_model,Optional[UnstructuredElementExtractionModel]rDOptional[List[TextRegion]]c tjd|d tj|}|j}g}t t j|D]6\} } | d} || _| | 7nf#t$rY} tj |stj |r| td|d| d} ~ wwxYwg} t |D]2\} }t!j|f|| |||d|} | | 3|| S)z,Creates a DocumentLayout from an image file.zReading image file: r;RGBzFile "z " not found!N)rWrBrZr\rD)rrEr rMformatrKr IteratorconvertrP ExceptionospathisdirisfileFileNotFoundErrorrNrOr3)r1r5rZr\rDrQrXraimagesrVimerr)s rfrom_image_filezDocumentLayout.from_image_fileUs  98999::: PJx((E\F(*F"=#9%#@#@AA " "2ZZ&&"  b!!!! " P P Pw}}X&& P"'..*B*B P'(G(G(G(GHHaO  P !&))  HAu(# /)A) D LL    ~~e$$$sA5B C3AC..C3rr!r")r!r.)rr.r!r)Nr4)r5r"r6r7r8r9r!r)NNN) r5r"rZr[r\r]rDr^r!r) __name__ __module__ __qualname____doc__rr-propertyr classmethodr3rYrmr&r rrrs????X[ EI ")")")")[")HGKQU37 #%#%#%#%[#%#%#%r rceZdZdZ d.d/dZd0dZ d1d2dZ d3d4dZd5dZ d6d7d'Z d8d9d)Z e d:d;d-Z dS)B!2 .(@%35 59r r!r"cJdd|jDS)Nr$c,g|]}t|Sr&r')r(elements rr*z&PageLayout.__str__..sFFFWCLLFFFr )r+r}r,s rr-zPageLayout.__str__s%{{FF FFFGGGr TOptional[List[LayoutElement]]c|jtd|jJ||j}|r ||_dS|S)zRUses end-to-end text element extraction model to extract the elements on the page.NzMCannot get elements using image extraction, no image extraction model defined)r\r{rXr})rinplacer}s r#get_elements_using_image_extractionz.PageLayout.get_elements_using_image_extractions`  ( 0_ z%%%00<<  $DM4r Frbool array_onlycr|j:t}t|tr||_nt d|jJ||j}|j|}|r$||_|s||_ dS|S)z8Uses specified model to detect the elements on the page.Nz)Default model should be a detection model) rZr isinstancerNotImplementedErrorrXdeduplicate_detected_elementsr~as_listr})rrrmodelinferred_layouts r!get_elements_with_detection_modelz,PageLayout.get_elements_with_detection_models   'KKE%!ABB W',$$)*UVVVz%%%*.*>*>tz*J*J.LL     "1D  : / 7 7 9 9 4&&(((r !Union[np.ndarray[Any, Any], None]c|jX|jrtj|j|_n2t j|j}tj||_|jS)z*Converts the raw image into a numpy array.)r|rXnparrayr rMrW)rrXs r_get_image_arrayzPageLayout._get_image_arraysV   #z 3#%8DJ#7#7   4?33#%8E?? r r4colorsOptional[Union[List[str], str]] image_dpiannotation_dataOptional[dict[str, dict]] add_detailssourcesOptional[List[str]]c L|d|jD}t|tr|g}t|t|jkr,t|jt|zdz}||z}|jr|j}nB|jrtj|j}n!| |j |j |}|:t|j|D]#\}} | |j |vrt||| |}$n|D]s\} } t!|| r^t#|| rN| d} | d} t#|| D]-} t#| dd}|||vrt|| | | |}.t|S) a{Annotates the elements on the page image. if add_details is True, and the elements contain type and source attributes, then the type and source will be added to the image. sources is a list of sources to annotate. If sources is ["all"], then all sources will be annotated. Current sources allowed are "yolox","detectron2_onnx" and "detectron2_lp" Ncg|]}dS)redr&r?s rr*z'PageLayout.annotate..s333e333r rA)colordetailsrwidthsource)rrr)r}rr"rIrXcopyrWr rM _get_imagerCrBrLrritemshasattrgetattr)rrrrrrn_copiesimgelr attributestylerregionrequired_sources rannotatezPageLayout.annotates >33T]333F fc " " XF v;;T]++ + +DM**c&kk9Q>Hh&F : R*//##CC _ R*T_--CC//$"8$+yQQC  " 77 O O E?bi7&:&:#C5+NNNC O%4$9$9$;$;   54++ i0H0H !'NE!'NE")$ ":":  *1&(D*I*I#OG1K1K"+ # &&+&+(3 ###C r r8ctj5}tj|||d}t t t |}|t|krtd|dtj ||dz 5}| cdddcdddS#1swxYwY ddddS#1swxYwYdS)z&Hotloads a page image from a pdf file.Tdpir< paths_onlyz Page number z0 is greater than the number of pages in the PDF.rAN) rFrG pdf2imageconvert_from_pathr rr"rIr{r rMr)rr5 page_numberr8rRrSrTrXs rrzPageLayout._get_image s ( * * $h$6!& L tCy,77KS---- `;```K a899 $Uzz|| $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $s6A> )T " " a GGDIIKK GGIII               )                       Ms51B2 >C#(  ADKKAABBB   &  +%=       %  +%=''       " Mr rr<ryr=#Union[List[Image.Image], List[str]]c|r|std|tj||||}ntj|||}|S)z9Get the image renderings of the pdf pages using pdf2imagez4output_folder must be specified if path_only is trueNr)rr)r{rr)r5rr<r=rjs rrHrHsxQQOPPP , '    ,      Mr )rrrrrQrr!r)FNr4)r5r"rrrrr6r7r8r9rQrr!r)r4NF) r5r"rr9r<ryr=rr!r)' __future__rrerFpathlibrtypingrrrrr r r numpyrrPILr r )unstructured_inference.inference.elementsr.unstructured_inference.inference.layoutelementrrunstructured_inference.loggerr"unstructured_inference.models.baser/unstructured_inference.models.unstructuredmodelrr unstructured_inference.visualizerrrNrrrHr&r rrs"""""" IIIIIIIIIIIIIIIIII$$$$$$$$YXXXXXXX000000888888766666`%`%`%`%`%`%`%`%FFFFFFFFFR4@D %%%%%T48 r