Ng=ZddlZddlmZmZddlmZmZmZmZm Z m Z m Z m Z ddl mZddlmZddlmZddlmZmZmZmZddlmZd ed efd Zd d d efdZe eeeedfZ GddedZGddedZ GddedZ!GddedZ"GddedZ#GddedZ$Gdd eedZ%Gdde%Z&Gd d!e%Z'Gd"d#edZ(Gd$d%edZ)d&d'd(d)d*d+Z*Gd,d-eZ+dS).N)ABCabstractmethod)AnyCallableDictListLiteralOptionalSequenceUnion)CallbackManagerForRetrieverRun)Document) BaseRetriever) BaseModelFieldmodel_validator validator) Annotatedexcerptreturncb|s|Stjdd|dd}|S)zClean an excerpt from Kendra. Args: excerpt: The excerpt to clean. Returns: The cleaned excerpt. z\s+ z...)resubreplace)rress a/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/retrievers/kendra.py clean_excerptrs9  &g & & . .ub 9 9C Jitem ResultItemcd}|}|r |d|dz }t|}|r |d|dz }|S)zCombine a ResultItem title and excerpt into a single string. Args: item: the ResultItem of a Kendra search. Returns: A combined text of the title and excerpt of the given item. rzDocument Title:  zDocument Excerpt: ) get_titler get_excerpt)r!texttitlers r combined_textr)*sr D NN  E - ,5,,,,D,,..//G3 2w2222 Kr cZeZdZUdZeed< eed< eeed< eeed<dS) Highlightz8Information that highlights the keywords in the excerpt. BeginOffset EndOffset TopAnswerTypeN) __name__ __module__ __qualname____doc__int__annotations__r boolstrr rr+r+FsVBBLNNNJ~7 3-< "> ! > "> !   (' '   $# #tr N) r0r1r2r3r r7r5r4rpropertyDocumentAttributeValueTyperTr8r rrNrNws((}3}d3i(((( # 1Xr rNc*eZdZUdZeed< eed<dS)DocumentAttributezDocument attribute.rDrGN)r0r1r2r3r7r5rNr8r rrXrXs0 HHH# !!!!%%r rXc.eZdZUdZeeed< eeed< eeed< gZeee ed< ee ed< e defdZ e defd Z de fd Zdeeeffd Zdefd Zefd edgefdefdZdS)r"zBase class of a result item.Id DocumentId DocumentURIDocumentAttributesScoreAttributesrcdS)zDocument title.Nr8rJs rr%zResultItem.get_titler cdS)zDDocument excerpt or passage original content as retrieved by Kendra.Nr8rJs rr&zResultItem.get_excerptr`r ciS)zDocument additional metadata dict. This returns any extra metadata except these: * result_id * document_id * source * title * excerpt * document_attributes r8rJs rget_additional_metadataz"ResultItem.get_additional_metadatas  r c(d|jpgDS)zDocument attributes dict.c2i|]}|j|jjSr8)rDrGrT).0attrs r z;ResultItem.get_document_attributes_dict..s!WWWt$**WWWr )r]rJs rget_document_attributes_dictz'ResultItem.get_document_attributes_dictsWW8O8USUWWWWr c.|j |jdSdS)zDocument Score ConfidenceNScoreConfidence NOT_AVAILABLE)r^rJs rget_score_attributezResultItem.get_score_attributes   +'(9: :"?r page_content_formatterc J||}|}||j|j|j||||dt||S)z!Converts this item to a Document.) result_id document_idsourcer(rdocument_attributesscore) page_contentmetadata) rcupdaterZr[r\r%r&rirmr)rKrnrurvs rto_doczResultItem.to_docs.-d33 //11!W#*))++--'+'H'H'J'J1133   \HEEEEr N)r0r1r2r3r r7r5r]rrXdictrr%r&rcrrVrirmr)rrrxr8r rr"r"s~&& - #<>&7!89>>>"d^###%3^SSSSS^S     Xd38R3R.SXXXX#S####GTFF&. ~s/B&CF FFFFFFr ceZdZUdZeed< eeed< eeed< eeed< gZee e ed< eeed< defd Z defd Z defd Z defd Zd S)QueryResultItemzQuery API result item. DocumentTitle FeedbackTokenFormatr/AdditionalAttributesDocumentExcerptrc|jjSrI)r|r=rJs rr%zQueryResultItem.get_titles!&&r cp|jsdS|jdsdS|jdS)Nrr)rrLrJs rget_attribute_valuez#QueryResultItem.get_attribute_valuesB( 2(+ A2,Q/>>@@ @r c|jr+|jdjdkr|}n|jr |jj}nd}|S)Nr AnswerTextr)rrDrrr=)rKrs rr&zQueryResultItem.get_excerptsX  % )!,0L@@..00GG  ! */GGGr cd|ji}|S)Ntype)r/)rKadditional_metadatas rrcz'QueryResultItem.get_additional_metadata s%ty1""r N)r0r1r2r3r<r5r r7rrrCr%rr&ryrcr8r rr{r{s   %%%%C=   A SM 3-?FH(4(A#BCHHHG01111''3''''ASAAAA S    #######r r{c\eZdZUdZeeed< eeed< defdZdefdZdS)RetrieveResultItemzRetrieve API result item.r|Contentrc|jpdSNr)r|rJs rr%zRetrieveResultItem.get_titles!'R'r c|jpdSr)rrJs rr&zRetrieveResultItem.get_excerpts|!r!r N) r0r1r2r3r r7r5r%r&r8r rrrsx##C=    c]"(3(((("S""""""r rc*eZdZUdZeeed<dS) QueryResulta`Amazon Kendra Query API` search result. It is composed of: * Relevant suggested answers: either a text excerpt or table excerpt. * Matching FAQs or questions-answer from your FAQ file. * Documents including an excerpt of each document with its title. ResultItemsN)r0r1r2r3rr{r5r8r rrr"s/o&&&&r rc6eZdZUdZeed< eeed<dS)RetrieveResultz`Amazon Kendra Retrieve API` search result. It is composed of: * relevant passages or text excerpts given an input query. QueryIdrN)r0r1r2r3r7r5rrr8r rrr0s< LLL())))r rg?g?g??)rlLOWMEDIUMHIGH VERY_HIGHc2eZdZUdZeed<dZeeed<dZeeed<dZ e ed<dZ ee ed<dZ eee ed <eZeegefed <eed <dZee ed <eeeed dfed<edde de fdZedede eefdefdZdedeefdZdeedeefdZ deedeefdZ!dede"deefdZ#dS) AmazonKendraRetrievera`Amazon Kendra Index` retriever. Args: index_id: Kendra index id region_name: The aws region e.g., `us-west-2`. Fallsback to AWS_DEFAULT_REGION env variable or region specified in ~/.aws/config. credentials_profile_name: The name of the profile in the ~/.aws/credentials or ~/.aws/config files, which has either access keys or role information specified. If not specified, the default credential profile or, if on an EC2 instance, credentials from IMDS will be used. top_k: No of results to return attribute_filter: Additional filtering of results based on metadata See: https://docs.aws.amazon.com/kendra/latest/APIReference document_relevance_override_configurations: Overrides relevance tuning configurations of fields/attributes set at the index level See: https://docs.aws.amazon.com/kendra/latest/APIReference page_content_formatter: generates the Document page_content allowing access to all result item attributes. By default, it uses the item's title and excerpt. client: boto3 client for Kendra user_context: Provides information about the user context See: https://docs.aws.amazon.com/kendra/latest/APIReference Example: .. code-block:: python retriever = AmazonKendraRetriever( index_id="c0806df7-e76b-4bce-9b5c-d5582f6b1a03" ) index_idN region_namecredentials_profile_nametop_kattribute_filter*document_relevance_override_configurationsrnclient user_contextrr)gelemin_score_confidencerTrc8|dkrtd|d|S)Nrtop_k () cannot be negative.) ValueError)clsrTs rvalidate_top_kz$AmazonKendraRetriever.validate_top_k{s* 199CuCCCDD D r before)modevaluesc|d}||dkrtd|d|d|S ddl}|dr||d}n|}i}|dr |d|d<|jd i||d<|S#t $rt d t $r}td |d}~wwxYw) Nrrrrrr) profile_namerkendrazRCould not import boto3 python package. Please install it with `pip install boto3`.zCould not load credentials to authenticate with AWS client. Please check that credentials in the specified profile name are valid.)r)getrboto3Sessionr ImportError Exception)rrrrsession client_paramses r create_clientz#AmazonKendraRetriever.create_clientsQ 7##  CuCCCDD D ::h   +M  LLLzz455 *--V    *   s A=C"C>)C99C>queryc|j|dd|jd}|j |j|d<|j |j|d<|j |j|d<|jjdi|}t |}|j r|j S|jj di|}t |}|j S)Nri)IndexId QueryTextPageSizeAttributeFilter'DocumentRelevanceOverrideConfigurations UserContextr8) rstriprrrrrretriever parse_objrrr)rKr kendra_kwargsresponser_resultq_results r _kendra_queryz#AmazonKendraRetriever._kendra_querys}qu-     ,/3/DM+ ,  : F? C D   (+/+fd|djD}|S)NcDg|]}|jSr8)rxrnrfr!rKs r z9AmazonKendraRetriever._get_top_k_docs..s8    KK3 4 4   r )r)rKrtop_docss` r_get_top_k_docsz%AmazonKendraRetriever._get_top_k_docss=    $\tz\2   r docsc6js|Sfd|D}|S)zr Filter out the records that have a score confidence greater than the required threshold. cg|]m}|jdQt|jdtr1t|jddjkk|nS)rtNr)rvr isinstancer7KENDRA_CONFIDENCE_MAPPINGrrs rrzEAmazonKendraRetriever._filter_by_score_confidence..s     !!'**6t}W5s;;7-11$-2H#NN,--  ---r )r)rKr filtered_docss` r_filter_by_score_confidencez1AmazonKendraRetriever._filter_by_score_confidencesE ( K         r run_managerc||}||}||S)zRun search on Kendra index and get top k documents Example: .. code-block:: python docs = retriever.invoke('This is my query') )rrr)rKrrr top_k_docss r_get_relevant_documentsz-AmazonKendraRetriever._get_relevant_documentss>))%00 )),77 // ;;;r )$r0r1r2r3r7r5rr rrr4rrrrr)rnrr"rrrfloatrrrr classmethodrr rrrrr rr8r rrrFs*''RMMM!%K#%%%.2hsm222E3NNN'+htn+++GK.d0DKKK:GHj\3%67GGG KKK#'L(4.'''#HUOUUcc5J5J5J$JKKKKYw33 _(###"4S>"c"""[$#"H$3$8J+?$$$$6HZ,@T(^X4>&<<4 < h <<<<<rs ########                    DCCCCC------333333 (''''' 3 3      (#3T#Y#<= = = = = =  = = = ='***** W**** 7 7 7 7 7  7 7 7 7Yg>&&&&& &&&&>F>F>F>F>FCw>F>F>F>FB.#.#.#.#.#j.#.#.#b " " " " " " " "     )7         Yg       f<f<f<f<f