NgRddlmZddlZddlZddlmZmZmZmZm Z m Z m Z m Z m Z ddlZddlmZddlmZddlmZddlmZerddlZejZdZGd d eZdS) ) annotationsN) TYPE_CHECKINGAnyDictIterableListOptionalSetTupleType)Document) Embeddings) VectorStore)maximal_marginal_relevancec\eZdZUdZdZded<edddfdGdZedHdZ dIdJdZ dKdZ e ddfdLd$Z e ddfdMd&Z dNdOd(Zde dddfdPd,Z dQdRd2Z dQdSd4Z dTdUd;Z dVdWd<Z dVdXd=ZdKd>ZdKd?ZdYd@ZdZdAZeddeddfd[dDZededdfd\dFZdS)]AwaDBz`AwaDB` vector store.langchain_awadbstr_DEFAULT_TABLE_NAMEN table_name embeddingOptional[Embeddings]log_and_data_dir Optional[str]clientOptional[awadb.Client]kwargsrreturnNonec  ddl}n#t$rtdwxYw|||_n,||j||_n|j|_||jkrA|dz }|t t jddz }|j |i|_ | ||j |<||_ dS)a@Initialize with AwaDB client. If table_name is not specified, a random table name of `_DEFAULT_TABLE_NAME + last segment of uuid` would be created automatically. Args: table_name: Name of the table created, default _DEFAULT_TABLE_NAME. embedding: Optional Embeddings initially set. log_and_data_dir: Optional the root directory of log and data. client: Optional AwaDB client. kwargs: Any possible extend parameters in the future. Returns: None. rNzRCould not import awadb python package. Please install it with `pip install awadb`._-) awadb ImportError awadb_clientClientrruuiduuid4splitCreatetable2embeddingsusing_table_name)selfrrrrrr%s b/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/vectorstores/awadb.py__init__zAwaDB.__init__s.  LLLL   >     &D  +$0EL1A$B$B!!$0ELNN! 1 1 1 # J #djll++11#66r: :J   ,,,79  09D !* - *s!cF|j|jvr|j|jSdSN)r.r-)r/s r0 embeddingszAwaDB.embeddingsKs)  D$9 9 9()>? ?ttexts Iterable[str] metadatasOptional[List[dict]]is_duplicate_textsOptional[bool] List[str]c |jtdd}|j|jvr2|j|jt |}|jdd||||S)aRun more texts through the embeddings and add to the vectorstore. Args: texts: Iterable of strings to add to the vectorstore. metadatas: Optional list of metadatas associated with the texts. is_duplicate_texts: Optional whether to duplicate texts. Defaults to True. kwargs: any possible extend parameters in the future. Returns: List of ids from adding the texts into the vectorstore. NAwaDB client is None!!!embedding_texttext_embedding)r' ValueErrorr.r-embed_documentslistAddTexts)r/r6r8r:rr4s r0 add_textszAwaDB.add_textsQs"   $677 7  D$9 9 9.t/DEUUU J ))          r5boolc b|jtd|j|S)zLoad the local specified table. Args: table_name: Table name kwargs: Any possible extend parameters in the future. Returns: Success or failure of loading the local specified table Nr>)r'rALoad)r/rrs r0 load_localzAwaDB.load_localts3   $677 7 %%j111r5querykinttext_in_page_content meta_filterOptional[dict]List[Document]c |jtdd}|j|jvr&|j|j|}n#ddlm}||}hd}||||||S)a8Return docs most similar to query. Args: query: Text query. k: The maximum number of documents to return. text_in_page_content: Filter by the text in page_content of Document. meta_filter (Optional[dict]): Filter by metadata. Defaults to None. E.g. `{"color" : "red", "price": 4.20}`. Optional. E.g. `{"max_price" : 15.66, "min_price": 4.20}` `price` is the metadata field, means range filter(4.20<'price'<15.66). E.g. `{"maxe_price" : 15.66, "mine_price": 4.20}` `price` is the metadata field, means range filter(4.20<='price'<=15.66). kwargs: Any possible extend parameters in the future. Returns: Returns the k most similar documents to the specified text query. Nr>r AwaEmbedding>_idscorer@rMrNnot_include_fields_in_metadata) r'rAr.r- embed_queryr%rS Embeddingsimilarity_search_by_vector) r/rJrKrMrNrrrSnot_include_fieldss r0similarity_searchzAwaDB.similarity_searchs4   $677 7  D$9 9 9-d.CDPPQVWWII * * * * * *$ 0077I'I'I'I//  !5#+= 0   r5List[Tuple[Document, float]]c |jtdd}|j|jvr&|j|j|}n#ddlm}||}g}ddh} |||||| } | D]0} | j d} | j d=| | f} | | 1|S) aThe most k similar documents and scores of the specified query. Args: query: Text query. k: The k most similar documents to the text query. text_in_page_content: Filter by the text in page_content of Document. meta_filter: Filter by metadata. Defaults to None. kwargs: Any possible extend parameters in the future. Returns: The k most similar documents to the specified text query. 0 is dissimilar, 1 is the most similar. Nr>rrRr@rTrVrU) r'rAr.r-rXr%rSrYrZmetadataappend)r/rJrKrMrNrrrSresultsr[retrieval_docsdocrU doc_tuples r0similarity_search_with_scorez"AwaDB.similarity_search_with_scores,   $677 7  D$9 9 9-d.CDPPQVWWII * * * * * *$ 0077I02(8%'@99  !5#+= :  " & &CL)E W%e I NN9 % % % %r5rc  |j||fi|Sr3)re)r/rJrKrs r0(_similarity_search_with_relevance_scoresz.AwaDB._similarity_search_with_relevance_scoress! 1t0DDVDDDr5Optional[List[float]]rWOptional[Set[str]]c n|jtdg}||S|j|||||}|dkr|S|ddD]P} d} i} | D]#} | dkr | | } || |vr| | | | <$|t | | Q|S) a Return docs most similar to embedding vector. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. text_in_page_content: Filter by the text in page_content of Document. meta_filter: Filter by metadata. Defaults to None. not_incude_fields_in_metadata: Not include meta fields of each document. Returns: List of Documents which are the most similar to the query vector. Nr>)rMrNr[r ResultItemsr? page_contentr_)r'rASearch__len__r`r ) r/rrKrMrNrWrra show_results item_detailcontent meta_dataitem_keys r0rZz!AwaDB.similarity_search_by_vectors,   $677 7"$  N(//  !5#= 0      ! !Q & &N'?=9 O OKGI' < <///)(3G3?#AAA &1(&; (## NN89MMM N N N Nr5?fetch_k lambda_multfloatc N|jtdg}|j|jvr&|j|j|}n#ddlm} | |}|dkrgS| ||||||} | S)aReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. text_in_page_content: Filter by the text in page_content of Document. meta_filter (Optional[dict]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. Nr>rrR)ryrMrN) r'rAr.r-rXr%rSrYrp'max_marginal_relevance_search_by_vector) r/rJrKrxryrMrNrrrSras r0max_marginal_relevance_searchz#AwaDB.max_marginal_relevance_search)s:   $677 7!#  D$9 9 9-d.CDPPQVWWII * * * * * *$ 0077I     ! # #I>>  #!5# ?  r5 List[float]c |jtdg}||Sddh} |||||| } g} | D]"} | | jd#t t j|t j| } | D]:}d| |jvr)| |jd=|| |;|S) aReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. text_in_page_content: Filter by the text in page_content of Document. meta_filter (Optional[dict]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. Nr>rTrUrVr@)dtype)embedding_list) r'rArZr`r_rnparrayfloat32)r/rrKrxryrMrNrrar[retrieved_docstop_embeddingsrc selected_docss_ids r0r|z-AwaDB.max_marginal_relevance_search_by_vector^s<   $677 7"$  N#('"299  !5#+= :  ! B BC  ! !#,/?"@ A A A A2 HYbj 1 1 1.   " 5 5D>$#7#@@@"4(12BC~d3444r5idsOptional[List[str]]r[limit Optional[int]Dict[str, Document]c |jtd|j|||||}i}|D]N} d} i} | D])} | dkr | | } | dks| dkr| | | | <*t| | } | || d<O|S) aReturn docs according ids. Args: ids: The ids of the embedding vectors. text_in_page_content: Filter by the text in page_content of Document. meta_filter: Filter by any metadata of the document. not_include_fields: Not pack the specified fields of each document. limit: The number of documents to return. Defaults to 5. Optional. Returns: Documents which satisfy the input conditions. Nr>)rrMrNr[rrlr?r@rTrm)r'rAGetr )r/rrMrNr[rr docs_detailra doc_detailrs meta_infofieldrcs r0getz AwaDB.gets,   $677 7'++!5#1 ,  (*% - -JGI# 5 5,,,(/G...%5..#-e#4 %  )DDDC),GJu% & &r5c |jtdd}||dkr|S|j|}|S)aJDelete the documents which have the specified ids. Args: ids: The ids of the embedding vectors. **kwargs: Other keyword arguments that subclasses might use. Returns: Optional[bool]: True if deletion is successful. False otherwise, None if not implemented. Nr>r)r'rArpDelete)r/rrrets r0deletez AwaDB.deletesX   $677 7" ;#++--1,,J&&s++ r5c j|jtd|j|d||S)a@Update the documents which have the specified ids. Args: ids: The id list of the updating embedding vector. texts: The texts of the updating documents. metadatas: The metadatas of the updating documents. Returns: the ids of the updated documents. Nr>r?)rtext_field_namer6r8)r'rA UpdateTexts)r/rr6r8rs r0updatez AwaDB.updatesF"   $677 7 ,,%5Ui-   r5c ^|jdS|j|}|r||_|S)zCreate a new table.NF)r'r,r.r/rrrs r0 create_tablezAwaDB.create_tables=   $5&&z22  /$.D ! r5c ^|jdS|j|}|r||_|S)zJUse the specified table. Don't know the tables, please invoke list_tables.NF)r'User.rs r0usez AwaDB.use s=   $5##J//  /$.D ! r5c F|jgS|jS)z*List all the tables created by the client.)r' ListAllTablesr/rs r0 list_tableszAwaDB.list_tabless'   $I ..000r5c |jS)zGet the current table.)r.rs r0get_current_tablezAwaDB.get_current_table(s $$r5cls Type[AwaDB]c R|||||}||||S)a3Create an AwaDB vectorstore from a raw documents. Args: texts (List[str]): List of texts to add to the table. embedding (Optional[Embeddings]): Embedding function. Defaults to None. metadatas (Optional[List[dict]]): List of metadatas. Defaults to None. table_name (str): Name of the table to create. log_and_data_dir (Optional[str]): Directory of logging and persistence. client (Optional[awadb.Client]): AwaDB client Returns: AwaDB: AwaDB vectorstore. )rrrr)r6r8)rE) rr6rr8rrrrr's r0 from_textszAwaDB.from_texts0sG0s!-    Ui@@@r5 documentsc hd|D}d|D}|||||||S)avCreate an AwaDB vectorstore from a list of documents. If a log_and_data_dir specified, the table will be persisted there. Args: documents (List[Document]): List of documents to add to the vectorstore. embedding (Optional[Embeddings]): Embedding function. Defaults to None. table_name (str): Name of the table to create. log_and_data_dir (Optional[str]): Directory to persist the table. client (Optional[awadb.Client]): AwaDB client. Any: Any possible parameters in the future Returns: AwaDB: AwaDB vectorstore. cg|] }|j S)rn.0rcs r0 z(AwaDB.from_documents..js777c!777r5cg|] }|j Sr)r_rs r0rz(AwaDB.from_documents..ks777cS\777r5)r6rr8rrr)r) rrrrrrrr6r8s r0from_documentszAwaDB.from_documentsQsW287Y77777Y777 ~~!-    r5) rrrrrrrrrrrr )rr)NN) r6r7r8r9r:r;rrrr<)rrrrrrF) rJrrKrLrMrrNrOrrrrP) rJrrKrLrMrrNrOrrrr])r)rJrrKrLrrrr])rrhrKrLrMrrNrOrWrirrrrP)rrvrwNN)rJrrKrLrxrLryrzrMrrNrOrrrrP)rr~rKrLrxrLryrzrMrrNrOrrrrP)NNNNN)rrrMrrNrOr[rirrrrrrr3)rrrrrr;) rr<r6r7r8r9rrrr<)rrrr<)rrrr)rrr6r<rrr8r9rrrrrrrrrr)rrrrPrrrrrrrrrrrr)__name__ __module__ __qualname____doc__r__annotations__r1propertyr4rErI DEFAULT_TOPNr\rergrZr}r|rrrrrrr classmethodrrrr5r0rrs00000.*.*.)- /+/+/+/+/+bX+/-1 ! ! ! ! ! F2222..2&* , , , , , b.2&* 22222nEEEEE,0.2&*=A 55555t .2&*33333p .2&*<<<<<@$(.2&*15# 00000h$(6+/      0  1 1 1 1%%%%+/*.-*.)-[@+/-*.)- ! ! ! ! [! ! ! r5r) __future__rloggingr)typingrrrrrr r r r numpyrlangchain_core.documentsr langchain_core.embeddingsrlangchain_core.vectorstoresr&langchain_community.vectorstores.utilsrr% getLoggerloggerrrrr5r0rs#"""""" WWWWWWWWWWWWWWWWWWWWWW------000000333333MMMMMMLLL     ^  ^  ^  ^  ^  K^  ^  ^  ^  ^  r5