Ng= ddlmZddlZddlZddlmZmZmZmZm Z m Z m Z m Z m Z ddlmZmZmZmZmZmZmZddlmZmZmZ ddlmZn#e$r ddlmZYnwxYwddlmZddl m!Z!dd l"m#Z#dd l$m%Z%d Z&d Z'eZ(Gd de%Z)dS)) annotationsN) AnyCallableDictIterableListOptionalSequenceTupleType)REALColumnStringTable create_engineinserttext)ARRAYJSONTEXT)declarative_base)Document) Embeddings)get_from_dict_or_env) VectorStoreilangchain_documentc6eZdZdZeedddfdCdZ dDdEdZedFdZ dGdZ dHdZ dHdZ dHdZ dIdJd&Z dKdLd,Z dKdMd.Z dKdNd1Z dKdOd2ZdDdPd4ZedeedddfdQd7ZedRd9ZeeedddfdSd;ZedTdBZdS)U AnalyticDBa `AnalyticDB` (distributed PostgreSQL) vector store. AnalyticDB is a distributed full postgresql syntax cloud-native database. - `connection_string` is a postgres connection string. - `embedding_function` any embedding function implementing `langchain.embeddings.base.Embeddings` interface. - `collection_name` is the name of the collection to use. (default: langchain) - NOTE: This is not the name of the table, but the name of the collection. The tables will be created when initializing the store (if not exists) So, make sure the user has the right permissions to create tables. - `pre_delete_collection` if True, will delete the collection if it exists. (default: False) - Useful for testing. FNconnection_stringstrembedding_functionrembedding_dimensionintcollection_namepre_delete_collectionboolloggerOptional[logging.Logger] engine_argsOptional[dict]returnNonec||_||_||_||_||_|pt jt|_| |dSN) rr!r"r$r%logging getLogger__name__r' __post_init__)selfrr!r"r$r%r'r)s g/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/vectorstores/analyticdb.py__init__zAnalyticDB.__init__+s]"3"4#6 .%:"; 1( ; ;  ;'''''cv|pi}d|vrd|d<t|jfi||_|dS)z' Initialize the store. pool_recycleiN)rrenginecreate_collection)r3r) _engine_argss r4r2zAnalyticDB.__post_init__=s^#(b  , . .  ($D$:KKlKK       r6c|jSr.)r!r3s r4 embeddingszAnalyticDB.embeddingsQs &&r6Callable[[float], float]c|jSr.)_euclidean_relevance_score_fnr=s r4_select_relevance_score_fnz%AnalyticDB._select_relevance_score_fnUs 11r6c t|jtjt dt dt jt dttt dtdt dtdd|j 5}|5tj||jd }t!d |d }||}|s8t!d |d |jd|jd}||dddn #1swxYwYddddS#1swxYwYdS)NidT) primary_keydefault embeddingdocumentnullablemetadataextend_existing_embedding_idxzi SELECT 1 FROM pg_indexes WHERE indexname = 'z'; z& CREATE INDEX z ON zY USING ann(embedding) WITH ( "dim" = z\, "hnsw_m" = 100 ); )rr$BaserKrruuiduuid4rr rrr9connectbegin create_allrexecutescalarr")r3conn index_name index_queryresultindex_statements r4create_table_if_not_existsz%AnalyticDB.create_table_if_not_existsXs$  M 44 D D D ;d , , :v 5 5 5 :td 3 3 3     [ " " 2d 2 2 ((...!% 4DDD ")3 k2299;; 2&*&0 0&*%=  ' 'OLL1117 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2s7)F>BE( F(E, ,F/E, 0FFFcd|jr||dSr.)r%delete_collectionr\r=s r4r:zAnalyticDB.create_collections7  % %  " " $ $ $ '')))))r6cP|jdtd|jd}|j5}|5||dddn #1swxYwYddddS#1swxYwYdS)NzTrying to delete collectionzDROP TABLE IF EXISTS ;)r'debugrr$r9rRrSrU)r3drop_statementrWs r4r^zAnalyticDB.delete_collections 7888Md6JMMMNN [ " " -d - - ^,,, - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -s6 B!B7 BB B B BB"Btexts Iterable[str] metadatasOptional[List[dict]]idsOptional[List[str]] batch_sizekwargsr List[str]c | d|D}|jt|}|s d|D}t|jt jtdtdtdtttdtd td td d }g}|j 5} | 5t!||||D]}\} } } } || | | | d t%||krI| t)|||~|r5| t)||dddn #1swxYwYdddn #1swxYwY|S) arRun more texts through the embeddings and add to the vectorstore. Args: texts: Iterable of strings to add to the vectorstore. metadatas: Optional list of metadatas associated with the texts. kwargs: vectorstore specific parameters Returns: List of ids from adding the texts into the vectorstore. NcNg|]"}ttj#S)r rPrQ.0_s r4 z(AnalyticDB.add_texts..s&4443tz||$$444r6cg|]}iSrororps r4rsz(AnalyticDB.add_texts..s++++++r6rDTrErGrHrIrKrL)rDrGrHrK)r!embed_documentslistrr$rOrKrrrr rrr9rRrSzipappendlenrUrvaluesclear)r3rdrfrhrjrkr> chunks_tablechunks_table_datarWrHrKchunk_idrGs r4 add_textszAnalyticDB.add_textss$ ;44e444C,<> ::1V;   r6rG List[float]c ddlm}n#t$rtdwxYwd}|6d|D}dd|}djd |d }||d }j5} | t|| } dddn #1swxYwYfd | D} | S) Nr)RowzRCould not import Row from sqlalchemy.engine. Please 'pip install sqlalchemy>=1.4'.c$g|] \}}d|d|S)z metadata->>z = ro)rqkeyvalues r4rszEAnalyticDB.similarity_search_with_score_by_vector.. s:6@c51c1111r6zWHERE z AND zW SELECT *, l2_distance(embedding, :embedding) as distance FROM z zL ORDER BY embedding <-> :embedding LIMIT :k )rGrcfg|]-}t|j|jj|jndf.S)) page_contentrKN)rrHrKr!distance)rqrZr3s r4rszEAnalyticDB.similarity_search_with_score_by_vector..!s\ ! ! !  !'#_$(#:#FD   ! ! ! r6) sqlalchemy.enginer ImportErroritemsjoinr$r9rRrUrfetchall) r3rGrrrfilter_condition conditions sql_queryparamsrWresultsdocuments_with_scoress ` r4rz1AnalyticDB.similarity_search_with_score_by_vectors  - - - - - - -   8     DJLLNNJ C Z(@(@BB  &      )q11[ " " Vd%)\\$y//6%J%J%S%S%U%UG V V V V V V V V V V V V V V V ! ! ! ! " ! ! ! %$s $ 6CCCc J||||}d|DS)akReturn docs most similar to embedding vector. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query vector. rcg|]\}}|Sroro)rqdocrrs r4rsz:AnalyticDB.similarity_search_by_vector..As222Q222r6)r)r3rGrrrkdocs_and_scoress r4rz&AnalyticDB.similarity_search_by_vector-s<"EE1VF  32/2222r6Optional[bool]c &|tdt|jtjt dt dt dttt dtdt d tdd } |j 5}| 5|jj|}||| dddddddS#1swxYwY ddddS#1swxYwYdS#t($r(}t+d t-|Yd}~d Sd}~wwxYw) zUDelete by vector IDs. Args: ids: List of ids to delete. NzNo ids provided to delete.rDTrurGrHrIrKrLzDelete operation failed:F) ValueErrorrr$rOrKrrrr rrr9rRrScrDin_rUdeletewhere Exceptionprintr )r3rhrkr}rWdelete_conditiones r4rzAnalyticDB.deleteCs ;9:: :  M 44 0 0 0 ;d , , :v 5 5 5 :td 3 3 3     $$&& $ZZ\\  '3~'8'<'.s333A333r6cg|] }|j Sro)rKrs r4rsz-AnalyticDB.from_documents..s333AQZ333r6r)rdr%rGr"rfrhr$r)ro)rr) rrrGr"r$rhr%r)rkrdrfrs r4from_documentszAnalyticDB.from_documentss&4333333333 55f==&7"#s~  "7 3+#      r6driverhostportdatabaseuserpasswordc *d|d|d|d|d|d| S)z2Return connection string from database parameters.z postgresql+z://:@/ro)rrrrrrrs r4 connection_string_from_db_paramsz+AnalyticDB.connection_string_from_db_paramss;SVRRRRxRR$RRRRRRRr6)rr r!rr"r#r$r r%r&r'r(r)r*r+r,r.)r)r*r+r,)r+r)r+r?)r+r,)NNrc) rdrerfrgrhrirjr#rkrr+rl)rN) rr rr#rr*rkrr+r)rr rr#rr*r+r)rGrrr#rr*r+r) rGrrr#rr*rkrr+r)rhrirkrr+r)rrrdrlrGrrfrgr"r#r$r rhrir%r&r)r*rkrr+r)rkrr+r )rrrrrGrr"r#r$r rhrir%r&r)r*rkrr+r)rr rr rr#rr rr rr r+r )r1 __module__ __qualname____doc__ _LANGCHAIN_DEFAULT_EMBEDDING_DIM"_LANGCHAIN_DEFAULT_COLLECTION_NAMEr5r2propertyr>rBr\r:r^rrrrrr classmethodrrrrror6r4rrsS($DA&++/&*(((((('+!!!!!('''X'2222'2'2'2'2R**** ----+/#' >>>>>F!%      6!% 2!% 0%0%0%0%0%j!% 33333,< +/#CA#'&+&*[B!!![!  $DA#'&+&*" " " " [" H S S S[ S S Sr6r)* __future__rr/rPtypingrrrrrr r r r sqlalchemyr rrrrrrsqlalchemy.dialects.postgresqlrrrsqlalchemy.ormrrsqlalchemy.ext.declarativelangchain_core.documentsrlangchain_core.embeddingsrlangchain_core.utilsrlangchain_core.vectorstoresrrrrOrror6r4rs"""""" WWWWWWWWWWWWWWWWWWWWWWOOOOOOOOOOOOOOOOOO<<<<<<<<<<<///////<<<;;;;;;;;<.-----000000555555333333#' %9"jSjSjSjSjSjSjSjSjSjSsA AA