Ng]ddlZddlmZmZmZmZmZmZddlZ ddl m Z ddl m Z ddlmZddlmZdZGdd eZdS) N)AnyDictIterableListOptionalTuple)Document) Embeddings) VectorStore)maximal_marginal_relevanceceZdZdZdededdfdZd)dZede efdZ d*d e e d e e edede e fd Z d*d e e d e e edede e fd Z d*d e e e dede efdZ d*d e e e dede efdZefddde edede ee e fdede eeeeff dZefddde dede ee e fdede eeeff dZefddde dede ee e fdede eeeff dZefddde dede ee e fdede eeeff dZefddde dede ee e fdede eeeff dZefddde edede ee e fdede ef dZefddde edede ee e fdede ef dZ efddde dede ee e fdede ef dZ!efddde dede ee e fdede ef dZ"eddfddde eded ed!ede ee e fdede efd"Z#eddfddde eded ed!ede ee e fdede efd#Z$ d+ddde ded ed!ede ee e fdede efd%Z%eddfddde ded ed!ede ee e fdede efd&Z&e' d*d e e ded e e ededdf d'Z(e' d*d e e ded e e ededdf d(Z)dS),SurrealDBStorea/ SurrealDB as Vector Store. To use, you should have the ``surrealdb`` python package installed. Args: embedding_function: Embedding function to use. dburl: SurrealDB connection url ns: surrealdb namespace for the vector store. (default: "langchain") db: surrealdb database for the vector store. (default: "database") collection: surrealdb collection for the vector store. (default: "documents") (optional) db_user and db_pass: surrealdb credentials Example: .. code-block:: python from langchain_community.vectorstores.surrealdb import SurrealDBStore from langchain_community.embeddings import HuggingFaceEmbeddings model_name = "sentence-transformers/all-mpnet-base-v2" embedding_function = HuggingFaceEmbeddings(model_name=model_name) dburl = "ws://localhost:8000/rpc" ns = "langchain" db = "docstore" collection = "documents" db_user = "root" db_pass = "root" sdb = SurrealDBStore.from_texts( texts=texts, embedding=embedding_function, dburl, ns, db, collection, db_user=db_user, db_pass=db_pass) embedding_functionkwargsreturnNc  ddlm}n"#t$r}td|d}~wwxYw|dd|_|jdddkr||j|_nt d|d d |_|d d |_|d d|_ ||_ ||_ dS)Nr)SurrealzZCannot import from surrealdb. please install with `pip install surrealdb`.dburlzws://localhost:8000/rpcwsz6Only websocket connections are supported at this time.ns langchaindbdatabase collection documents) surrealdbr ImportErrorpoprsdb ValueErrorrrrrr)selfrrres f/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/vectorstores/surrealdb.py__init__zSurrealDBStore.__init__5s   ) ) ) ) ) ) )   @   ZZ)BCC :ac?d " "wtz**DHHUVV V**T;//**T:.. **\;??"4 s (#(cpK|jd{Vd|jvr`d|jvrW|jd}|jd}|j||dd{V|j|j|jd{VdS)zr Initialize connection to surrealdb database and authenticate if credentials are provided Ndb_userdb_pass)userpass)r!connectrgetsigninuserr)r#r*passwords r% initializezSurrealDBStore.initializeOs h          # # T[(@(@;??9--D{y11H(//4"B"BCC C C C C C C Chll47DG,,,,,,,,,,,cHt|jtr|jndSN) isinstancerr )r#s r% embeddingszSurrealDBStore.embeddings[s)$1:>> D # # r2texts metadatascvK|jt|}g}t|D]}\}}|||d}||t |kr |||d<ng|d<|j|j|d{V} || dd~|S)zAdd list of text along with embeddings to the vector store asynchronously Args: texts (Iterable[str]): collection of text to add to the database Returns: List of ids for the newly inserted documents )text embeddingNmetadatarid) rembed_documentslist enumeratelenr!createrappend) r#r7r8rr6idsidxr:datarecords r% aadd_textszSurrealDBStore.aadd_textscs,<Z  #%Z 8??F JJvay ' ' ' ' r2c ddttdtttdt dttffd }t j|||fi|S)zAdd list of text along with embeddings to the vector store Args: texts (Iterable[str]): collection of text to add to the database Returns: List of ids for the newly inserted documents Nr7r8rrcfKd{Vj||fi|d{VSr4r1rH)r7r8rr#s r% _add_textsz,SurrealDBStore.add_texts.._add_textssY //## # # # # # # #( DDVDDDDDDDD Dr2r4)rstrrrdictrasynciorun)r#r7r8rrLs` r% add_textszSurrealDBStore.add_textss"/3 E EC= ET + E E#Y  E E E E E E{::eYAA&AABBBr2rDcHK|'jjd{VdSt|tr"j|d{VdSt|t r)t |dkrfd|Dd{V}dSdS)aDelete by document ID asynchronously. Args: ids: List of ids to delete. **kwargs: Other keyword arguments that subclasses might use. Returns: Optional[bool]: True if deletion is successful, False otherwise. NTrcTKg|]"}j|d{V#Sr4)r!delete).0r=r#s r% z*SurrealDBStore.adelete..s=AAArtxr22222222AAAr2F)r!rTrr5rMr?rA)r#rDr_s` r%adeletezSurrealDBStore.adeletes ;(//$/22 2 2 2 2 2 2 24#s## hooc*********tc4(( SXX\\AAAASAAAAAAAAAA4ur2c dtttdtdttffd }t j||fi|S)a Delete by document ID. Args: ids: List of ids to delete. **kwargs: Other keyword arguments that subclasses might use. Returns: Optional[bool]: True if deletion is successful, False otherwise. rDrrcfKd{Vjdd|i|d{VS)NrD)r1rX)rDrr#s r%_deletez&SurrealDBStore.delete.._deletesX//## # # # # # # #%88#888888888 8r2)rrrMrboolrOrP)r#rDrr\s` r%rTzSurrealDBStore.deletesg  9xS 2 9c 9htn 9 9 9 9 9 9{77311&11222r2)filterr;kr^cK|j|||ddd}d}|rJ|D]G}t||ttfvr d||d}n ||}|d|d|dz }Hd |d d |d } |j| |d {V} t| dkrgS| d} | ddkr'ddlm } | dd} | | d| dDS)aRun similarity search for query embedding asynchronously and return documents and scores Args: embedding (List[float]): Query embedding. k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with scores score_thresholdr)rr;r_ra'z and metadata.z =  u select id, text, metadata, embedding, vector::similarity::cosine(embedding, $embedding) as similarity from ⟨rub⟩ where vector::similarity::cosine(embedding, $embedding) >= $score_threshold z4 order by similarity desc LIMIT $k; NstatusOK)SurrealExceptionresultz Unknown Errorc g|]G}t|dd|di|dpi|d|dfHS)r:r=r<) page_contentr< similarityr;)r r-)rUdocs r%rVzKSurrealDBStore._asimilarity_search_by_vector_with_score.. sy    !$V"CIM#''*2E2E2KML!K      r2) rr-typerMr]r!queryrA surrealdb.wsrg)r#r;r_r^rargs custom_filterkey filter_valuernresultsrhrgerrs r%(_asimilarity_search_by_vector_with_scorez7SurrealDBStore._asimilarity_search_by_vector_with_scores(/"%zz*;Q??     I I Is $$d 33#5vc{#5#5#5LL&,Sk#3L!H!H!H!H!H!HH   l#        ud33333333 w<<1  I ( t # # 5 5 5 5 5 5**X77C""3'' '   h'    r2rnc|K|j|}d|j||fd|i|d{VDS)afRun similarity search asynchronously and return relevance scores Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with relevance scores cg|] \}}}||f Sr[r[rUdocumentrkrWs r%rVzKSurrealDBStore.asimilarity_search_with_relevance_scores..*3   '*az "   r2r^Nr embed_queryrvr#rnr_r^rquery_embeddings r%(asimilarity_search_with_relevance_scoresz7SurrealDBStore.asimilarity_search_with_relevance_scores$1==eDD  DdC#Q/59?    r2c dttttfffd }t j|S)aeRun similarity search synchronously and return relevance scores Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with relevance scores rcjKd{Vjfdid{VSNr^)r1rr^r_rrnr#sr%(_similarity_search_with_relevance_scoreszhSurrealDBStore.similarity_search_with_relevance_scores.._similarity_search_with_relevance_scoresFsz//## # # # # # # #FFq!'+1 r2rrr floatrOrP)r#rnr_r^rrs````` r%'similarity_search_with_relevance_scoresz6SurrealDBStore.similarity_search_with_relevance_scores3sk&  x' (          {CCEEFFFr2c|K|j|}d|j||fd|i|d{VDS)anRun similarity search asynchronously and return distance scores Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with relevance distance scores cg|] \}}}||f Sr[r[rys r%rVz@SurrealDBStore.asimilarity_search_with_score..cr{r2r^Nr|r~s r%asimilarity_search_with_scorez,SurrealDBStore.asimilarity_search_with_scorePrr2c dttttfffd }t j|S)amRun similarity search synchronously and return distance scores Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with relevance distance scores rcjKd{Vjfdid{VSr)r1rrsr%_similarity_search_with_scorezRSurrealDBStore.similarity_search_with_score.._similarity_search_with_scoresx//## # # # # # # #;;q!'+1 r2r)r#rnr_r^rrs````` r%similarity_search_with_scorez+SurrealDBStore.similarity_search_with_scorelsj& T%%:P5Q           {88::;;;r2cHKd|j||fd|i|d{VDS)adRun similarity search on query embedding asynchronously Args: embedding (List[float]): Query embedding k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query cg|]\}}}| Sr[r[)rUrzrWs r%rVz?SurrealDBStore.asimilarity_search_by_vector..s.   !Q    r2r^N)rv)r#r;r_r^rs r%asimilarity_search_by_vectorz+SurrealDBStore.asimilarity_search_by_vectorsn$  (U(U1))%+)/5))######    r2c |dttffd }tj|S)aURun similarity search on query embedding Args: embedding (List[float]): Query embedding k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query rcjKd{Vjfdid{VSr)r1r)r;r^r_rr#sr%_similarity_search_by_vectorzPSurrealDBStore.similarity_search_by_vector.._similarity_search_by_vectorsx//## # # # # # # #::1%+/5 r2rr rOrP)r#r;r_r^rrs````` r%similarity_search_by_vectorz*SurrealDBStore.similarity_search_by_vectors`& DN           {7799:::r2chK|j|}|j||fd|i|d{VS)aDRun similarity search on query asynchronously Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query r^N)rr}rr~s r%asimilarity_searchz!SurrealDBStore.asimilarity_searchsm$1==eDD6T6 Q  '- 17         r2c |dttffd }tj|S)a5Run similarity search on query Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query rcjKd{Vjfdid{VSr)r1rrsr%_similarity_searchz._similarity_searchs]//## # # # # # # #00SS&SFSSSSSSSS Sr2r)r#rnr_r^rrs````` r%similarity_searchz SurrealDBStore.similarity_searchsk& T$x. T T T T T T T T T T{--//000r2?fetch_k lambda_multc K|j||fd|i|d{V}d|D d|D}ttj|tj|||} fd| DS)aHReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. r^Ncg|] }|d S)rr[rUsubs r%rVzKSurrealDBStore.amax_marginal_relevance_search_by_vector.. s)))3A)))r2cg|] }|d S)r[rs r%rVzKSurrealDBStore.amax_marginal_relevance_search_by_vector..s000#c"g000r2)dtype)r_rc g|] }| Sr[r[)rUidocss r%rVzKSurrealDBStore.amax_marginal_relevance_search_by_vector..s...AQ...r2)rvr nparrayfloat32) r#r;r_rrr^rrhr6 mmr_selectedrs @r%(amax_marginal_relevance_search_by_vectorz7SurrealDBStore.amax_marginal_relevance_search_by_vectors8EtD w  '- 17         *)&)))00000 1 HYbj 1 1 1 #    /.......r2c dttffd }tj|S)aIReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. rcnKd{Vjfdid{VSr)r1r)r;rr^r_rrr#sr%(_max_marginal_relevance_search_by_vectorzhSurrealDBStore.max_marginal_relevance_search_by_vector.._max_marginal_relevance_search_by_vector6s|//## # # # # # # #FF1g{;AEK r2r)r#r;r_rrr^rrs``````` r%'max_marginal_relevance_search_by_vectorz6SurrealDBStore.max_marginal_relevance_search_by_vectorsl: X             {CCEEFFFr2r cpK|j|}|j||||fd|i|d{V}|S)a@Return docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. r^N)rr}r) r#rnr_rrr^rr;rs r%amax_marginal_relevance_searchz-SurrealDBStore.amax_marginal_relevance_search>su:+77>> BTB q';  7= AG         r2c dttffd }tj|S)a?Return docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. rcnKd{Vjfdid{VSr)r1r)rr^r_rrrnr#sr%_max_marginal_relevance_searchzTSurrealDBStore.max_marginal_relevance_search.._max_marginal_relevance_search}s|//## # # # # # # #<<q';7=AG r2r)r#rnr_rrr^rrs``````` r%max_marginal_relevance_searchz,SurrealDBStore.max_marginal_relevance_searchasl8 d8n             {99;;<<$(33 d3i 33 $ 33332I ,0 I I I ;I  I c3h( I  I  eHeS() *I I I I \ ,0       c3h(     eHeO$ %    >G ,0 GGGG G c3h( G  G eHeO$ %GGGG@ ,0       c3h(     eHeO$ %    >< ,0 <<<< < c3h( <  < eHeO$ %<<<<< ,0    ;   c3h(     h    8; ,0 ;;;;; ; c3h( ;  ; h;;;;< ,0       c3h(     h    41 ,0 1111 1 c3h( 1  1 h11118 ,/,0,/,/,/;,/ ,/ ,/  ,/c3h(,/,/ h,/,/,/,/b #G,0#G#G#G;#G #G #G  #Gc3h(#G#G h#G#G#G#GP !,0!!!! ! !  !c3h(!! h!!!!L "=,0"="="="= "= "=  "=c3h("="= h"="="="=H +/ CyDJ'     [> +/ CyDJ'     [r2r)rOtypingrrrrrrnumpyrlangchain_core.documentsr langchain_core.embeddingsr langchain_core.vectorstoresr &langchain_community.vectorstores.utilsr rrr[r2r%rs================------000000333333MMMMMM q q q q q [q q q q q r2