Ng;ddlmZddlZddlmZddlmZddlmZm Z m Z m Z m Z ddl ZddlmZddlmZddlmZdd lmZdd lmZejeZGd d eZdS) ) annotationsN)deepcopy)Enum)AnyIterableListOptionalTuple)Document) Embeddings)run_in_executor) VectorStore)maximal_marginal_relevancec>eZdZdZ dBdCd ZedDdZ dEdFdZe dGdHdZ Gdd e Z d!e j dfdId(Z d!e j dfdJd*Zd!e j dfdKd,Zd!e j dfdLd-Z dMdd0dNd4Z dOdPd9ZdQd<ZdRd>ZdSdTd@Z dSdTdAZdS)URockseta`Rockset` vector store. To use, you should have the `rockset` python package installed. Note that to use this, the collection being used must already exist in your Rockset instance. You must also ensure you use a Rockset ingest transformation to apply `VECTOR_ENFORCE` on the column being used to store `embedding_key` in the collection. See: https://rockset.com/blog/introducing-vector-search-on-rockset/ for more details Everything below assumes `commons` Rockset workspace. Example: .. code-block:: python from langchain_community.vectorstores import Rockset from langchain_community.embeddings.openai import OpenAIEmbeddings import rockset # Make sure you use the right host (region) for your Rockset instance # and APIKEY has both read-write access to your collection. rs = rockset.RocksetClient(host=rockset.Regions.use1a1, api_key="***") collection_name = "langchain_demo" embeddings = OpenAIEmbeddings() vectorstore = Rockset(rs, collection_name, embeddings, "description", "description_embedding") commonsclientr embeddingsr collection_namestrtext_key embedding_key workspacecZ ddlm}n#t$rtdwxYwt||st dt |||_||_||_||_ ||_ ||_ |j ddS#t$rYdSwxYw)aNInitialize with Rockset client. Args: client: Rockset client object collection: Rockset collection to insert docs / query embeddings: Langchain Embeddings object to use to generate embedding for given text. text_key: column in Rockset collection to use to store the text embedding_key: column in Rockset collection to use to store the embedding. Note: We must apply `VECTOR_ENFORCE()` on this column via Rockset ingest transformation. r) RocksetClient]Could not import rockset client python package. Please install it with `pip install rockset`.z;client should be an instance of rockset.RocksetClient, got langchainN)rocksetr ImportError isinstance ValueErrortype_client_collection_name _embeddings _text_key_embedding_key _workspaceset_applicationAttributeError)selfrrrrrrrs f/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/vectorstores/rocksetdb.py__init__zRockset.__init__1s*  - - - - - - -   @   &-00 &F||&&   /%!+#  L ( ( 5 5 5 5 5    DD s #B B*)B*returnc|jSN)r%r+s r,rzRockset.embeddingsas N texts Iterable[str] metadatasOptional[List[dict]]idsOptional[List[str]] batch_sizeintkwargs List[str]c g}g}t|D]\}} t||kr|||z }g}i} |r(t||krt||} |rt||kr ||| d<| | |j<|j| | |j<|| t|dkr|||z }g}|S)aRun more texts through the embeddings and add to the vectorstore Args: texts: Iterable of strings to add to the vectorstore. metadatas: Optional list of metadatas associated with the texts. ids: Optional list of ids to associate with the texts. batch_size: Send documents in batches to rockset. Returns: List of ids from adding the texts into the vectorstore. _idr) enumeratelen_write_documents_to_rocksetrr&r% embed_queryr'append) r+r4r6r8r:r<batch stored_idsitextdocs r, add_textszRockset.add_textses(  ''  GAt5zzZ''d>>uEEE C -S^^a//y|,, $s3xx!|| VE "&C '+'7'C'CD'I'IC# $ LL     u::>> $::5AA AJEr2 embeddingc  | Jd|s Jd|s Jd|s Jd||||||} | |||| | S)znCreate Rockset wrapper with existing texts. This is intended as a quicker way to get started. NzRockset Client cannot be NonezCollection name cannot be emptyzText key name cannot be emptyzEmbedding key cannot be empty)rJ) clsr4rLr6rrrrr8r:r<rs r, from_textszRockset.from_textss&!!#B!!!AA AAAA888888======#fi(MRR%C<<<r2c"eZdZdZdZdZddZdS) Rockset.DistanceFunction COSINE_SIMEUCLIDEAN_DIST DOT_PRODUCTr.rc |jdkrdSdS)NrSASCDESC)valuer1s r,order_byz!Rockset.DistanceFunction.order_bysz---u6r2N)r.r)__name__ __module__ __qualname__rRrSrTrYr2r,DistanceFunctionrQs7! )#       r2r^queryk distance_func where_str Optional[str]List[Tuple[Document, float]]c T|j|j||||fi|S)aPerform a similarity search with Rockset Args: query (str): Text to look up documents similar to. distance_func (DistanceFunction): how to compute distance between two vectors in Rockset. k (int, optional): Top K neighbors to retrieve. Defaults to 4. where_str (Optional[str], optional): Metadata filters supplied as a SQL `where` condition string. Defaults to None. eg. "price<=70.0 AND brand='Nintendo'" NOTE: Please do not let end-user to fill this and always be aware of SQL injection. Returns: List[Tuple[Document, float]]: List of documents with their relevance score )1similarity_search_by_vector_with_relevance_scoresr%rCr+r`rarbrcr<s r,'similarity_search_with_relevance_scoresz/Rockset.similarity_search_with_relevance_scoressH2FtE   ( ( / /         r2List[Document]c T|j|j||||fi|S)zaSame as `similarity_search_with_relevance_scores` but doesn't return the scores. )similarity_search_by_vectorr%rCrhs r,similarity_searchzRockset.similarity_searchsG0t/   ( ( / /         r2 List[float]c <|j||||fi|}d|DS)zZAccepts a query_embedding (vector), and returns documents with similar embeddings.cg|]\}}|Sr]r]).0rI_s r, z7Rockset.similarity_search_by_vector..s222Q222r2)rg)r+rLrarbrcr<docs_and_scoress r,rlz#Rockset.similarity_search_by_vectorsEQ$P q-  6<  32/2222r2c jd}d|vr|d}||||||} |jjd|i}n4#t$r'} t d| gcYd} ~ Sd} ~ wwxYwg} |jD](} i} t| ts*Jd t| | D]\}} ||j krHt| ts0Jd |j t| | }X|d krBt| ts*Jd t| | }|d vr| | |<| t#|| |f*| S) z|Accepts a query_embedding (vector), and returns documents with similar embeddings along with their relevance scores.Texclude_embeddingsr`)sqlz$Exception when querying Rockset: %s Nz;document should be of type `dict[str,Any]`. But found: `{}`zIpage content stored in column `{}` must be of type `str`. But found: `{}`distzDComputed distance between vectors must of type `float`. But found {})r? _event_time_meta) page_contentmetadata)_build_query_sqlr#Queriesr` Exceptionloggererrorresultsr dictformatr"itemsr&rfloatrDr )r+rLrarbrcr<rvq_strquery_responsee finalResultdocumentr|vr{scores r,rgz9Rockset.similarity_search_by_vector_with_relevance_scoress(" 6 ) )!'(.Ss#PPPCcl4#67PPPr2)rrac g|] }| Sr]r])rqrG initial_docss r,rsz9Rockset.max_marginal_relevance_search..as:::A Q:::r2)r%rCrlrnparrayr|r') r+r`rarrrcr<query_embeddingrselected_indicesrGrs ` @r,max_marginal_relevance_searchz%Rockset.max_marginal_relevance_search-s:*66u==7t7  $      QPPP<PPP 5 H_ % % #    " > >AQ()<==::::)9::::r2Trrvboolc0dtt|}|jd|jd|d}|rd|dnd}|r d|jd nd}d |d |d |jd |jd|d|dt|dS)zABuilds Rockset SQL query to query similar vectors to query_vector,(z, [z ]) as distzWHERE  rKz EXCEPT(z),zSELECT * z FROM .zORDER BY dist z LIMIT )joinmaprrXr'r(r$rY) r+rrbrarcrvq_embedding_str distance_strselect_embeddings r,r}zRockset._build_query_sqles((3sO#<#<==)/  $2E     .7?*Y****R 2D M .t* . . . .#  ( o- %%''   1vv  r2rE List[dict]c||jj|j||j}d|jDS)N collectiondatarcg|] }|j Sr])r?)rq doc_statuss r,rsz7Rockset._write_documents_to_rockset..sBBB: BBBr2)r# Documents add_documentsr$r(r)r+rE add_doc_ress r,rBz#Rockset._write_documents_to_rockset~sGl,::,5DO;  CB1ABBBBr2Nonec ddlmn#t$rtdwxYw|jj|jfd|D|jdS)z1Delete a list of docs from the Rockset collectionr)DeleteDocumentsRequestDatarc(g|]}|S))idr])rqrGrs r,rsz(Rockset.delete_texts..s(@@@q,,222@@@r2rN)rockset.modelsrrr#rdelete_documentsr$r()r+r8rs @r, delete_textszRockset.delete_textss  A A A A A A A   @   //,@@@@C@@@o 0     s $Optional[bool]c  |g}||n3#t$r&}td|Yd}~dSd}~wwxYwdS)Nz.Exception when deleting docs from Rockset: %s FT)rrrr)r+r8r<rs r,deletezRockset.deletesp {   c " " " "    LLJA N N N55555 ts A AA c:Ktd|j|fi|d{VSr0)r r)r+r8r<s r,adeletezRockset.adeletes6%T4;FFvFFFFFFFFFr2)r) rrrr rrrrrrrr)r.r )NNr3) r4r5r6r7r8r9r:r;r<rr.r=)NNrKrKrKNr3)r4r=rLr r6r7rrrrrrrrr8r9r:r;r<rr.r) r`rrar;rbr^rcrdr<rr.re) r`rrar;rbr^rcrdr<rr.rj) rLrnrar;rbr^rcrdr<rr.rj) rLrnrar;rbr^rcrdr<rr.re)r_rr)r`rrar;rr;rrrcrdr<rr.rj)r_NT) rrnrbr^rar;rcrdrvrr.r)rErr.r=)r8r=r.rr0)r8r9r<rr.r)rZr[r\__doc__r-propertyrrJ classmethodrOrr^rRrirmrlrgrr}rBrrrr]r2r,rrsIH#.....`   X +/#' &&&&&P +/!#'[8     4   *:*E#'      H*:*E#'      ,*:*E#' 33333&*:*E#' 22222n 4;$(4;4;4;4;4;4;x#'#' 2CCCC         *.GGGGGGGr2r) __future__rloggingcopyrenumrtypingrrrr r numpyrlangchain_core.documentsr langchain_core.embeddingsr langchain_core.runnablesr langchain_core.vectorstoresr&langchain_community.vectorstores.utilsr getLoggerrZrrr]r2r,rs.""""""77777777777777------000000444444333333MMMMMM  8 $ $OGOGOGOGOGkOGOGOGOGOGr2