Ngm/ddlmZddlZddlZddlmZmZmZmZm Z ddl Z ddl m Z ddlmZddlmZejeZGddeZdS) ) annotationsN)AnyIterableListOptionalTypeDocument) Embeddings) VectorStoreceZdZUdZdZded< d1d2dZed3dZ d4d5d Z d6d!Z d7d8d'Z e d9d:d-Z e d9d;d0ZdS)<AtlasDBa`Atlas` vector store. Atlas is the `Nomic's` neural database and `rhizomatic` instrument. To use, you should have the ``nomic`` python package installed. Example: .. code-block:: python from langchain_community.vectorstores import AtlasDB from langchain_community.embeddings.openai import OpenAIEmbeddings embeddings = OpenAIEmbeddings() vectorstore = AtlasDB("my_project", embeddings.embed_query) atlas_idstr_ATLAS_DEFAULT_ID_FIELDNA description for your projectTFnameembedding_functionOptional[Embeddings]api_key Optional[str] description is_publicboolreset_project_if_existsreturnNonec: ddl}ddlm}n#t$rtdwxYw|td||||_d} |jd} |||| ||t j|_|j dS) a  Initialize the Atlas Client Args: name (str): The name of your project. If the project already exists, it will be loaded. embedding_function (Optional[Embeddings]): An optional function used for embedding your data. If None, data will be embedded with Nomic's embed model. api_key (str): Your nomic API key description (str): A description for your project. is_public (bool): Whether your project is publicly accessible. True by default. reset_project_if_exists (bool): Whether to reset this project if it already exists. Default False. Generally useful during development and testing. rN) AtlasProjectzRCould not import nomic python package. Please install it with `pip install nomic`.z/No API key provided. Sign up at atlas.nomic.ai!text embedding)rrmodalityrrunique_id_field) nomicr ImportError ValueErrorlogin_embedding_functionrrproject_latest_project_state) selfrrrrrrr$rr"s b/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/vectorstores/atlas.py__init__zAtlasDB.__init__"s4  LLL * * * * * * *   >   ?NOO O G#5   # /"H$|#$;#;      **,,,,,s 'c|jS)N)r()r+s r, embeddingszAtlasDB.embeddingsYs ''texts Iterable[str] metadatasOptional[List[dict]]idsOptional[List[str]]refreshkwargsr List[str]c ^|>t|dkr+d|dvrtdt dD|j|j}t j|}|fdtD}nMtt|D].} | || tj <| || d</|}|j gtj ||j 5|j ||dddn #1swxYwYn|fd tD}n=tD]+\} } || d<| || tj <,|}|j gtj ||j 5|j |dddn #1swxYwY|rgt|j jdkrJ|j 5|j dddn #1swxYwYS) aRun more texts through the embeddings and add to the vectorstore. Args: texts (Iterable[str]): Texts to add to the vectorstore. metadatas (Optional[List[dict]], optional): Optional list of metadatas. ids (Optional[List[str]]): An optional list of ids. refresh(bool): Whether or not to refresh indices with the updated data. Default True. Returns: List[str]: List of IDs of the added texts. Nrr z#Cannot accept key text in metadata!cNg|]"}ttj#S)ruuiduuid4).0_s r, z%AtlasDB.add_texts..zs&4443tz||$$444r0cNg|]!\}}tj|d|i"Sr rr)r?ir@r5r1s r,rAz%AtlasDB.add_texts..sA14c!ffeAhOr0)id_fielddata)r/rGcBg|]\}}d|tj|iSrCrD)r?rEr r5s r,rAz%AtlasDB.add_texts..s=4T7#BCFKr0)lenkeysr&listr(embed_documentsnpstack enumeraterangerrr)_validate_map_data_inputswait_for_project_lockadd_embeddingsadd_textindices rebuild_maps) r+r1r3r5r7r8 _embeddingsr/rGrEr s ` ` r, add_textszAtlasDB.add_texts]s*  !I"")A,++----BCC CU  ;44e444C  # /2BB5IIK+..J  )% 0 0 s9~~..44ADGFIaL!@A+08IaL((  L 2 2W<4 3   3355 N N ++z+MMM N N N N N N N N N N N N N N N #,U#3#3 )//KKGAt+0IaL(DGFIaL!@AA  L 2 2W<4 3   3355 , , %%d+++ , , , , , , , , , , , , , , ,  04<'((1,,\779900L--///000000000000000 s6E++E/2E/H99H=H=<J""J&)J&c |j5|jjdi|cdddS#1swxYwYdS)zCreates an index in your project. See https://docs.nomic.ai/atlas_api.html#nomic.project.AtlasProject.create_index for full detail. Nr<)r)rR create_index)r+r8s r,rZzAtlasDB.create_indexs\ / / 1 1 7 7,4<,66v66 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7s 9==querykintList[Document]c  |jtd|j|gd}tj|dd}|j5|jjd ||\}}|j |d dddn #1swxYwY fdt|D}|S) aRun similarity search with AtlasDB Args: query (str): Query text to search for. k (int): Number of results to return. Defaults to 4. Returns: List[Document]: List of documents most similar to the query text. NzBAtlasDB requires an embedding_function for text similarity search!r)queriesr])r5c\g|](\}}t|d|)S)r ) page_contentmetadatar )r?rEneighborrGs r,rAz-AtlasDB.similarity_search..sF   8 $q'&/DG D D D   r0) r(NotImplementedErrorrLrMarrayreshaper)rR projections vector_searchget_datarO) r+r\r]r8 _embeddingr! neighborsr@docsrGs @r,similarity_searchzAtlasDB.similarity_searchsM  # +%T -==ugFFqI HZ((00B77 \ / / 1 1 ; ;<3A6DD!QELIq<((Yq\(::D  ; ; ; ; ; ; ; ; ; ; ; ; ; ; ;     (33    s:A CCCcls Type[AtlasDB]r! index_kwargsOptional[dict]c  L||td|dzdd} | | D] \} }|| | < ||||d|| }|j5|||||jd i| dddn #1swxYwY|S) a^Create an AtlasDB vectorstore from a raw documents. Args: texts (List[str]): The list of texts to ingest. name (str): Name of the project to create. api_key (str): Your nomic API key, embedding (Optional[Embeddings]): Embedding function. Defaults to None. metadatas (Optional[List[dict]]): List of metadatas. Defaults to None. ids (Optional[List[str]]): Optional list of document IDs. If None, ids will be auto created description (str): A description for your project. is_public (bool): Whether your project is publicly accessible. True by default. reset_project_if_exists (bool): Whether to reset this project if it already exists. Default False. Generally useful during development and testing. index_kwargs (Optional[dict]): Dict of kwargs for index creation. See https://docs.nomic.ai/atlas_api.html Returns: AtlasDB: Nomic's neural database and finest rhizomatic instrument N$`name` and `api_key` cannot be None._indexr )r indexed_fieldr)rrrrr)r1r3r5r<)r&itemsr)rRrXrZ)rrr1r!r3r5rrrrrrtr8all_index_kwargsr]vatlasDBs r, from_textszAtlasDB.from_textss5J <7?CDD D%)8OfMM  #$**,, ( (1&' ### (8$;    _ 2 2 4 4 5 5   EYC  H H H G 4 4#3 4 4 4 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5s'&BB B documentspersist_directoryc ||tdd|D} d|D} |||| || |||| |  S)a+Create an AtlasDB vectorstore from a list of documents. Args: name (str): Name of the collection to create. api_key (str): Your nomic API key, documents (List[Document]): List of documents to add to the vectorstore. embedding (Optional[Embeddings]): Embedding function. Defaults to None. ids (Optional[List[str]]): Optional list of document IDs. If None, ids will be auto created description (str): A description for your project. is_public (bool): Whether your project is publicly accessible. True by default. reset_project_if_exists (bool): Whether to reset this project if it already exists. Default False. Generally useful during development and testing. index_kwargs (Optional[dict]): Dict of kwargs for index creation. See https://docs.nomic.ai/atlas_api.html Returns: AtlasDB: Nomic's neural database and finest rhizomatic instrument Nrwcg|] }|j Sr<)rer?docs r,rAz*AtlasDB.from_documents..9s777c!777r0cg|] }|j Sr<)rfrs r,rAz*AtlasDB.from_documents..:s777cS\777r0) rrr1r!r3r5rrrrt)r&r~)rrrr!r5rrrrrrrtr8r1r3s r,from_documentszAtlasDB.from_documentss|H <7?CDD D77Y77777Y777 ~~#$;%   r0)NNrTF)rrrrrrrrrrrrrr)rr)NNT) r1r2r3r4r5r6r7rr8rrr9)r8rrr)r[)r\rr]r^r8rrr_) NNNNNrTFN)rrrsr1r9r!rr3r4r5r6rrrrrrrrrrrtrur8rrr)rrrsrr_r!rr5r6rrrrrrrrrrrrrtrur8rrr)__name__ __module__ __qualname____doc__r__annotations__r-propertyr/rXrZrq classmethodr~rr<r0r,rrsa $.---- 48!%;(-5-5-5-5-5-n(((X( +/#' LLLLL\7777     D+/*.#'"!%;(-'+9999[9v+/#'"!%+/;(-'+2 2 2 2 [2 2 2 r0r) __future__rloggingr=typingrrrrrnumpyrMlangchain_core.documentsr langchain_core.embeddingsr langchain_core.vectorstoresr getLoggerrloggerrr<r0r,rs"""""" 66666666666666------000000333333  8 $ $w w w w w kw w w w w r0