NgddlZddlmZddlmZddlmZmZmZm Z m Z m Z m Z ddl mZddlmZddlmZddlmZmZGd d eZGd d eZdS) N) TextIOWrapper)Path)AnyDictIteratorListOptionalSequenceUnion)Document) BaseLoader)detect_file_encodings)UnstructuredFileLoadervalidate_unstructured_versionceZdZdZ ddddeeefdeedeed ee d eed e d eefd Z de e fdZdede e fdZdS) CSVLoaderaLoad a `CSV` file into a list of Documents. Each document represents one row of the CSV file. Every row is converted into a key/value pair and outputted to a new line in the document's page_content. The source for each document loaded from csv is set to the value of the `file_path` argument for all documents by default. You can override this by setting the `source_column` argument to the name of a column in the CSV file. The source of each document will then be set to the value of the column with the name specified in `source_column`. Output Example: .. code-block:: txt column1: value1 column2: value2 column3: value3 Instantiate: .. code-block:: python from langchain_community.document_loaders import CSVLoader loader = CSVLoader(file_path='./hw_200.csv', csv_args={ 'delimiter': ',', 'quotechar': '"', 'fieldnames': ['Index', 'Height', 'Weight'] }) Load: .. code-block:: python docs = loader.load() print(docs[0].page_content[:100]) print(docs[0].metadata) .. code-block:: python Index: Index Height: Height(Inches)" Weight: "Weight(Pounds)" {'source': './hw_200.csv', 'row': 0} Async load: .. code-block:: python docs = await loader.aload() print(docs[0].page_content[:100]) print(docs[0].metadata) .. code-block:: python Index: Index Height: Height(Inches)" Weight: "Weight(Pounds)" {'source': './hw_200.csv', 'row': 0} Lazy load: .. code-block:: python docs = [] docs_lazy = loader.lazy_load() # async variant: # docs_lazy = await loader.alazy_load() for doc in docs_lazy: docs.append(doc) print(docs[0].page_content[:100]) print(docs[0].metadata) .. code-block:: python Index: Index Height: Height(Inches)" Weight: "Weight(Pounds)" {'source': './hw_200.csv', 'row': 0} NF)content_columns file_path source_columnmetadata_columnscsv_argsencodingautodetect_encodingrcl||_||_||_||_|pi|_||_||_dS)a Args: file_path: The path to the CSV file. source_column: The name of the column in the CSV file to use as the source. Optional. Defaults to None. metadata_columns: A sequence of column names to use as metadata. Optional. csv_args: A dictionary of arguments to pass to the csv.DictReader. Optional. Defaults to None. encoding: The encoding of the CSV file. Optional. Defaults to None. autodetect_encoding: Whether to try to autodetect the file encoding. content_columns: A sequence of column names to use for the document content. If not present, use all columns that are not part of the metadata. N)rrrrrrr)selfrrrrrrrs k/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/document_loaders/csv_loader.py__init__zCSVLoader.__init__csC2#* 0   B #6 .returnc#hK t|jd|j5}||Ed{VddddS#1swxYwYdS#t$r}|jr|t |j}|D]d} t|jd|j5}||Ed{V dddn;#1swxYwYU#t$rYawxYwntd|j|Yd}~dSYd}~dSd}~wt$r}td|j|d}~wwxYw)N)newlinerzError loading ) openrr_CSVLoader__read_fileUnicodeDecodeErrorrr RuntimeError Exception)rcsvfileedetected_encodingsrs r lazy_loadzCSVLoader.lazy_loads\ Idnb4=III 5W++G444444444 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5! M M M' M%:4>%J%J" 2!!H!! NBAR"$'+'7'7'@'@@@@@@@@! """"""""""""""""" .!!! !!##DDN#D#DEE1L!!!!!!"""""" I I I@@@AAq H IsAA  A A  AA A D1 DCC9 CDC C C CD CDCD D1D,,D1r)c#Ktj|fij}t|D]\}} j |jnt j}n&#t$rtdjdwxYwd fd| D}||d}j D]/} ||||<#t$rtd|dwxYwt||VdS)NzSource column 'z' not found in CSV file. c 3bK|]\}}jr |jvrn |jv||n|dt|tr|nCt|t r-dttj|n|VdS)Nz: ,)rrstrip isinstancestrlistjoinmap).0kvrs r z(CSVLoader.__read_file..s  Aq+  ----$"777#$-QWWYYYQ22a%%;.!'')))a&&,.388C 14E4E+F+F+F,-228777  r)sourcerowzMetadata column ') page_contentmetadata) csv DictReaderr enumeraterr3rKeyError ValueErrorr5itemsrr ) rr) csv_readerir<r;contentr>cols ` r __read_filezCSVLoader.__read_files^G==t}==  ++ D DFAs )5*++T^,,     Rd&8RRR ii     IIKK     G#)33H, X XX$'HHSMMXXX$%V%V%V%VWWWX(CCC C C C C9 D Ds(A#A< C  C+)NrNNF)__name__ __module__ __qualname____doc__r r3rr r rboolrrr r,rr%rrrrrsPPj(,*,#'"&$)/*,///d#/ }/#3- / 4. / 3- /"/"#////BI8H-IIII*D=DXh5GDDDDDDrrc@eZdZdZ d dededeffd ZdefdZxZ S) UnstructuredCSVLoadera|Load `CSV` files using `Unstructured`. Like other Unstructured loaders, UnstructuredCSVLoader can be used in both "single" and "elements" mode. If you use the loader in "elements" mode, the CSV file will be a single Unstructured Table element. If you use the loader in "elements" mode, an HTML representation of the table will be available in the "text_as_html" key in the document metadata. Examples -------- from langchain_community.document_loaders.csv_loader import UnstructuredCSVLoader loader = UnstructuredCSVLoader("stanley-cups.csv", mode="elements") docs = loader.load() singlermodeunstructured_kwargsc `tdtjd||d|dS)a Args: file_path: The path to the CSV file. mode: The mode to use when loading the CSV file. Optional. Defaults to "single". **unstructured_kwargs: Keyword arguments to pass to unstructured. z0.6.8)min_unstructured_version)rrRNr)rsuperr)rrrRrS __class__s rrzUnstructuredCSVLoader.__init__sA &wGGGGO94OO;NOOOOOrr c6ddlm}|dd|ji|jS)Nr) partition_csvfilenamer)unstructured.partition.csvrYrrS)rrYs r _get_elementsz#UnstructuredCSVLoader._get_elementss4<<<<<<}QQdnQ8PQQQr)rQ) rJrKrLrMr3rrrr\ __classcell__)rWs@rrPrPs&+3 P P P$' PKN P P P P P PRtRRRRRRRRrrP)r?iorpathlibrtypingrrrrr r r langchain_core.documentsr )langchain_community.document_loaders.baser ,langchain_community.document_loaders.helpersr1langchain_community.document_loaders.unstructuredrrrrPrrrres@ GGGGGGGGGGGGGGGGGG------@@@@@@NNNNNN gDgDgDgDgD gDgDgDT$R$R$R$R$R2$R$R$R$R$Rr