Skip to content

Commit

Permalink
Merge pull request #69 from llm-tools/csv
Browse files Browse the repository at this point in the history
add CsvLoader
  • Loading branch information
adhityan authored May 31, 2024
2 parents 80567d7 + d8105ab commit 5dbde53
Show file tree
Hide file tree
Showing 6 changed files with 410 additions and 63 deletions.
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ The author(s) are looking to add core maintainers for this opensource project. R
- [Sitemap](#sitemap)
- [Text](#text)
- [Json](#json)
- [Csv](#csv)
- [Add a custom loader](#add-a-custom-loader)
- [More loaders coming soon](#more-loaders-coming-soon)
- [LLMs](#llms)
Expand Down Expand Up @@ -407,6 +408,16 @@ To add a parsed Javascript object to your embeddings, use `JsonLoader`. The libr

**Note:** if you want to restrict the keys that get added to the vectorDb in a dynamically obtained object, you can use the `pickKeysForEmbedding` optional parameter in the `JsonLoader` constructor.

## Csv

To add a Csv file (or URL) to your embeddings, use `CsvLoader`. The library will parse the Csv and add each row to its vector database.

```TS
.addLoader(new CsvLoader({ filePathOrUrl: '...' }))
```

**Note:** You can control how the `CsvLoader` parses the file in great detail by passing in the optional `csvParseOptions` constructor parameter.

## Add a custom loader

You can pass along a custom loader to the `addLoader` method by extending and implementing the abstract class `BaseLoader`. Here's how that would look like -
Expand Down
Loading

0 comments on commit 5dbde53

Please sign in to comment.