Add a vectordb module #2263
Merged
Add a vectordb module #2263
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sonichi
reviewed
Apr 10, 2024
sonichi
approved these changes
Apr 10, 2024
|
@jackgerrits this PR is blocked by your change request. Could you take a look? |
jackgerrits
approved these changes
Apr 10, 2024
whiskyboy
pushed a commit
to whiskyboy/autogen
that referenced
this pull request
Apr 17, 2024
* Added vectordb base and chromadb * Remove timer and unused functions * Added filter by distance * Added test utils * Fix format * Fix type hint of dict * Rename test * Add test chromadb * Fix test no chromadb * Add coverage * Don't skip test vectordb utils * Add types * Fix tests * Fix docs build error * Add types to base * Update base * Update utils * Update chromadb * Add get_docs_by_ids * Improve docstring * Add get all docs * Move chroma_results_to_query_results to utils * Improve type hints * Update logger * Update init, add embedding func * Improve docstring of vectordb, add two attributes * Improve test workflow
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
We've been asked for supporting different vector databases in RAG and teachability for many times, we can also foresee that different dbs could be useful to memory, history management, etc. in the future. In order to better support using different vectordbs in different scenarios, and also make it easier for accepting contributions from the community; a vectordb protocol is proposed in this PR.
As we've discussed, some basic APIs are defined in the protocol, any special APIs related to the vectordb itself can also be added in the implementation. @siluhuang, @HongleiZhuang, @Knucklessg1
This PR is also a step of the RAG refactor. #1657
Related issue number
Checks