[Question]: How to store two index in redis?

[Ninlawat-Puhu]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

I have two index that keep different data as below code

index_policy = VectorStoreIndex(nodes = base_nodes+objects)
index_leave = VectorStoreIndex.from_documents(list_of_documents_leave)
query_engine_leave = index_leave.as_query_engine(similarity_top_k=2,filters = filters,streaming=True)
query_engine_policy = index_policy.as_query_engine(similarity_top_k=6,streaming=True)

But I want to save those index (index_policy,index_leave) in redis database to persist data and I want to call index from redis database instead of local paths. Moreover, with index_leave, I also used ExactMatchFilter for filter information in some columns.

Can you recommend how to develop the code from my requirement?

[dosubot]

To store and retrieve your index_policy and index_leave indexes in Redis, including using an ExactMatchFilter for index_leave, you'll need to integrate Redis functionality into your VectorStoreIndex instances. This involves using the RedisVectorStore class from the LlamaIndex framework. Here's a step-by-step guide:

1. Initialize RedisVectorStore Instances: First, create RedisVectorStore instances for both your indexes. Specify the index name, Redis URL, and any other relevant configurations.

from llama_index.legacy.vector_stores.redis import RedisVectorStore

redis_vector_store_policy = RedisVectorStore(
    index_name="policy_index",
    redis_url="redis://localhost:6379/",
    overwrite=True
)

redis_vector_store_leave = RedisVectorStore(
    index_name="leave_index",
    redis_url="redis://localhost:6379/",
    overwrite=True,
    metadata_fields=["index_leave"]
)

2. Store Data in Redis: Convert your data into BaseNode objects and add them to Redis using the add method of your RedisVectorStore instances. You'll need to subclass BaseNode to fit your data structure.

from llama_index.core.schema import BaseNode

class CustomNode(BaseNode):
    # Implement necessary methods and properties here

nodes_policy = [CustomNode(index_policy=data) for data in base_nodes+objects]
nodes_leave = [CustomNode(index_leave=data) for data in list_of_documents_leave]

redis_vector_store_policy.add(nodes_policy)
redis_vector_store_leave.add(nodes_leave)

3. Retrieve Indexes and Apply Filters: To query your data, use the query method of the RedisVectorStore class. For exact matches, especially for index_leave, you'll need to construct a VectorStoreQuery object with appropriate MetadataFilters.

from llama_index.core.vector_stores.types import MetadataFilters, MetadataFilter, VectorStoreQuery

metadata_filters = MetadataFilters(
    filters=[
        MetadataFilter(key="index_leave", value="desired_leave_value", operator="eq")
    ]
)

query_embedding = [0.1, 0.2, 0.3]  # Use your actual query embedding
query = VectorStoreQuery(query_embedding=query_embedding, filters=metadata_filters, similarity_top_k=10)

result = redis_vector_store_leave.query(query)

This approach allows you to persist your indexes in Redis and query them efficiently, including applying filters for exact matches. Make sure to adjust the placeholders and example values to match your actual data and requirements.

Details

Sources

• llama-index-legacy/llama_index/legacy/vector_stores/redis.py
• llama-index-core/llama_index/core/schema.py
• llama-index-legacy/llama_index/legacy/schema.py
• llama-index-integrations/vector_stores/llama-index-vector-stores-redis/llama_index/vector_stores/redis/base.py
• llama-index-legacy/llama_index/legacy/vector_stores/types.py
• llama-index-legacy/llama_index/legacy/readers/redis/utils.py
• llama-index-integrations/vector_stores/llama-index-vector-stores-redis/llama_index/vector_stores/redis/init.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[Ninlawat-Puhu]

I got an error

Deprecated kwarg, index_name, found upon initialization. RedisVectorStore now requires an IndexSchema object

[Ninlawat-Puhu]

@logan-markewich Can you help me for this case? if you want more information don't hesitate ask

[logan-markewich]

@Ninlawat-Puhu You can use the redis vector store
https://docs.llamaindex.ai/en/stable/examples/vector_stores/RedisIndexDemo/?h=redisve