Feature Request: RAG support for multimodal models

[azaylamba]

Currently, we can upload an image to get answers about it using multimodal models. But this feature doesn't support RAG workflow.
It would be good if the model first understand the image and then answers the question about it using the documents stored in the vector storage. This way, users would be able to get answers by uploading images of errors.

[charles-marion]

I agree with you, it would mean moving the multimodal logic (at least for bedrock) to the Langchain interface. (Or add Rag support toe the idefics one)

I will keep this issue updated if it's being added.