Error: OpenAI or Azure OpenAI API key or Token Provider not found with Ollama

[donsimn]

I'm just trying out embedjs with my local Ollama server. I wrote this super simple code which fails:

const { RAGApplicationBuilder, Ollama } = await import('@llm-tools/embedjs');
const { LanceDb } = await import('@llm-tools/embedjs/vectorDb/lance');

const rag = await new RAGApplicationBuilder()
    .setModel(new Ollama({
        modelName: "llama3",
        baseUrl: 'http://localhost:11434'
    })).setVectorDb(new LanceDb({ path: 'lance-', isTemp: true }))
    .build();

console.log(await rag.query("Hello"));

The error:

file:///C:/Users/simon/Documents/GitHub/william-yeye/node_modules/@langchain/openai/dist/embeddings.js:128
            throw new Error("OpenAI or Azure OpenAI API key or Token Provider not found");
                  ^

Error: OpenAI or Azure OpenAI API key or Token Provider not found
    at new OpenAIEmbeddings (file:///C:/Users/simon/Documents/GitHub/william-yeye/node_modules/@langchain/openai/dist/embeddings.js:128:19)
    at new OpenAi3SmallEmbeddings (file:///C:/Users/simon/Documents/GitHub/william-yeye/node_modules/@llm-tools/embedjs/dist/embeddings/openai-3small-embeddings.js:10:22)
    at new RAGApplication (file:///C:/Users/simon/Documents/GitHub/william-yeye/node_modules/@llm-tools/embedjs/dist/core/rag-application.js:69:61)
    at RAGApplicationBuilder.build (file:///C:/Users/simon/Documents/GitHub/william-yeye/node_modules/@llm-tools/embedjs/dist/core/rag-application-builder.js:71:24)
    at file:///C:/Users/simon/Documents/GitHub/william-yeye/dist/index.js:8:6

[adhityan]

Yes, this is expected behaviour. Basically, you need two things to run a RAG stack -

1. LLM
2. Embedding model

In your case, you are using Ollama as the LLM and by default (unless you specify otherwise), the library uses OpenAI's LargeEmedding as its embedding model. The error you see is coming from the embedding model unable to reach out to OpenAI. Right now, there is no support for local embedding models via Ollama (only Ollama based LLMs are supported) but there is a plan to add it soon. There are other embedding models to choose from though.