Is there a way to use a system message?

[ananidze]

I'm currently working on an integration where I need to use a system message similar to the OpenAI library's chat completion call. Here’s the specific OpenAI code snippet I'm referring to:

const completion = await this.openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [
    {
      "role": "system",
      "content": system_message,
    },
    { role: "user", content: question }
  ],
});

In this code, the system message helps control the conversation context and guide the assistant’s behavior.

Does embedJs support anything similar? If not, are there any plans to add support for this kind of functionality, or is there a recommended workaround I could use within the current API?

Any guidance or suggestions would be greatly appreciated!

Thank you!

[adhityan]

The application already adds in a system messages when it initiates a LLM calls. The default system message used by embedJs can be updated using the setQueryTemplate method on RAGApplicationBuilder. The default system message used is -

You are a helpful human like chat bot. Use relevant provided context and chat history to answer the query at the end. Answer in full. If you don't know the answer, just say that you don't know, don't try to make up an answer.

Do not use words like context or training data when responding. You can say you do not have all the information but do not indicate that you are not a reliable source.

Introducing the system message in the conversation chain is handled by the internal prepare method on this file https://github.com/llm-tools/embedJs/blob/main/core/embedjs-interfaces/src/interfaces/base-model.ts.

If you want to have custom logic, you can also subclass the default OpenAi class provided. Just override the runQuery method in the subclass of https://github.com/llm-tools/embedJs/blob/main/models/embedjs-openai/src/openai-model.ts.

Hope that helps.

[ananidze]

Thanks @adhityan for your help!

Quick question: why doesn't the loader include URLs? Example of json data. when i use query or search it returns everything correctly but url is empty 🥲

[adhityan]

It should reply the source information in the sources parameter of the query response. If you used a url to a json, the url should be returned in there.

Could you tell me more about what the issue is and how to reproduce it?

[ananidze]

The problem is that, in the search results, the map_url field is being returned as an empty string, even though the JSON at the source contains valid URLs in the map_url field. Below is a sample of the relevant code and the output I am receiving.

async onModuleInit() {
    this.llmApplication = await new RAGApplicationBuilder()
        .setModel(SIMPLE_MODELS.OPENAI_GPT4_TURBO)
        .setEmbeddingModel(new OpenAiEmbeddings({ model: 'text-embedding-3-large' }))
        .addLoader(
            new WebLoader({
                urlOrContent:
                    'https://gist.githubusercontent.com/ananidze/4e3ceb24b9a28cc38112eacfd880f11c/raw/2080b31a6ef686c3385f9178dabc745ee72b8fdc/generated_data.json',
            })
        )
        .setVectorDb(new LanceDb({ path: path.resolve('./db') }))
        .build();

    const searchResult = await this.llmApplication.search('Where can I eat sinori?');
    console.log('searchResult', searchResult);
}

Search result i'm getting:

[
  {
    score: 0.2920462817285077,
    pageContent: '[ { "name": "Adjarian Kokhi", "cuisine": ["Georgian", "Adjarian"], "location": { "address": "Keda Municipality, Batumi", "coordinates": { "latitude": 41.575814, "longitude": 41.853835 }, "map_url": " }, "specialties": ["Adjarian Khachapuri", "Sinori", "Borano"], "features": ["Authentic atmosphere", "Local ingredients"], "rating": 4.6 }, { "name": "Alaverdi", "cuisine": ["Georgian", "Adjarian"], "location": { "address": "Konstantine Gamsakhurdia St, 36, Batumi", "coordinates": { "latitude": 41.650351, "longitude": 41.636021 }, "map_url": " }, "specialties": ["Achma", "Kaimagi", "Borano"], "features": ["Live music", "Family-friendly"], "rating": 4.8 }, { "name": "House of Adjarian Khachapuri", "cuisine": ["Georgian", "Adjarian"], "location": { "address": "Chavchavadze St, 17, Keda", "coordinates": { "latitude": 41.600300, "longitude": 41.869000 }, "map_url": " }, "specialties": ["Adjarian Khachapuri", "Malakhto", "Borano"], "features": ["Local dairy specialties", "Vegetarian options"], "rating": 4.5 }, { "name": "Tskhmorisi Guesthouse Restaurant", "cuisine": ["Georgian", "Adjarian"], "location": { "address": "Tskhmorisi Village, Keda", "coordinates": { "latitude": 41.558800, "longitude": 41.861200 }, "map_url": " }, "specialties": ["Kaimagi", "Sinori", "Adjarian Halva"], "features": ["Mountain views", "Traditional hospitality"], "rating": 4.7 } ]',
    metadata: {
      type: 'WebLoader',
      source: 'https://gist.githubusercontent.com/ananidze/4e3ceb24b9a28cc38112eacfd880f11c/raw/2080b31a6ef686c3385f9178dabc745ee72b8fdc/generated_data.json',
      id: 'WebLoader_df7619b5adfa7f085220a5d4abd68789_0',
      uniqueLoaderId: 'WebLoader_df7619b5adfa7f085220a5d4abd68789'
    }
  }
]

Expected Behavior:

{
  "name": "Tskhmorisi Guesthouse Restaurant",
  "location": {
    "address": "Tskhmorisi Village, Keda",
    "coordinates": { "latitude": 41.5588, "longitude": 41.8612 },
    "map_url": "https://maps.google.com/?q=41.5588,41.8612"
  }
}

However, as you can see in the search result, the map_url is being returned as an empty string, even though the source JSON contains valid URLs.

[adhityan]

The gist is deleted now. Could you get me the sample again so I can take a look?

[ananidze]

This is an example of data i provided. I have fixed my issue by removing this code from web-loader file

  const text = convert(data, {
      wordwrap: false,
      preserveNewlines: false,
  }).replace(/(?:https?|ftp):\/\/[\n\S]+/g, '');

https://github.com/llm-tools/embedJs/blob/main/loaders/embedjs-loader-web/src/web-loader.ts