Unable to Configure Model in Notebook

[yaonan]

Version: 1.86.0

Description

When trying to configure a model in a Notebook through the script({ model: 'some-model' }) in JS script cell or the page config YAML, the model configuration does not take effect.

The Notebook still prompts that no model has been selected or defaults to the model configured in the .env file.

During debugging, I found that in the src/docsnotebook.ts file, the activateNotebookExecutor function correctly parses the model from the meta (const meta = parsePromptScriptMeta(jsSource)), but it seems that the meta.model is not properly used as a parameter in the subsequent startAIRequest function in src/state.ts.

To temporary resolve this, I manually modified the compiled JS file in the VS Code extension directory as follows:

        const { runId, request } = await client.startScript(
            template.id,
            files,
            {
                jsSource: options.jsSource,
                signal,
                trace,
                infoCb,
                partialCb,
                label,
                cache: cache ? template.cache : undefined,
                vars: structuredClone(options.parameters),
                model: template.model
            }
        )

After passing model: template.model in the third parameter, it seems to work as expected.

[github-actions]

Feedback: The issue you're encountering arises from a potential mismatch between how the model is parsed and passed through different functions within the system. Your temporary workaround addresses the core problem by ensuring that the model parameter is correctly propagated from the meta data into the AI request process.

To ensure a more robust and maintainable solution, consider implementing a check to validate and apply the model setting consistently across all relevant functions. This could involve adding a default model in place of undefined values or handling cases where no model is explicitly set.

Additionally, reviewing the startAIRequest function for any potential side effects or redundant parameter passing can help identify if there are other points in the system that also need adjustments to align with the intended flow.

Once these changes are made and tested thoroughly, ensure that they do not introduce new bugs or regressions. It might be a good idea to include comprehensive unit tests covering various scenarios of model configuration to validate future functionality.

AI-generated content issue-reviewer may be incorrect

[yaonan]

At the same time, I found that when I try to use system prompts in the Notebook, it seems that they cannot be used either.

Currently, I can modify the src/docsnotebook.ts file mentioned above to add:

model: template.model,
system: template.system,

Additionally, the runScriptInternal function in src/run.ts needs to be modified as follows:

    const prj = await buildProject({
        toolFiles,
    })
    if (jsSource)
        prj.scripts.push({
            id: scriptId,
            jsSource,
        })
    const script = prj.scripts.find(
        (t) =>
            t.id === scriptId ||
            (t.filename &&
                GENAI_ANY_REGEX.test(scriptId) &&
                resolve(t.filename) === resolve(scriptId))
    )
    if (!script) throw new Error(`script ${scriptId} not found`)
    const fragment: Fragment = {
        files: Array.from(resolvedFiles),
    }

In the following section, add model and system:

        prj.scripts.push({
            id: scriptId,
            jsSource,
            model: options.model,
            system: options.system,
        })
        

In the original code, the script for the current Notebook Cell was added to prj.scripts, but it only included template.id and jsSource. I don’t know if this was by design or a bug. In any case, from this point on, the script no longer contains information such as model and system, and they are no longer included when called later.

[pelikhan]

Thanks for digging.