Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using OpenAI tools with Assistant fails #852

Open
3 tasks done
Fakamoto opened this issue Feb 16, 2024 · 2 comments
Open
3 tasks done

Using OpenAI tools with Assistant fails #852

Fakamoto opened this issue Feb 16, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@Fakamoto
Copy link

First check

  • I added a descriptive title to this issue.
  • I used the GitHub search to try to find a similar issue and didn't find one.
  • I searched the Marvin documentation for this issue.

Bug summary

Using
marvin.beta.assistants.Retrieval
marvin.beta.assistants.CodeInterpreter

in Assistant tools fails

Reproduction

from marvin.beta import Assistant
from marvin.beta.assistants import pprint_messages, CodeInterpreter

ai = Assistant(name='Marvin', tools=[CodeInterpreter])
response = ai.say("Generate a plot of sin(x)")

# pretty-print the response
pprint_messages(response)

Error

Traceback (most recent call last):
  File "/Users/test/main.py", line 5, in <module>
    response = ai.say("Generate a plot of sin(x)")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/marvin/utilities/asyncio.py", line 190, in sync_wrapper
    return run_sync(coro)
           ^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/marvin/utilities/asyncio.py", line 103, in run_sync
    return asyncio.run(coroutine)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/marvin/beta/assistants/assistants.py", line 96, in say_async
    async with self:
  File "/opt/homebrew/lib/python3.11/site-packages/marvin/beta/assistants/assistants.py", line 117, in __aenter__
    await self.create_async(_auto_delete=True)
  File "/opt/homebrew/lib/python3.11/site-packages/marvin/beta/assistants/assistants.py", line 139, in create_async
    response = await client.beta.assistants.create(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/openai/resources/beta/assistants/assistants.py", line 411, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/openai/_base_client.py", line 1725, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/openai/_base_client.py", line 1428, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/openai/_base_client.py", line 1519, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "5 validation errors for Request\nbody -> tools -> 0 -> function\n  extra fields not permitted (type=value_error.extra)\nbody -> tools -> 0 -> type\n  unexpected value; permitted: <ToolTypeParam.RETRIEVAL: 'retrieval'> (type=value_error.const; given=code_interpreter; permitted=(<ToolTypeParam.RETRIEVAL: 'retrieval'>,))\nbody -> tools -> 0 -> function\n  extra fields not permitted (type=value_error.extra)\nbody -> tools -> 0 -> type\n  unexpected value; permitted: <ToolTypeParam.FUNCTION: 'function'> (type=value_error.const; given=code_interpreter; permitted=(<ToolTypeParam.FUNCTION: 'function'>,))\nbody -> tools -> 0 -> function\n  none is not an allowed value (type=type_error.none.not_allowed)", 'type': 'invalid_request_error', 'param': None, 'code': None}}

Versions

marvin version
Version:                2.1.5
Python version:         3.11.7
OS/Arch:                darwin/arm64

openai --version
openai 1.12.0

>>> import pydantic
>>> pydantic.__version__
'2.6.1'

Additional context

No response

@Fakamoto Fakamoto added the bug Something isn't working label Feb 16, 2024
@Fakamoto
Copy link
Author

Fakamoto commented Feb 16, 2024

the problem seems to be that

class RetrievalTool(Tool[T]):
    type: Literal["retrieval"] = "retrieval"

and

class CodeInterpreterTool(Tool[T]):
    type: Literal["code_interpreter"] = "code_interpreter"

inherit from Tool

class Tool(MarvinType, Generic[T]):
    type: str
    function: Optional[Function[T]] = None

which has a function field that is invalid in the Retrieval and CodeInterpreter case, as OpenAI is expecting {"type": "retrieval"}

so when you do

    @expose_sync_method("create")
    async def create_async(self, _auto_delete: bool = False):
        [....]

        response = await client.beta.assistants.create(
            **self.model_dump(
                include={"name", "model", "metadata", "file_ids", "metadata"}
            ),
            tools=[tool.model_dump() for tool in self.get_tools()],
            instructions=self.get_instructions(),
        )

the tool.model_dump() returns {"type": "retrieval", "function": None}

which makes it fail as OpenAI does not like this...

The workaround I found is to delete the function field from the models as soon as I import them

from marvin.beta.assistants import Retrieval
del Retrieval.function

@Fakamoto
Copy link
Author

I would imagine a better solution would be to modify the Tool model so it does not include the "function" field and name it BaseTool, make RetrievalTool and CodeInterpreterTool inherit from BaseTool. Then create a Tool model that inherits from BaseTool and adds the "function" field

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant