Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature]: Ollama schema structured output #7131

Open
vitreuz opened this issue Dec 10, 2024 · 2 comments · May be fixed by #7344
Open

[Feature]: Ollama schema structured output #7131

vitreuz opened this issue Dec 10, 2024 · 2 comments · May be fixed by #7344
Labels
enhancement New feature or request

Comments

@vitreuz
Copy link

vitreuz commented Dec 10, 2024

The Feature

Add support for structured schemas in ollama

Motivation, pitch

As of v0.5.0, ollama now supports full schema structured format. Currently, litellm only seems to support JSON mode.

Are you a ML Ops Team?

No

Twitter / LinkedIn details

No response
@vitreuz vitreuz added the enhancement New feature or request label Dec 10, 2024
@vitreuz
Copy link
Author

vitreuz commented Dec 10, 2024

litellm can be modified relatively easily to support the previous JSON mode in addition to a structured schema by leveraging the existing non_default_params["response_format"] field differently in the litellm/ollama.py and litellm/ollama_chat.py.

I can make this change if desired, but I just need to wrap my head around testing in this library.

@adrianlyjak adrianlyjak mentioned this issue Dec 10, 2024
Open
@adrianlyjak
Copy link

I for one would love that!

@vitreuz vitreuz linked a pull request Dec 21, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Add support for ollama structured outputs vitreuz/litellm
2 participants
@adrianlyjak @vitreuz