New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explore Alternative ML Model Providers #300

Open

1 task

davidgxue opened this issue Feb 16, 2024 · 0 comments

Collaborator

davidgxue commented Feb 16, 2024 •

edited

Please describe the feature you'd like to see
During downtimes of services such as OpenAI, Cohere and Azure OpenAI, have hot swappable.

For cost reduction, we should also do A/B testing and see how other models perform compared to GPT-4 as GPT-4 itself is very expensive.

Describe the solution you'd like
Explore using OpenAI and Anthropic for LLM services as backup for Azure OpenAI's GPT-3.5 and GPT-4.

Are there any alternatives to this feature?
Explore open source LLMs

Additional context

Acceptance Criteria

Implement env variable based mechanism to hot swap an existing LLM to another provider during service downtime

davidgxue mentioned this issue

Cost Reduction for Ask Astro #295

Closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment