Skip to content

Issues: openai/evals

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

85 Open 114 Closed
85 Open 114 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Support for GPT-4o
#1529 opened May 15, 2024 by PrashantDixit0
What is this
#1527 opened May 3, 2024 by DXv-3
1
Setting completion function args via CLI does not work bug Something isn't working
#1504 opened Mar 27, 2024 by LoryPack
1
OpenAIChatCompletionFn should __init__ should accept **kwargs bug Something isn't working
#1493 opened Mar 15, 2024 by ezraporter
1
Support multiple completions for ModelbasedClassify
#1484 opened Mar 14, 2024 by tom-christie
Support for Azure OpenAI client
#1469 opened Feb 21, 2024 by pkt1583
2
Request to change arithmetical_puzzles prompting
#1448 opened Dec 24, 2023 by ArcticBeat05
Improvements to Match: case insensitive and strip
#1421 opened Nov 30, 2023 by LoryPack
Proposal for Adding a New Evaluation Metric: Sentiment Analysis Accuracy
#1419 opened Nov 23, 2023 by Sarfaraz021
Using different models in evaluating mode-graded eval and in generating the completion
#1393 opened Nov 3, 2023 by LoryPack
5
In the task "balance_chemical_equation", many instances have incorrect labels. bug Something isn't working
#1386 opened Oct 19, 2023 by dongZheX
1
Eval-running often hangs on last sample bug Something isn't working
#1384 opened Oct 17, 2023 by sjadler2004
4
Should random collection of values be supported?
#1382 opened Oct 17, 2023 by assert6
Multiple evals not found bug Something isn't working
#1379 opened Oct 16, 2023 by SUMEETRM
5
Context window of completion functions not accounted for bug Something isn't working
#1377 opened Oct 13, 2023 by pskl
Use github.com/apssouza22/chatflow as a conversational layer. It would enable actual API requests to be carried out from natural language inputs.
#1362 opened Sep 25, 2023 by GiovanniSmokes
Evaluate the cost of running tests
#1350 opened Sep 15, 2023 by onjas-buidl
Feature request for evals: Add support for function call.
#1346 opened Sep 7, 2023 by srenault
Publish latest evals framework to PyPI
#1344 opened Aug 31, 2023 by robatwilliams
How to eval output with ideal_answer directly without having to define the completion_fn ?
#1342 opened Aug 29, 2023 by liuyaox
1
Having trouble building Evals locally? Try this. bug Something isn't working
#1340 opened Aug 25, 2023 by silverfoxf7
1
Find claims from research paper
#1338 opened Aug 21, 2023 by ghost
Sample evaluations completing after timeout cause duplicate results bug Something isn't working
#1333 opened Aug 10, 2023 by robatwilliams
1
Accuracy Score
#1328 opened Aug 3, 2023 by jeyarajcs
oaieval hangs a lot bug Something isn't working
#1292 opened Jul 5, 2023 by shamas-
1
ProTip! Find all open issues with in progress development work with linked:pr.