Skip to content

Issues: confident-ai/deepeval

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

114 Open 198 Closed
114 Open 198 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

CLI improvement: option to display only failed tests
#1235 opened Dec 18, 2024 by yuriykuzin
1
ToolCorrectnessMetric raises ZeroDivisionError when expected_tools is empty
#1234 opened Dec 18, 2024 by j-mesnil
Doc Improvement
#1225 opened Dec 13, 2024 by SamyAteia
4
Stop deepeval from storing sensitive information in dotenv files.
#1221 opened Dec 11, 2024 by thomend
3
Can't pass trust_remote_code to hallucination_score
#1220 opened Dec 10, 2024 by mausch
2
Generate Your First Golden - Using Custom model
#1216 opened Dec 8, 2024 by pratikchhapolika
3
Documentation for ContextualPrecision
#1212 opened Dec 6, 2024 by enrico-stauss
Commented piece of code for Golden creation
#1211 opened Dec 5, 2024 by jaroslaw-pawluszka
1
Add image support to LLMTestCase
#1210 opened Dec 5, 2024 by ethanelasky
11
Make TELEMETRY_DATA_FILE located into temp folder or configurable by env var
#1200 opened Nov 29, 2024 by cisarpavel
AssertionError: n_contexts_per_doc must be a positive integer.
#1199 opened Nov 29, 2024 by tenchanhe
2
Issue regarding consistent loading of source_files data (from goldens original values)
#1173 opened Nov 21, 2024 by CAW-nz
3
How to integrate llama3 to deepeval?
#1163 opened Nov 15, 2024 by drbhushan
1
GEval not focusing on expected_output & Relying on OpenAI instead
#1149 opened Nov 11, 2024 by pavan-growexxer
2
Asynchronous test runs are sometimes not completed correctly
#1147 opened Nov 9, 2024 by jmaczan
16
Hallucination metric score is set to 0.0 in one run and 1.0 in another run, despite having the same input values
#1146 opened Nov 9, 2024 by jmaczan
1
Hallucination metric assigns only either 0.0 or 1.0 score
#1145 opened Nov 9, 2024 by jmaczan
1
EvaluationDataset in deepeval suddenly requires an OpenAi key since update
#1140 opened Nov 4, 2024 by kbarendrecht
5
Error while calculating Knowledge retention ; Evaluation LLM outputted an invalid JSON. Please use a better evaluation model.
#1139 opened Nov 4, 2024 by jaysudhakaran
4
Improving Answer Relevancy Template
#1137 opened Nov 3, 2024 by dipanjanS
4
Get tokens for each request
#1114 opened Oct 22, 2024 by luqianjiang
1
TypeError while evaluating a test case for Role Adherence metric using CustomLlama3_8B.
#1105 opened Oct 21, 2024 by Qca-2023
HumanEval doesn't actually runs test
#1102 opened Oct 19, 2024 by cfytrok
The latest versions of deepeval and Instructor can't be installed together
#1100 opened Oct 18, 2024 by milest
5
APIConnectionError: Connection error.
#1094 opened Oct 17, 2024 by paulacanva
3
ProTip! Add no:assignee to see everything that’s not assigned.