You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Giskard RAGet uses the reference context when calling Ragas.
https://github.com/Giskard-AI/giskard/blob/main/giskard/rag/metrics/ragas_metrics.py
ragas_sample = {
"question": question_sample["question"],
"answer": answer,
"contexts": question_sample["reference_context"].split("\n\n"),
"ground_truth": question_sample["reference_answer"],
}
According to Ragas documentation the retrieved context should be used (the one used for the answer Generation).
As an example, when computing Precision or Recall which both uses {"question", "contexts", "ground_truth"}, if you are giving the reference context, then you are evaluating your testset generation pipeline and not your RAG pipeline.
Standalone code OR list down the steps to reproduce the issue
.
Relevant log output
No response
The text was updated successfully, but these errors were encountered:
Issue Type
Bug
Source
source
Giskard Library Version
2.11
Giskard Hub Version
OS Platform and Distribution
No response
Python version
No response
Installed python packages
No response
Current Behaviour?
Standalone code OR list down the steps to reproduce the issue
.
Relevant log output
No response
The text was updated successfully, but these errors were encountered: