New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add BERTScore feature descriptor and feature #1335

Draft

projectultra wants to merge 1 commit into evidentlyai:main from projectultra:main

+136 −0

projectultra commented Oct 5, 2024

PR for the BERTScore feature from #1317 . The main changes include adding the BERTScore descriptor and feature, updated registry, added tests.

New Feature Implementation:

src/evidently/features/BERTScore_feature.py: Introduced the BERTScoreFeature class that generates BERT-based similarity scores between two text columns. This class uses a pretrained BERT model (bert-base-uncased) to compute the cosine similarity between token embeddings, then calculates the precision, recall, then calculates the F1 score (BERTScore).

Testing:

tests/features/test_bertscore_feature.py: Added tests for the BERTScoreFeature to ensure the feature generation works as expected using predefined test data and expected BERTScore values.

Obtained the expected values using Repo of the paper referenced in the paper BERTScore: Evaluating Text Generation with BERT. The results are available here.


feat: Add BERTScore feature descriptor and feature

d47ed87

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment