Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom fine-tuned model gets weird results #803

Open
penguinwang96825 opened this issue Sep 6, 2024 · 1 comment
Open

Custom fine-tuned model gets weird results #803

penguinwang96825 opened this issue Sep 6, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@penguinwang96825
Copy link

penguinwang96825 commented Sep 6, 2024

I'm experiencing unusual results with the custom fine-tuned model textattack/roberta-base-SST-2. The clean accuracy is significantly lower than expected, at 50.69%. I've included the attack results below, which show a low accuracy under attack and a high number of skipped attacks. I'd appreciate it if you could look into this issue.

截圖 2024-09-06 下午4 02 10
@penguinwang96825
Copy link
Author

I'm also encountering an issue with the bert-base-uncased model fine-tuned on several datasets. It suddenly behaves abnormally. I've tried reinstalling TextAttack, but the problem persists.

@jxmorris12 jxmorris12 added the bug Something isn't working label Oct 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants