Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[UPRISE]What is the use of some keys in hellaswag and copa? #282

Open
zhouchang123 opened this issue Nov 12, 2024 · 2 comments
Open

[UPRISE]What is the use of some keys in hellaswag and copa? #282

zhouchang123 opened this issue Nov 12, 2024 · 2 comments

Comments

@zhouchang123
Copy link

What is the use of some keys in hellaswag and copa?
For example:
copa:{
"premise": "The woman filed a restraining order against the man.",
"choice1": "The man called her.",
"choice2": "The man stalked her.",
"question": "cause",
"idx": 302,
"label": 1,
"id": 0,
"task_name": "copa"
}
What's the use of “idx” in copa?
hellaswag:{
"ind": 34409,
"activity_label": "Home and Garden",
"ctx_a": "[header] How to make a butterfly out of plastic spoons [title] Gather the materials you will need for this project, listed below. [title] Put a craft cloth or some newspaper down on your working surface. [title] Cut the top portion of the four spoons off (leaving about half an inch of the handle left.",
"ctx_b": "",
"ctx": "[header] How to make a butterfly out of plastic spoons [title] Gather the materials you will need for this project, listed below. [title] Put a craft cloth or some newspaper down on your working surface. [title] Cut the top portion of the four spoons off (leaving about half an inch of the handle left.",
"endings": [
"[step] ) the top portion is the flat spoon part. [title] Using the cut-off handle of one of the spoons, make that the " body " of the butterfly.",
"[step] As the buttons stretch they will process the solution. ) [title] Put one of the red plastic buttons (left over for the butterfly) on top of the red plastic spoon.",
"[step] And when done, you should have 8 small, square, circular spoons. [title] Use these four spoons as either a mirror, or a butterfly.",
"[step] ) [title] Thread a thin needle into a cast iron ring with four ends attached. [step] Thread the needle to the ring, and then sew the top portion onto the handle of the spoon at the top, or the side which should be facing away from the spoon."
],
"source_id": "wikihow~56680",
"split": "train",
"split_type": "indomain",
"label": "0",
"id": 0,
"task_name": "hellaswag"
}
What's the use of "ind",“activity_label”,"ctx_a","ctx_b","source_id","split","split_type" in copa?

@zhouchang123
Copy link
Author

Could you please explain the quest@cdxeve

@cdxeve
Copy link
Contributor

cdxeve commented Nov 17, 2024

Hi, I've included code in task.py for processing the relevant information in each dataset example. Any additional information (such as those keys you've listed above) that isn't needed is simply ignored during processing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants