Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Details for GPT4 evaluation #145

Open
jongwooko opened this issue Dec 26, 2023 · 0 comments
Open

Details for GPT4 evaluation #145

jongwooko opened this issue Dec 26, 2023 · 0 comments

Comments

@jongwooko
Copy link

Hi. Can I ask about the query of GPT-4 evaluation in detail?

I tried to

"""
We would like to request your feedback on the performance of two AI assistants in response to the user instruction and input displayed above.
Please rate the helpfulness, relevance, accuracy, and level of detail of their responses. Each assistant receives an overall score on a scale of 1 to 10, where a higher score indicates better overall performance.
Please first output a single line containing only two values indicating the scores for Assistant 1 and 2, respectively. The two scores are separated by a space.
In the subsequent line, please provide a comprehensive explanation of your evaluation, avoiding any potential bias and ensuring that the order in which the responses were presented does not affect your judgment.

Below is an instruction that describes a task.
Write a response that appropriately completes the request.

Instruction:

Determine the sentiment of the input sentence. Please respond as positive or negative.

Input:

{sentence}

Assistant 1:

{output}

Assistant 2:

{ground truth}
"""

with my query, the GPT-4 evaluation is 10 larger than you reported. can you share the detailed query and GPT-4 API model name for your experiments?
Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant