Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output randomization? #1

Open
hzyjerry opened this issue May 26, 2023 · 1 comment
Open

Output randomization? #1

hzyjerry opened this issue May 26, 2023 · 1 comment

Comments

@hzyjerry
Copy link

Hi, thanks for this wonderful effort.

I'm wondering if there is any result on reproducing the output randomization findings (Min et al) using GINC? Have you tried it, and if so, would you provide the code/instruction for doing so?

Thank you.

@sangmichaelxie
Copy link
Member

Sorry for the late response! We haven't run that before, but it would be interesting. To do it, you just have to modify the list here:

prompt = [all_values[values[j], slots[j]] for j in range(len(slots))]

The structure of the prompt list is a concatenation of prompt_length tokens + a delimiter for n_example_per_prompt + 1 times. You just have to replace every (example_index + 1) * (prompt_length + 1) - 2-th element, e.g. permute them and add them back in. Don't replace the last example's label though, since that's the test example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants