Skip to content

modify value_outputs_unwrap method #61

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

whatchang
Copy link

First, thank you for releasing your awesome paper and code.

I found one problem when I ran your Quick Start code that is written in README.
(That is below)

import argparse
from tot.methods.bfs import solve
from tot.tasks.game24 import Game24Task

args = argparse.Namespace(backend='gpt-4', temperature=0.7, task='game24', naive_run=False, prompt_sample=None, method_generate='propose', method_evaluate='value', method_select='greedy', n_generate_sample=1, n_evaluate_sample=3, n_select_sample=5)

task = Game24Task()
ys, infos = solve(args, task, 900)
print(ys[0])

The problematic line of code I found is shown below.
(Thas is in src->tot->tasks->game24.py line 91)

sum(value * value_names.count(name) for name, value in value_map.items())

In the above code, the problem was caused by the value_names being in the list type rather than a string.

As a result, we can see that the first string of value_names does not count even though it has 'sure' in it, as shown in the WATCH part of the figure below.

오류 n이 3일때

This also happens when n_evaluate_sample=1.

오류 n이 1일때

So, I modified the code to look like this.

sum(value * target_value.count(name) for name, value in value_map.items() for target_value in value_names)

Thank you for reading and have a great day!

@ysymyth
Copy link
Member

ysymyth commented Jan 16, 2025

thanks! i think the problem is, what if target_value is a string like "sure sure likely impossible" or "i am sure i do not know"? exact match is more conservative but maybe less risky

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants