Why can't I beat the difficulty level 1 of the first map of LLM-PySC2 using gpt3.5_turbo and gpt_3.5_turbo_16k?

[mingtouyizu]

[734162621]

This is a normal situation, not an error.

The decision-making capability of GPT-3.5-turbo is relatively low. It requires fine-tuning or other technical solutions to enhance its decision-making ability.

In our tests, Claude-haiku won in Task 1 Level 1 with an approximate winning rate of 90% over 20 games of testing, while the winning rate of GPT-3.5 is about 60%. After finishing the benchmark tests, we will display relevant results in both arXiv paper and this repository.

Ideas of increasing LLMs' ability depend on researchers' future works. You can test your ideas in our environment.