transform model checkpoints

[hehulingxiao]

Hi, thanks for your nice work!

I trained the model and obtain multiple *.pt files, how can I transform them to Huggingface model, so that I can use transformers to load the model conveniently?

[mantle2048]

Hi, thanks for your attention!

https://github.com/MiniMax-AI/One-RL-to-See-Them-All/blob/main/scripts/model_merger.sh
https://github.com/MiniMax-AI/One-RL-to-See-Them-All/blob/main/scripts/model_merger.py

You can see these two files, which can convert the intermediate states saved during training into HF format.

[hehulingxiao]

Thanks for your reply!

[hehulingxiao]

Hi, I have another question.

What is the difference between the online test-set benchmarks and the final evaluation? For example, for COCO or CountBench evaluation, is the reported results in the paper same with the online testing results?

[mantle2048]

Hi, the online test-sets are solely used to monitor whether the training is proceeding normally. This data is often a subset of the benchmark (for example, the sample with a answer that can be easily verified by rules) and doesn't represent the final performance.

The performance metrics presented in the paper are all based on offline evaluations conducted after saving the checkpoint.

[hehulingxiao]

I see. Thanks for your response!