🙅 PPO value_model can't be None, so it shouldn't be Optional

[AMindToThink]

Simple change to the typing and default value of value_model in PPOTrainer. Since it cannot be None, it should not default to None or be marked Optional. See this issue.

[qgallouedec]

This looks reasonable, thanks! 🙏

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qgallouedec]

@AMindToThink are your latest commits for this PR?

[AMindToThink]

Oops, sorry, no. Reverted.

I'm working to allow users to save the value model along with the policy model.

I'm running into some odd errors. I'll make a bug report.

[AMindToThink]

Hi @qgallouedec,
It turns out that this issue was already raised a year and a half ago, and you decided to keep the value model Optional with a None default to keep the documentation easier to read. Sorry for rehashing old questions!

I have made an issue with the bugs I ran into while trying to implement saving the value model. I would appreciate your help!

Thank you,
Matthew (@AMindToThink)