Add warning when the model has eval mode parameters #20573
Description
Outline & Motivation
After the update for 2.2.0, parameters that are in eval mode before the training are not changed to train mode in the training loop.
https://github.com/Lightning-AI/pytorch-lightning/releases/tag/2.2.0
In some libraries, for example HuggingFace, models are created in evaluation mode by default (e.g. HFModel.from_pretrained(...)). Starting from 2.2, you will have to set .train() on these models if you intend to train them.
This update is effective for unfortunate side effects.
However, I think this feature can often be overlooked in many cases because users (including me) would expect the train/eval mode to be automatically changed in the loop.
Pitch
To avoid this problem, it would be effective to display a warning before training that the model parameters are in eval mode.
Additional context
No response