Low GPU utilization

[jbuckman]

This is my first time trying out PyTorch Lightning. On a node with 8 H100s, I'm running the code from this example: https://lightning.ai/lightning-ai/studios/pretrain-an-llm-with-pytorch-lightning?tab=overview

All three versions, small medium large, get extremely low GPU utilization, nvidia-smi fluctuating but typically around 50%. Is the example code not correct? What changes need to be made?