Question about logging with DDP

[hypefolder]

Does calling self.log from a lightning module during distributed training only save the results from rank 0? It is my understanding that if sync_dist is set to False then the metrics are not reduced and only rank 0 metrics are logged. Could you also confirm that the actual logging is only performed on the rank 0 GPU?

Thanks in advance for your help!

[alvaroprat97]

If you want rank zero only then pass the flag rank_zero_only = True. If not, I think the last rank to perform the logging operation will log the value, leading to some inconsistencies if the dataset is not large enough.