how to freeze embedding

[threestone965]

i want to freeze EmbeddingCollection mid training, but setting the learning rate to 0.0 does not save cuda memory and computing resources. is there a method similar to the following code. looking forward to your reply, thanks

for param in module.parameters():
        param.requires_grad = False

[TroyGarden]

You might want to provide more context on your use case. I assume you are referring to the ShardedEmbeddingCollection after the DMP wrapper.

is this your main purpose to freeze the embedding weights during training?

but setting the learning rate to 0.0 does not save cuda memory and computing resources.