Search results

WIP: Unscale the gradients before gradient clipping in manual optimization

bugSomething isn't workingSomething isn't working

funStaff contributions outside working hours - to differentiate from the "community" labelStaff contributions outside working hours - to differentiate from the "community" label

precision: ampAutomatic Mixed PrecisionAutomatic Mixed Precision

Status: Draft (not ready).

Lightning-AI/pytorch-lightningnumber 19536

#19536 In Lightning-AI/pytorch-lightning;

· awaelchli opened on Feb 27, 2024

· 

2.4.x

Example of gradient clipping with manual optimization does not handle gradient unscaling properly

bugSomething isn't workingSomething isn't working

precision: ampAutomatic Mixed PrecisionAutomatic Mixed Precision

Status: Open.

#18089 In Lightning-AI/pytorch-lightning;

· function2-llx opened on Jul 15, 2023

· 

2.4.x

Support AMP with TPUs

fabriclightning.fabric.Fabriclightning.fabric.Fabric

featureIs an improvement or enhancementIs an improvement or enhancement

precision: ampAutomatic Mixed PrecisionAutomatic Mixed Precision

strategy: xla

trainer

Status: Open.

#17927 In Lightning-AI/pytorch-lightning;

· carmocca opened on Jun 26, 2023

· 

future

Precision output conversion in Lite wrappers

discussionIn a discussion stageIn a discussion stage

fabriclightning.fabric.Fabriclightning.fabric.Fabric

precision: ampAutomatic Mixed PrecisionAutomatic Mixed Precision

precision: doubleDouble precisionDouble precision

Status: Open.

#14859 In Lightning-AI/pytorch-lightning;

· awaelchli opened on Sep 22, 2022

· 

future

MPS AMP Support

accelerator: mpsApple Silicon GPUApple Silicon GPU

featureIs an improvement or enhancementIs an improvement or enhancement

precision: ampAutomatic Mixed PrecisionAutomatic Mixed Precision

priority: 1Medium priority taskMedium priority task

Status: Open.

#14358 In Lightning-AI/pytorch-lightning;

· justusschock opened on Aug 23, 2022

· 

future

[RFC] "auto" precision support

designIncludes a design discussionIncludes a design discussion

featureIs an improvement or enhancementIs an improvement or enhancement

plugin

precision: ampAutomatic Mixed PrecisionAutomatic Mixed Precision

precision: apex (removed)NVIDIA/apex precisionNVIDIA/apex precision

trainer: argument

Status: Open.

#12895 In Lightning-AI/pytorch-lightning;

· justusschock opened on Apr 27, 2022

· 

future

Mixed precision: scheduler and optimizer are called in the wrong order

bugSomething isn't workingSomething isn't working

lr scheduler

precision: ampAutomatic Mixed PrecisionAutomatic Mixed Precision

priority: 2Low priority taskLow priority task

Status: Open.

#5558 In Lightning-AI/pytorch-lightning;

· manifoldhiker opened on Jan 18, 2021

· 

future