readme

Author: lucidrains

README.md

@@ -1,2 +1,16 @@
-# adam-atan2-pytorch
-Implementation of the proposed Adam-atan2 in Pytorch
+## Adam-atan2 - Pytorch (wip)
+
+Implementation of the proposed <a href="https://arxiv.org/abs/2407.05872">Adam-atan2</a> optimizer in Pytorch
+
+A multi-million dollar paper out of google deepmind basically proposes a small change to Adam (using `atan2`) for greater stability
+
+## Citations
+
+```bibtex
+@inproceedings{Everett2024ScalingEA,
+    title   = {Scaling Exponents Across Parameterizations and Optimizers},
+    author  = {Katie Everett and Lechao Xiao and Mitchell Wortsman and Alex Alemi and Roman Novak and Peter J. Liu and Izzeddin Gur and Jascha Narain Sohl-Dickstein and Leslie Pack Kaelbling and Jaehoon Lee and Jeffrey Pennington},
+    year    = {2024},
+    url     = {https://api.semanticscholar.org/CorpusID:271051056}
+}
+```