[Paper] FourCastNet3

[jacobbieker]

Arxiv/Blog/Paper Link

https://d1qx31qr3h6wln.cloudfront.net/publications/FourCastNet3.pdf

Training code: https://github.com/NVIDIA/makani?tab=readme-ov-file

Detailed Description

Lots of interesting details here, training code does lots of different kinds of parallelism, only using convolutions on a sphere and some diffiusion (but not really) noise. Spectral loss would be cool to have here.

[aniket2405]

Hey @jacobbieker, how are you doing?
I had worked on Generating physically-consistent high-resolution climate data with hard-constrained neural networks
ref: ff43cc1

Had a great learning experience and wanted to work on something new and interesting here!
This looks pretty exciting, wanted to try my hands on this.

Will start reading and understanding the paper. Do you have some initial thoughts on what do we expect here? Thanks for your time!

[jacobbieker]

Hey! Thanks for that, and yeah, very happy to have you work on this.

The initial things that I found interesting in this, and this would be split among multiple PRs I would suggest, is their per-channel group encoder, their adjusted CRPS loss function, and the model/data parallelism, if possible. Add more spherical convolutions on the sphere would also be interesting, but those first few are the ones that I think are the most generally applicable and useful to other models as well.

[aniket2405]

can you assign this to me? @jacobbieker i've started working on this already

[jacobbieker]

Great! Yes, I can now

[glitch401]

hey @jacobbieker , is this issue still being worked on?

[jacobbieker]

Hi, I'm not sure if it is, @aniket2405 have you been working on this? I would say feel free to go ahead @glitch401 if you want to try doing it!

[aniket2405]

Hey @jacobbieker @glitch401 , I'm actually working on this, yeah. I'm towards the last testing part of my code, will raise a patch soon!

[glitch401]

Thanks for the update, @aniket2405! Glad to hear you're close to finishing.
Since I've already set up the full development environment and have been implementing the foundational layers for the loss fn. and SpectralConv, I'd be happy to help test your pull request or review the code when it's ready. Looking forward to seeing your implementation!