Improve the tutorials for optimization on a manifold #1035
Description
Currently OptimizationManopt is significantly under-documented and isn't really showcased. There should be something that is bumped up to being a tutorial, i.e. an actual top level tutorial of the repo on "Optimization on Manifolds" and not just examples inside of the https://docs.sciml.ai/Optimization/stable/optimization_packages/manopt/ page. However, it's not currently clear what the best tutorial to write is. That said, the top level tutorial should be something both simpler and more natural than the things currently documented as examples. It should be something a first physicist working on manifolds would expect to find. The manifold of SPD matrices is an extremely odd manifold so it's just not expected. I would think something like "optimizing a function on a sphere" is likely a good choice. It should probably showcase OptimizationManopt as one option, with another being constrained optimization, and another using the new DAE approach, and should have a discussion of the differences between the approaches and when one or the other is likely to be a good idea.
@kellertuer any comments on what you think a good first tutorial on the topic would likely be for the Optimization.jl audience? I think the goal here would be to make a tutorial that is inviting so that people who don't know or understand manifolds stuff can look at this and go "oh, actually my problem would be better suited to use OptimizationManopt", so it's a bit different than say a first tutorial in Manopt.jl which assumes the user understands manifolds, instead this is probably better served as more of a discovery avenue for folks outside of that domain. Then of course extra examples can go into further detail, but the main point really is to point users into the right direction via a good motivating example.