[ENH] extension templates for neural network classes

[fkiraly]

As suggested in a recent tech meeting, we should add extension templates for the v1 networks and network packages API, to enable users to add new networks and get them tested immediately in the CI.

This could be modelled upon the extension template in sktime
https://github.com/sktime/sktime/tree/main/extension_templates

although, unlike in sktime, adding a neural network will mean adding at least two files - a lightning model, and a package class with tags. Thus, the extension template should probably be not a single file, but a folder, which mimicks the structure of the folder within the models module.

[fkiraly]

I think this is a good first issue, though I recommend getting in touch on dev-chat (discord) for anyone who has not been somewhat following the recent rework towards version 2.
https://discord.com/invite/54ACzaFsn7

[fkiraly]

One issue I just encountered, when is get_train_test_params necessary, when is get_base_test_params necessary instead? This seems to depend on which tags are populated, e.g., "info:pred_type".

We should simplify the extension contract here, this is hard to understand.
FYI @phoeenniixx, @PranavBhatP, @agobbifbk, opinions appreciated!

[phoeenniixx]

The get_base_test_params are used to generate "Base-params" that would be paired with all the losses the model is compatible with. The tags like "info:pred_type" are used to get these losses. To see if the model could do classification tasks as well or not.
So, the tags just tell us the losses models are compatible with, and the base-params would work for all these losses. This saves the effort of adding loss to each param as each param is tested against ever compatible loss, to make sure there are no "special errors" like the LogNormalDistributionLoss error we encountered as mentioned in #1921

[phoeenniixx]

The main downside to this structure, that I've been thinking about for a while now, is that as we add more models, the number of tests will increase very fastly (each model would have base_params*all_compatible_losses numbers of fixtures). We should think of a better way, like do this "all loss tests" for only the file changed, and for rest of the models, remaining unchanged, should get tested using something like get_train_test_params, that would have a smaller fixture pool with losses - not all params being tested with each loss...

[MAVRICK-1]

@fkiraly can i work on this ?

[phoeenniixx]

Yes, thank you!
You can go ahead and give it a try with v1 for now, since v2 isn’t quite mature yet.
As this isn’t a good first issue, could you please outline your approach or plan for tackling this issue?
It would be helpful to understand how you’re thinking of proceeding.