Change workflow to use JobSets

[VIPQualityPost]

• Added workflow for ERC/DRC and production documentation for AYAB Shield
• Changed workflow from huge pile of kicad-cli to using jobsets so that all output settings are managed between all AYAB KiCad based projects
• Merges in reference material (schematic pdf, assembly pdf, renders, step model of PCBA) in after a PR is merged into main, keeping live repo up to date
• Add README for AYAB Shield with nice workflow badges.

[VIPQualityPost]

Wow, that looks like a mess with all my commits - I thought it would squash all of them!
I tested this on my local fork, so I can confirm everything is working already - see my ayab-hardware main branch if you want to see what it looks like.

[jonathanperret]

Wow, that looks like a mess with all my commits - I thought it would squash all of them!

In case you don't know, you can squash your branch's commits by running git rebase -i main, replace some or all ps by ss and after the rebase git push -f. Might be a good idea indeed, looking at some of those commit titles 😅

[VIPQualityPost]

@jonathanperret Is the only remaining item on this that we should remove the push to repo as part of the action and rely on contributors to include up-to-date assets when opening PRs that touch hardware?

[dl1com]

@jonathanperret Is the only remaining item on this that we should remove the push to repo as part of the action and rely on contributors to include up-to-date assets when opening PRs that touch hardware?

We could handle it similar to the automatic builds we do for the Desktop now - so making the generated artifacts easily available for test by make them accessible in the action, and attach them to a github release in case of a release tag is pushed.
So no need to actually store generated information into the repo, I guess?

[VIPQualityPost]

@jonathanperret Is the only remaining item on this that we should remove the push to repo as part of the action and rely on contributors to include up-to-date assets when opening PRs that touch hardware?

We could handle it similar to the automatic builds we do for the Desktop now - so making the generated artifacts easily available for test by make them accessible in the action, and attach them to a github release in case of a release tag is pushed. So no need to actually store generated information into the repo, I guess?

I agree, it just becomes annoying that it's not easy to point people to a live file of the schematic for viewing. But maybe we should add a permalink in the readme pointing to KiCanvas for live view? That may be a very nice solution...
Also: The documentation workflow produces two artifacts, one with production files, and the other with schematic, renders, 3D model etc.

[VIPQualityPost]

Also so these issues are linked: This would address #50

[VIPQualityPost]

Okay, I think that we're all on the same page about this PR now?

[jonathanperret]

@jonathanperret Is the only remaining item on this that we should remove the push to repo as part of the action and rely on contributors to include up-to-date assets when opening PRs that touch hardware?

We could handle it similar to the automatic builds we do for the Desktop now - so making the generated artifacts easily available for test by make them accessible in the action, and attach them to a github release in case of a release tag is pushed. So no need to actually store generated information into the repo, I guess?

I agree, it just becomes annoying that it's not easy to point people to a live file of the schematic for viewing. But maybe we should add a permalink in the readme pointing to KiCanvas for live view? That may be a very nice solution... Also: The documentation workflow produces two artifacts, one with production files, and the other with schematic, renders, 3D model etc.

I agree that having up-to-date output files in the repo is actually pretty convenient for this kind of project.

Being able to point people to a schematic (of any particular revision, to boot) is one benefit; another one is having GitHub show diffs for e.g. board images, like what I did manually in #48 (comment) which can be very useful as the diffs of KiCad files aren't generally helpful.

So while I'm generally not a fan of committing generated files, I think it may be worthwhile in this case.

Now, one option is to do nothing workflow-wise and as you said @VIPQualityPost to rely on contributors to run the asset-generating actions. Ideally KiCad would have an option to auto-update the outputs on save but it looks like this doesn't exist yet. We could define a pre-commit hook but that would still require people to install it and run Git from the command-line.

An improvement over that is to have a GitHub Actions workflow that adds a file-generation commit onto any branch that has changes to the KiCad files — which is almost what you have here except that it currently pushes only to main, even when triggered by a pull_request event, which won't work.

I therefore suggest:

• removing the pull_request trigger for any workflow that pushes commits — it won't work for PRs from forks, and for internal PRs the push trigger is a better option as it is associated with a branch, unlike the pull_request trigger which runs on code from an ephemeral merge commit.
• for those workflows, removing the filter that makes the push trigger only run on the main branch — this will let the output-generating jobs run at least on internal PR branches. For external PRs, contributors can be instructed to enable those workflows on their forks (workflows are disabled by default on forks) so that their source branches can be automatically updated in their own repo. A PR made from a fork where those jobs don't run would be easily recognized as missing updates to the generated files.
• workflows that do checks (DRC/ERC) should not commit anything and instead report success or failure and attach their detailed output as artifacts. Presumably only commits with an empty DRC/ERC report would be allowed on main? Those workflows make sense to be run on a pull_request trigger. The current Design check & documentation build seem to conform to this already 🎉 (they don't seem to do any documentation building though, maybe a naming issue there?).
• when triggering based on changed paths, you probably want to include ayab-library/** as well since a change there can impact both projects.

I hope this doesn't sound like too much rework, but I understand if it does. Any version of this PR is already a great step forward (thank you!) so feel free to take or leave what you want from these remarks — and there's always the possibility to improve things incrementally in the future.

[jonathanperret]

Okay, I think that we're all on the same page about this PR now?

Heh, looks like I took so long to write that wall of text that you went ahead and picked an option. Which is great, let's move forward!

Not having the generated outputs in the repo is certainly a valid option. Maybe just look into triggering on ayab-library/** changes?