Integrate `embedme` in our Markdown documentation for keeping documention up-to-date

[koslambrou]

Many parts of our documentation is copy-pasted from parts of the codebase, such as the CLI commands, and the different data types involded in the Plutus scripts.

I want to use embedme (but could be any similar tool), to be able to embed external files such as code files or other markdown files, into a target markdown file.

The goal is to reduce out-of-date documentation.

Additionnally, embedme should be integrated in CI and on a pre-commit hook to ensure that the documentation is updated by the PR submitter.

IOG Jira: https://input-output.atlassian.net/browse/ETCM-6862

[jstolarek]

I haven't used anything like this before, but anything that helps us keep documentation up-to-date is a good idea.

copy-pasted from parts of the codebase, such as the CLI commands, and the different data types involded in the Plutus scripts.

Looking at the readme I can see that embedme allows to insert a range of lines from the source file into a documentation file. Doesn't that put us into a risk of having garbage in the documenatation, and potentially increase the maintenance burden? Say, if I add documentation at the beginning of a file and all the lines shift by 10, won't that force me to find all the places in the documentation where we've been inserting from that file and update them? And if someone forgets to do that (or fails to update all the references) then we'll get garbage in the docs? I can see that there's a --verify flag but I don't understand how it prevents the scenario outlined above.

Also, please create a corresponding Jira ticket!

[koslambrou]

Doesn't that put us into a risk of having garbage in the documenatation, and potentially increase the maintenance burden?

That's a fair point, and it probably does increase the maintenance burden when linking to frequently changed files. We can probably use techniques to reduce the chance triggering a file change by isolating the parts we refer to in the documentation (such as putting datatypes in separate files). But of course, that's not perfect.

embedme doesn't support it yet, but there was an issue about the ability to specify tags in the source files to target the code snippets.

And if someone forgets to do that (or fails to update all the references) then we'll get garbage in the docs? I can see that there's a --verify flag but I don't understand how it prevents the scenario outlined above.

Yes, we'll get garbage in the docs, but a PR will show this garbage to the reviewer and even the contributor. How? By using a Git pre-commit hook for auto-applying embedme and using --verify in CI to not allow merging if embedme provides a different output.