Experimental Typst backend

[lucifer1004]

I just wrote a Typst backend for Documenter.jl and managed to build the latest Julia documentation.

https://drive.google.com/file/d/1IlqEI5b4Qc0CjuJ5u0l0Ma_MymRpWcrD/view?usp=sharing

Footnotes are not supported yet, while math equations are kept as is since Typst uses a different grammar and there is no translator available at the moment.

A big advantage is the compilation speed. On my computer, the whole documentation can be compiled within ~13 min. And I believe there is still much room for improvement.

Official Typst does not support syntax highlighting for Julia yet, I have made a PR, but it is currently blocked by upstream packages. There are still some issues with syntax highlighting, e.g., in some cases, all code after # are taken as comments.

[mortenpi]

This looks interesting! I'll try to find some time to play with it properly. Would be awesome to also get more eyes on this, to see what people this about using this vs. the existing PDF/LaTeX backend.

I do have some immediate thoughts about merging it here though:

• I would prefer not to merge this into Documenter proper, unless we're at the point where it would deprecate the LaTeX backend for PDF generation (simply a matter of maintenance burden). I think that would require minimally solving the issues you listed in the OP, but also getting buy-in from the community to use this for the Julia manual (I, personally, don't have a strong opinion about which would be preferred).

• As such, it's perfectly fine to maintain this as a PR here if you wish, but it might be worth developing it in a separate repository for the time being, as a Documenter plugin package. This would also be a good opportunity to hash out the internal AST APIs that other backend implementation could rely on (together with the epub backend).

[lucifer1004]

I was not aware of the choice of Documenter plugins! That would be much better. Actually, I think the current HTML and LaTeX backend can also be made plugins.

[mortenpi]

I was not aware of the choice of Documenter plugins!

Just to maybe clarify -- Documenter plugins are not a thing that strictly exist as a feature (yet; I would like it to be an official feature though). So, right now, one has to dive into Documenter internals to make things work. However, I would like those necessary internal hooks to be documented and part of the public, semver-ed API, so that external plugin packages could depend on safely and cleanly.

I think the current HTML and LaTeX backend can also be made plugins

I don't think we want HTMLWriter and LaTeXWriter to be external plugin packages --- they're core features supported here. But, the APIs that they use to function should indeed all be exposed as public APIs that other output formats could depend on. In that sense, I agree that they should not be special in any way.

The process for getting us those APIs will be a bit laborious though. I don't really want to declare the current state of the internal APIs be public. I think it should all be reviewed and documented step-by-step, and refactored as needed, to actually get a good API. But having existing extension packages, like the typst backend, is a great first step in clarifying the requirements of the internal API.

A couple of examples of packages that extend Documenter in some ways that may be useful for you though:

• https://github.com/laborg/DocumenterEpub.jl
• https://github.com/ali-ramadhan/DocumenterCitations.jl
• https://github.com/JuliaDocs/DocumenterMarkdown.jl

[KronosTheLate]

I just want to express my preferance for typst, mainly due to the fact that unrendered typst math is much more readable than latex, meaning that it is a better choice for docstrings. The faster compile-times should also help keep the "flow" in "workflow" if someone is using low-power servers to check if the docs are buildt correctly.

Typst recently got footenote-support, and support for Julia syntax highlighting. Note that the footnotes can be a little jenky, as tracked in this issue.

[tecosaur]

I just want to express my preferance for typst, mainly due to the fact that unrendered typst math is much more readable than latex

My 2c is just that LaTeX should be unicode-rendered in the REPL, it's possible, I've got a local package that does it.

[tpapp]

Regarding

I would prefer not to merge this into Documenter proper, unless we're at the point where it would deprecate the LaTeX backend for PDF generation (simply a matter of maintenance burden).

I find typst technically superior and much more readable in the source code (no more forest of \s, a double win since in Julia docstrings we need to escape them), so I think it is a clear win, but at the same time it is unreasonable to expect people to rewrite existing documentation that just works.

This means that in the long run we need to find a way for the two to coexist. Since mixing LaTeX and Typst syntax in the same document is perhaps an uncommon use case, maybe it would be best to be able to select the preferred syntax as a global option. This means that whatever is inside the $$s gets sent to the appropriate backend.

[kapple19]

I would imagine eventually the docs are written in Typst using a special Julia-provided Typst template?

[andreeco]

Being able to use Typst instead of LaTeX would be a real improvement for me.

[asinghvi17]

FWIW https://typst.app/universe/package/mitex/ exists which seems to be able to translate latex to typst inline inside a typst document, so math blocks might be able to continue to work?

and maybe we declare

```math typst
...
```

to be special syntax for typst?