Optimize LALRPOP at build time transitively

[yannham]

Hi folks,

It's more of a cargo question than a pure LALRPOP one, but I expect that LALRPOP users might be particularly at risk of the issue I encounter, so I wondered if the community had ideas or work-around.

The situation is the following: for the Nickel configuration language, we use LALRPOP as the parser. We have a substantial grammar file, which takes ages to build using the default settings: by default, cargo optimizes build dependencies for compilation time so it doesn't apply many optimizations. The classical solution to this problem, which we implemented, is simply to tell cargo to optimize LALRPOP even a build time, because the whole compilation pipeline is overall quite faster:

# In the workspace's Cargo.toml
[profile.dev.package.lalrpop]
opt-level = 3

[profile.release.package.lalrpop]
opt-level = 3

So far so good. Unfortunately, those overrides aren't transitive, as noted in tweag/nickel#2123. As Nickel is also intended to be used as a library, this is an issue. Some ideas are proposed in that issue, such as checking in the generated grammar (but it's annoying since now you have to make sure it's up to date).

I wondered if other LALRPOP users faced similar issues, and if there is a good solution here.

[dburgener]

Some ideas are proposed in that issue, such as checking in the generated grammar (but it's annoying since now you have to make sure it's up to date).

I don't have any suggestions on the main topic unfortunately, but I did want to point out that generated lalrpop files store a sha3_256 sum of the .lalrpop file they were generated from. So it should be pretty straightforward to include a CI check that the .rs doesn't diverge from the .lalrpop if you choose to go the route of checking in the generated parser.

[yannham]

Thanks, I didn't know and it does make things better when pre-checking the generated file!

[yannham]

For the record, after some discussion on our repo about the cons of checking in unconditionally a file of 22MB (the generated parser), we ended up with the following solution: in our release script, we pack the generated LALRPOP parser together with the repo, so that it ends up on crates.io, but ditch it otherwise, so that it never ends up checked in the GH repo. We updated the build script to use the pre-generated file when it's there (because we only do that "internally" for the release, we don't bother checking that it's up to date, but that would be easy given that the SHA3 sum is included in the generated file).

We'll see how it works for the next release, but this looks like a reasonable solution and a rather small change.

[yannham]

Implemented in tweag/nickel#2145, if anyone is looking for inspiration.