Add support for external builders #78
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cole-h
reviewed
May 29, 2025
edolstra
force-pushed
the
refactor-derivation-builder
branch
2 times, most recently
from
5d095b4 to
79509d0
Compare
|
TODO: come up with a better name than "external builder" since "builder" is pretty overloaded (it's both the derivation build script and now the thing that executes the derivation). Build wrappers? |
edolstra
force-pushed
the
refactor-derivation-builder
branch
2 times, most recently
from
dc19372 to
df90dea
Compare
cole-h
reviewed
Jun 10, 2025
cole-h
reviewed
Jun 13, 2025
The use of a `buildUser` is an implementation detail of some types of sandboxes that shouldn't exposed.
Having the exact same doc comments isn't very useful/maintainable.
No point in computing this lazily, since it's pretty much the first thing the DerivationBuilder does.
These are helper programs that execute derivations for specified
system types (e.g. using QEMU to emulate another system type).
To use, set `external-builders`:
external-builders = [{"systems": ["aarch64-linux"], "program": "/path/to/external-builder.py"}]
The external builder gets one command line argument, the path to a JSON file containing all necessary information about the derivation:
{
"args": [...],
"builder": "/nix/store/kwcyvgdg98n98hqapaz8sw92pc2s78x6-bash-5.2p37/bin/bash",
"env": {
"HOME": "/homeless-shelter",
...
},
"realStoreDir": "/tmp/nix/nix/store",
"storeDir": "/nix/store",
"tmpDir": "/tmp/nix-shell.dzQ2hE/nix-build-patchelf-0.14.3.drv-46/build",
"tmpDirInSandbox": "/build"
}
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
cole-h
force-pushed
the
refactor-derivation-builder
branch
from
740aecb to
5842d54
Compare
cole-h
force-pushed
the
refactor-derivation-builder
branch
from
d8bda44 to
ef4e7df
Compare
Clang doesn't like the double indent that is needed for the `if...else` that is CPP'd away. Adding braces is fine in the `if...else...` case, and fine as a naked block in the CPP'd away case, and properly-indented both ways.
I don't want to figure out how to make nlohmann treat std::optional<> the same way Rust's serde_json treats Option<> (i.e. skip it if it's not there).
…s stdin" This reverts commit d1f57c5.
Add an `external-builders` experimental feature
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
External builders are helper programs that execute derivations for specified system types (e.g. using QEMU to emulate another system type).
To use, set
external-builders:The external builder gets one command line argument, the path to a JSON file containing all necessary information about the derivation:
{ "args": [...], "builder": "/nix/store/kwcyvgdg98n98hqapaz8sw92pc2s78x6-bash-5.2p37/bin/bash", "env": { "HOME": "/homeless-shelter", ... }, "realStoreDir": "/tmp/nix/nix/store", "storeDir": "/nix/store", "tmpDir": "/tmp/nix-shell.dzQ2hE/nix-build-patchelf-0.14.3.drv-46/build", "tmpDirInSandbox": "/build" }This PR incorporates upstream NixOS#13276.
TODO: Don't run external builders as root.
Context
Add 👍 to pull requests you find important.
The Nix maintainer team uses a GitHub project board to schedule and track reviews.