Compiler shards represented as `HostBuildDependency` objects

[staticfloat]

What if we went totally crazy and represented things like Glibc_jll, GCC_jll, LLVM_jll, etc… all as HostBuildDependencies and choose_shards() just gave a bunch of special HostBuildDependency objects back? We’d of course need to customize things a bit (we might want to have finer control over where these things get unpacked, and we’d have to integrate it into generate_compiler_wrappers()) but I can see a few advantages:

• unification of setup: right now we have so many different ways of bringing things into the build environment: Dependency(), apk add, compilers=[:c, :rust] and now HostBuildDependency(). We could conceivably get rid of lots of these things by unifying everything behind HostBuildDependency().

• Build manifests. I want to embed a .toml file that contains every piece that was used to build a JLL. Explicitly declaring what JLLs were used as the build tools would make this really granular.

• Really easy testing of new toolchains. Trying out a new compiler version could be as easy as just overriding the default version that BBB adds to your set with something else. I’m not sure what the API here should be, but I’m thinking something like:

dependencies = Dependency[
    CompilerSet([:c, :rust])...,
    # The rest of your compilers
]

We can have backwards-compatibility and add the C compilers by default if no HostBuildDependencies are specified.

[vchuravy]

We need a "crazy good idea" label.

[giordano]

I claim the ownership of the "good" side of the idea, Elliot added the "crazy" bit

[vchuravy]

and I take the blame for wanting HostDependencies.

[arhik]

Cool.

[giordano]

One thing that I'd like to mention is that the version of the compiler isn't always the same in a build recipe, think of all the expand_* functions. If we're going to include the version of the compiler in the dependency object, we should take this into account. But perhaps the provided version can be the equivalent of preferred_*_version and resolved at runtime to a concrete version, depending on all constraints (e.g., the target platform).