Tracking issue: easily reinstallable boot libraries

[TeofilC]

The status quo is that boot libraries in general and certain boot¹ libraries in particular are handled by cabal-install in a special way, which means that they cannot (easily) be re-installed. You are stuck with whatever version came with the version of GHC. This is undesirable.

There are multiple reasons why cabal-install might not allow certain packages to be reinstalled.

• Some packages contain wired-in identifiers that are hard-coded into the compiler.
• Some packages are dependencies of the ghc library, and we often want to force the use of the ghc:lib component that came bundled with the compiler. This only applies to build plans that include this library. But common tools like doctest have ghc as a dependency.

Recently GHC developers have been moving moving wired-in identifiers into ghc-internal, so this restriction is getting relaxed. But there is still some work to do on the cabal-install side to make it possible to reinstall more libraries.

In particular, it would be nice if we could easily reinstall base and template-haskell. Neither of these will be wired-in with GHC-9.12.

See also some of the points @mpickering made on this MR: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13297

Here are a list of issues that are currently blocking reinstalling these (in a user friendly way):

• Amend non-reinstallable packages list #10087
• Special treatment of pre-installed packages by the solver #9669
• Something like RFC: design for better GHC plugin support in cabal #7901 to allow for better support for a separate build plan for plugin/staged dependencies.

Please feel free to edit this. I wanted to collect these in one place because I don't think it's super clear what's blocking us otherwise.

Footnotes

1. a boot library is any library that came packaged in the global package database supplied with a GHC install. ↩

[tdammers]

IIUC, #9669 would not be a hard blocker for this. The "special treatment" mentioned there is about Cabal preferring already-installed versions of (boot) libraries, but if boot librariers are reinstallable, and a project's version bounds on a boot library conflicts with the already-installed one, Cabal should still pick a different one, just like it does with other (non-boot) libraries.

[TeofilC]

IIUC, #9669 would not be a hard blocker for this. The "special treatment" mentioned there is about Cabal preferring already-installed versions of (boot) libraries, but if boot librariers are reinstallable, and a project's version bounds on a boot library conflicts with the already-installed one, Cabal should still pick a different one, just like it does with other (non-boot) libraries.

Yes that's good to emphasize. This is more about usability and making things easier for users.

It's much nice to be able to say: if you want to use the latest boot libs, then pass this flag to cabal and make sure your bounds allow them.
Rather than: if you want to use the latest boot libs, then add constraints that disallow any version other than the latest one.

[tdammers]

Makes sense, yes.

Other question: upgraded/reinstallable boot libraries are obviously not supported by older GHC versions, and Cabal needs to know about that.

#10087 is about exposing wired-in unit IDs from GHC, and using that information in Cabal to determine which (boot) libraries to use, but what does it do with older compilers, which don't expose this information? Fall back to the old behavior of assuming that all boot libraries must be the installed versions? Or would it assume that all boot libraries are freely reinstallable (which is obviously wrong)?

[geekosaur]

That's not entirely true. The biggest problem is obviously base, and indeed older ghcs won't be able to reinstall it. But there are a bunch of bootlibs that cabal currently doesn't want you to upgrade that are perfectly upgradeable as long as you aren't linking against ghc-lib (text, containers, parsec, etc.) My understanding was that part of this was to allow those packages to be upgraded more easily (currently you need to beat cabal over the head to make it upgrade them)? Also there will still be bootlibs that can't be upgraded at all (ghc-prim, ghc-base, template-haskell, and of course ghc).

[TeofilC]

Fall back to the old behavior of assuming that all boot libraries must be the installed versions?

Note that not all boot libraries are on this list:

cabal/cabal-install/src/Distribution/Client/Dependency.hs

Lines 447 to 471 in dda541c

	-- | The set of non-reinstallable packages includes those which cannot be
	-- rebuilt using a GHC installation and Hackage-published source distribution.
	-- There are a few reasons why this might be true:
	--
	-- * the package overrides its unit ID (e.g. with ghc's @-this-unit-id@ flag),
	-- which can result in multiple indistinguishable packages (having potentially
	-- different ABIs) with the same unit ID.
	--
	-- * the package contains definitions of wired-in declarations which tie
	-- it to a particular compiler (e.g. we can't build link against
	-- @base-4.18.0.0@ using GHC 9.6.1).
	--
	-- * the package does not have a complete (that is, buildable) source distribution.
	-- For instance, some packages provided by GHC rely on files outside of the
	-- source tree generated by GHC's build system.
	nonReinstallablePackages :: [PackageName]
	nonReinstallablePackages =
	[ mkPackageName "base"
	, mkPackageName "ghc-bignum"
	, mkPackageName "ghc-prim"
	, mkPackageName "ghc"
	, mkPackageName "integer-gmp"
	, mkPackageName "integer-simple"
	, mkPackageName "template-haskell"
	]

But yeah my pitch is to just keep the behaviour the same for old GHCs, since we can't make guarantees there, and just be more permissive with GHC-9.14+.

My understanding was that part of this was to allow those packages to be upgraded more easily (currently you need to beat cabal over the head to make it upgrade them)?

Yeah I think that's #9669

Also there will still be bootlibs that can't be upgraded at all (ghc-prim, ghc-base, template-haskell, and of course ghc).

Note that this is only for /old/ GHCs. Since 9.12, the only packages with special treatment are ghc and ghc-internal.

---

In the future we could even go further and let ghc be reinstalled unless you are actually using plugins or have another reason why you actually want to link against the correct GHC. But that's future work.

[gbaz]

would suggest that we move the non-reinstallable library list responsibility to ghc, which cabal can query per-version. that way as ghc slims down the list, cabal can adapt automatically.