Description
I propose adopting pip-tools as a way to more flexibly and reliably manage the contents in requirements.txt.
Current Situation
Right now the requirements are managed in a bit of a hard to use manner and this is likely a hinderance to folks attempting to work on native Mac support (See #154 and #184).
The codebase today does not have a mechanism to define and or test on different platforms, effectively leaving it CUDA-only. There is also not a straightforward way to define tests and requirements against different python or CUDA versions.
The dependencies are managed between the ./requirements/p2.txt (I have a proposed simplification at #193) and along the path to running at test as defined in https://github.com/Stability-AI/generative-models/blob/main/pyproject.toml#L45
Rationale
Only specifying one pip-freeze produced requirements file has a number of downsides:
- It's often a post-hoc description of some system being in some working state, instead of being a source-controlled definition of requirements.
- Adopting pip-compile means that version resolution becomes a development-time step instead of a build or installation time step.
- pip-compile grants us an easier to manage and more reproducible description format while enabling an easy path to targeting multi-platform support.
Possible Downsides and Tradeoffs
- pip-compile is another tool folks have to learn or cope with, this might mitigated by documentation and scripting/tooling to assist.
Alternatives Considered
- Current approach.
- Adopting poetry.