Add poetry as an env manager (#460)

* add poetry as an env manager

* Bump version

* Add checklist for release process

* add poetry build system

* Tweak poetry help text to print properly

Author: pjbull

.github/workflows/tests.yml

@@ -81,6 +81,11 @@ jobs:
           cache: false
           run-install: false
 
+      - name: Set up Poetry
+        uses: abatilo/actions-poetry@v3
+        with:
+          poetry-version: latest
+
       - name: Cache conda packages
         uses: actions/cache@v4
         env:
@@ -125,6 +130,8 @@ jobs:
           virtualenv --version
           which pixi
           pixi --version
+          which poetry
+          poetry --version
           which python
           python --version
           python -c "import platform; print(f'Python architecture: {platform.architecture()}')"

HISTORY.md

@@ -1,5 +1,10 @@
 # cookiecutter-data-science Changelog
 
+## v2.3.0 (2025-07-23)
+
+ - Added `pixi` as a new environment manager option (supports `pyproject.toml` and `pixi.toml`). (PR [#459](https://github.com/drivendataorg/cookiecutter-data-science/pull/459), Issue [#406](https://github.com/drivendataorg/cookiecutter-data-science/issues/406))
+ - Added `poetry` as a new environment manager option (supports `pyproject.toml`). (PR [#460](https://github.com/drivendataorg/cookiecutter-data-science/pull/460), Issue [#374](https://github.com/drivendataorg/cookiecutter-data-science/issues/374))
+
 ## v2.2.0 (2025-03-23)
 
 - Added `pyproject.toml` as a dependencies file format option. (PR [#436](https://github.com/drivendataorg/cookiecutter-data-science/pull/436))

RELEASING.md

@@ -8,4 +8,9 @@ To give the utility and the template a bit more stability, PR [#336](https://git
 
 ## Issuing a new release
 
+ - [ ] Update version in `pyproject.toml` file
+ - [ ] Ensure `HISTORY.md` is up to date with the changes in this release and add version info and release date
+ - [ ] Create new release on the GitHub [releases page](https://github.com/drivendataorg/cookiecutter-data-science/releases) with the tag `vMAJOR.MINOR.PATCH` (e.g., `v2.3.0`) and the contents of the relevant `HISTORY.md` section as the release notes.
+ - [ ] Confirm release action runs successfully and the new release is available on [PyPI](https://pypi.org/project/cookiecutter-data-science/).
+
 `ccds` uses [semantic versioning](https://semver.org/). When issuing a new release, **ensure that your release version tag has the format `vMAJOR.MINOR.PATCH`. The `v` prefix is important because the utility will look for the tag with that name to download by default.

ccds-help.json

@@ -154,6 +154,13 @@
                     "more_information": "[Docs](https://pixi.sh/)"
                 } 
             },
+            {
+                "choice": "poetry",
+                "help": {
+                    "description": "A dependency management and packaging tool for Python with lock files for reproducible environments. Uses pyproject.toml for configuration. Requires poetry to be installed as a system binary.",
+                    "more_information": "[Docs](https://python-poetry.org/docs/)"
+                } 
+            },
             {
                 "choice": "none",
                 "help": {

ccds.json

@@ -17,6 +17,7 @@
         "pipenv",
         "uv",
         "pixi",
+        "poetry",
         "none"
     ],
     "dependency_file": [

ccds/hook_utils/dependencies.py

@@ -178,6 +178,13 @@ def write_dependencies(
             doc["project"].add("dependencies", sorted(packages))
             doc["project"]["dependencies"].multiline(True)
 
+            # poetry uses standard project dependencies, but we should use its build system
+            if environment_manager == "poetry":
+                # Update build system to use poetry-core
+                doc["build-system"] = tomlkit.table()
+                doc["build-system"]["requires"] = ["poetry-core>=2.0.0,<3.0.0"]
+                doc["build-system"]["build-backend"] = "poetry.core.masonry.api"
+
         with open(dependencies, "w") as f:
             f.write(tomlkit.dumps(doc))
 

dev-requirements.txt

@@ -12,6 +12,7 @@ mkdocs-gen-files
 mkdocs-include-markdown-plugin
 pexpect
 pipenv
+poetry
 pytest
 termynal
 twine

pyproject.toml

@@ -7,7 +7,7 @@ name = "ccds"
 
 [project]
 name = "cookiecutter-data-science"
-version = "2.2.0"
+version = "2.3.0"
 description = "A logical, reasonably standardized but flexible project structure for doing and sharing data science work."
 authors = [
   { name = "DrivenData", email = "info@drivendata.org" },

tests/conftest.py

@@ -62,6 +62,11 @@ def _is_valid(config):
             config["dependency_file"] not in ["pixi.toml", "pyproject.toml"]
         ):
             return False
+        # poetry only supports pyproject.toml
+        if (config["environment_manager"] == "poetry") and (
+            config["dependency_file"] != "pyproject.toml"
+        ):
+            return False
         return True
 
     # remove invalid configs

tests/poetry_harness.sh

@@ -0,0 +1,60 @@
+#!/bin/bash
+set -ex
+
+PROJECT_NAME=$(basename $1)
+CCDS_ROOT=$(dirname $0)
+MODULE_NAME=$2
+
+# Check if poetry is installed
+if ! command -v poetry &> /dev/null; then
+    echo "poetry is not installed. Please install poetry first:"
+    echo "  curl -sSL https://install.python-poetry.org | python3 -"
+    echo "  Or visit: https://python-poetry.org/docs/#installation"
+    exit 1
+fi
+
+# Configure exit / teardown behavior
+function finish {
+    # Cleanup poetry environment if it exists
+    if poetry env list | grep -q "$(basename $(pwd))"; then
+        poetry env remove --all
+    fi
+}
+trap finish EXIT
+
+# Source the steps in the test
+source $CCDS_ROOT/test_functions.sh
+
+cd $1
+
+# Setup and run tests
+make
+make create_environment
+make requirements
+
+# Run poetry-specific tests with simpler commands
+poetry run python --version
+echo "Testing basic import..."
+poetry run python -c "import $MODULE_NAME"
+
+# Test config importable if scaffolded  
+if [ -f "$MODULE_NAME/config.py" ]; then
+    echo "Testing config import..."
+    poetry run python -c "from $MODULE_NAME import config"
+fi
+
+# Run linting and formatting through poetry
+poetry run make lint
+poetry run make format
+
+# Custom poetry test function to avoid issues with test_functions.sh
+# Check that python is available in poetry environment
+echo "Testing poetry python availability..."
+if poetry run python -c "import sys" > /dev/null 2>&1; then
+    echo "Python is available in poetry environment"
+else
+    echo "ERROR: Python not available in poetry environment" 
+    exit 1
+fi
+
+echo "All done!"