Using pip-compile with pyproject.toml / TIL / From the Python and Django developers at Lincoln Loop

You can define requirements for pip-compile (part of pip-tools) in pyproject.toml. I hate having a bunch of one-off files in the root of my repo, so stuffing everything into a pyproject.toml (similar to how npm uses package.json) is good for me. Here's an example pyproject.toml:

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "ll"
version = "4.0"
dependencies = [
  "django",
  "dj-database-url",
  "django-alive",
  "django-amazon-ses",
  "django-storages[boto3]",
  "django-webserver[gunicorn]",
  "goodconf[yaml]",
  "python-json-logger",
  "readable-log-formatter",
  "sentry-sdk",
  "whitenoise",
  "wsgi-basic-auth",
  # Github Hotfix can be removed once 3.14.0 is released.
  # https://github.com/encode/django-rest-framework/pull/8599
  "djangorestframework @ https://github.com/encode/django-rest-framework/archive/df584350b4f77143d84615f05000f71408aec9c0.zip",
]

[project.optional-dependencies]
dev = [
  "black",
  "autoflake",
  "isort",
  "pip-tools",
  "ipdb",
]

[project.scripts]
"manage.py" = "ll:config.django_manage"

[tool.setuptools]
packages = {"find" = {}}
include-package-data = true

# ...

Generate a pinned set of dependencies via:

pip-compile --generate-hashes --output-file requirements.txt pyproject.toml

Generate your optional dev dependencies via:

pip-compile --generate-hashes --output-file requirements-dev.txt --extra dev pyproject.toml

There isn't a great workflow for preventing drift between the two files at the moment. Discussion for that is in jazzband/pip-tools#1092.