docs: add ADR-0011 (KubeRay), ADR-0012 (uv), update architecture docs
This commit is contained in:
195
decisions/0012-use-uv-for-python-development.md
Normal file
195
decisions/0012-use-uv-for-python-development.md
Normal file
@@ -0,0 +1,195 @@
|
||||
# Use uv for Python Development, pip for Docker Builds
|
||||
|
||||
* Status: accepted
|
||||
* Date: 2026-02-02
|
||||
* Deciders: Billy Davies
|
||||
* Technical Story: Standardizing Python package management across development and production
|
||||
|
||||
## Context and Problem Statement
|
||||
|
||||
Our Python projects use a mix of `requirements.txt` and `pyproject.toml` for dependency management. Local development with `pip` is slow, and we need a consistent approach across all repositories while maintaining reproducible Docker builds.
|
||||
|
||||
## Decision Drivers
|
||||
|
||||
* Fast local development iteration
|
||||
* Reproducible production builds
|
||||
* Modern Python packaging standards (PEP 517/518/621)
|
||||
* Lock file support for deterministic installs
|
||||
* Compatibility with existing CI/CD pipelines
|
||||
|
||||
## Considered Options
|
||||
|
||||
* pip only (traditional)
|
||||
* Poetry
|
||||
* PDM
|
||||
* uv (by Astral)
|
||||
* uv for development, pip for Docker
|
||||
|
||||
## Decision Outcome
|
||||
|
||||
Chosen option: "uv for development, pip for Docker", because uv provides extremely fast package resolution and installation for local development (10-100x faster than pip), while pip in Docker ensures maximum compatibility and reproducibility without requiring uv to be installed in production images.
|
||||
|
||||
### Positive Consequences
|
||||
|
||||
* 10-100x faster package installs during development
|
||||
* `uv.lock` provides deterministic dependency resolution
|
||||
* `pyproject.toml` is the modern Python standard (PEP 621)
|
||||
* Docker builds remain simple with standard pip
|
||||
* `uv pip compile` can generate `requirements.txt` from `pyproject.toml`
|
||||
* No uv runtime dependency in production containers
|
||||
|
||||
### Negative Consequences
|
||||
|
||||
* Two tools to maintain (uv locally, pip in Docker)
|
||||
* Team must install uv for local development
|
||||
* Lock file must be kept in sync with pyproject.toml
|
||||
|
||||
## Pros and Cons of the Options
|
||||
|
||||
### pip only (traditional)
|
||||
|
||||
* Good, because universal compatibility
|
||||
* Good, because no additional tools
|
||||
* Bad, because slow resolution and installation
|
||||
* Bad, because no built-in lock file
|
||||
* Bad, because `requirements.txt` lacks metadata
|
||||
|
||||
### Poetry
|
||||
|
||||
* Good, because mature ecosystem
|
||||
* Good, because lock file support
|
||||
* Good, because virtual environment management
|
||||
* Bad, because slower than uv
|
||||
* Bad, because non-standard `pyproject.toml` sections
|
||||
* Bad, because complex dependency resolver
|
||||
|
||||
### PDM
|
||||
|
||||
* Good, because PEP 621 compliant
|
||||
* Good, because lock file support
|
||||
* Good, because fast resolver
|
||||
* Bad, because less adoption than Poetry
|
||||
* Bad, because still slower than uv
|
||||
|
||||
### uv (by Astral)
|
||||
|
||||
* Good, because 10-100x faster than pip
|
||||
* Good, because drop-in pip replacement
|
||||
* Good, because supports PEP 621 pyproject.toml
|
||||
* Good, because uv.lock for deterministic builds
|
||||
* Good, because from the creators of Ruff
|
||||
* Bad, because newer tool (less mature)
|
||||
* Bad, because requires installation
|
||||
|
||||
### uv for development, pip for Docker (Chosen)
|
||||
|
||||
* Good, because fast local development
|
||||
* Good, because simple Docker builds
|
||||
* Good, because no uv in production images
|
||||
* Good, because pip compatibility maintained
|
||||
* Bad, because two tools in workflow
|
||||
* Bad, because must sync lock file
|
||||
|
||||
## Implementation
|
||||
|
||||
### Local Development Setup
|
||||
|
||||
```bash
|
||||
# Install uv (one-time)
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
|
||||
# Create virtual environment and install dependencies
|
||||
uv venv
|
||||
source .venv/bin/activate
|
||||
uv pip install -e ".[dev]"
|
||||
|
||||
# Or use uv sync with lock file
|
||||
uv sync
|
||||
```
|
||||
|
||||
### Project Structure
|
||||
|
||||
```
|
||||
my-handler/
|
||||
├── pyproject.toml # PEP 621 project metadata and dependencies
|
||||
├── uv.lock # Deterministic lock file (committed)
|
||||
├── requirements.txt # Generated from uv.lock for Docker (optional)
|
||||
├── src/
|
||||
│ └── my_handler/
|
||||
└── tests/
|
||||
```
|
||||
|
||||
### pyproject.toml Example
|
||||
|
||||
```toml
|
||||
[project]
|
||||
name = "my-handler"
|
||||
version = "1.0.0"
|
||||
requires-python = ">=3.11"
|
||||
dependencies = [
|
||||
"handler-base @ git+https://git.daviestechlabs.io/daviestechlabs/handler-base.git",
|
||||
"httpx>=0.27.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=8.0.0",
|
||||
"pytest-asyncio>=0.23.0",
|
||||
"ruff>=0.1.0",
|
||||
]
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
```
|
||||
|
||||
### Dockerfile Pattern
|
||||
|
||||
The Dockerfile uses uv for speed but installs via pip-compatible interface:
|
||||
|
||||
```dockerfile
|
||||
FROM python:3.13-slim
|
||||
|
||||
# Copy uv for fast installs (optional - can use pip directly)
|
||||
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
|
||||
|
||||
# Install from pyproject.toml
|
||||
COPY pyproject.toml ./
|
||||
RUN uv pip install --system --no-cache .
|
||||
|
||||
# OR for maximum reproducibility, use requirements.txt
|
||||
COPY requirements.txt ./
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
```
|
||||
|
||||
### Generating requirements.txt from uv.lock
|
||||
|
||||
```bash
|
||||
# Generate pinned requirements from lock file
|
||||
uv pip compile pyproject.toml -o requirements.txt
|
||||
|
||||
# Or export from lock
|
||||
uv export --format requirements-txt > requirements.txt
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
1. **Add dependency**: Edit `pyproject.toml`
|
||||
2. **Update lock**: Run `uv lock`
|
||||
3. **Install locally**: Run `uv sync`
|
||||
4. **For Docker**: Optionally generate `requirements.txt` or use `uv pip install` in Dockerfile
|
||||
5. **Commit**: Both `pyproject.toml` and `uv.lock`
|
||||
|
||||
## Migration Path
|
||||
|
||||
1. Create `pyproject.toml` from existing `requirements.txt`
|
||||
2. Run `uv lock` to generate `uv.lock`
|
||||
3. Update Dockerfile to use pyproject.toml
|
||||
4. Delete `requirements.txt` (or keep as generated artifact)
|
||||
|
||||
## Links
|
||||
|
||||
* [uv Documentation](https://docs.astral.sh/uv/)
|
||||
* [PEP 621 - Project Metadata](https://peps.python.org/pep-0621/)
|
||||
* [Astral (uv creators)](https://astral.sh/)
|
||||
* Related: handler-base already uses uv in Dockerfile
|
||||
Reference in New Issue
Block a user