fix(scripts): preserve uv extras across make dev restarts (#2754) (#2767)

`make dev` ran `uv sync` unconditionally on every restart, wiping any
optional extras the user had installed manually with
`uv sync --all-packages --extra postgres`. The Docker image-build path
already solved this via the `UV_EXTRAS` build-arg in backend/Dockerfile;
the local serve.sh path and the docker-compose-dev startup command
were the remaining outliers.

`scripts/serve.sh` now resolves extras before `uv sync`:
  1. honors `UV_EXTRAS` (parity with backend/Dockerfile and
     docker/docker-compose.yaml — no new convention introduced);
  2. falls back to parsing config.yaml — `database.backend: postgres`
     or legacy `checkpointer.type: postgres` auto-pins
     `--extra postgres`, so the common case needs zero extra config.
  3. detector stderr is no longer suppressed, so whitelist warnings or
     crashes surface to the dev terminal (review feedback).

Detection lives in `scripts/detect_uv_extras.py` (stdlib-only — has to
run before the venv exists). Extra names are validated against
`^[A-Za-z][A-Za-z0-9_-]*$` so a stray shell metacharacter in `.env`
cannot reach `uv sync` downstream (defense in depth).

`docker/docker-compose-dev.yaml`'s startup command is now extracted to
`docker/dev-entrypoint.sh` (review feedback — the inline command had
grown to a ~350-char one-liner). The script:
  - parses comma/whitespace-separated UV_EXTRAS, applying the same
    `^[A-Za-z][A-Za-z0-9_-]*$` whitelist as the local detector;
  - emits one `--extra X` flag per token, so `UV_EXTRAS=postgres,ollama`
    works in Docker dev too (harmonized with local — review feedback);
  - calls `uv sync --all-packages` (PR #2584) so workspace member
    extras (deerflow-harness's postgres extra) are installed;
  - keeps the existing self-heal `(uv sync || (recreate venv && retry))`
    branch;
  - exposes `--print-extras` for dry-run testing.

The compose file mounts the script read-only at runtime, so script
edits take effect on `make docker-restart` without an image rebuild.

The `--no-sync` alternative (a separate suggestion in the issue thread)
was considered but rejected for dev paths because it would drop the
self-heal branch and the auto-pickup of new pyproject deps. `--no-sync`
is already in use for the production CMD (`backend/Dockerfile:101`)
where it's appropriate.

Updates the asyncpg-missing error message to include the
`--all-packages` flag (matching #2584) plus the persistent install flow,
and expands `config.example.yaml` so all three install paths
(local / docker dev / docker image build) are documented with their
multi-extra capabilities.

Tests:
  - `tests/test_detect_uv_extras.py` (21 tests) — local-path env parsing,
    YAML edge cases, env-vs-config precedence, whitelist rejection of
    shell metacharacters.
  - `tests/test_dev_entrypoint.py` (15 tests) — docker-path validation
    via `--print-extras`, multi-extra parsing, metacharacter abort.
  - `tests/test_persistence_scaffold.py` (22 tests, unchanged) — passes
    with the merged `--all-packages --extra postgres` error message.

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
Xinmin Zeng
2026-05-10 22:28:29 +08:00
committed by GitHub
parent 5127f08e1a
commit 94da8f67d7
8 changed files with 636 additions and 6 deletions
+19 -3
View File
@@ -871,9 +871,25 @@ skill_evolution:
#
# Postgres mode: put your connection URL in .env as DATABASE_URL,
# then reference it here with $DATABASE_URL.
# Install the driver first:
# Local: uv sync --extra postgres
# Docker: UV_EXTRAS=postgres docker compose build
#
# Install the driver — Issue #2754 fix lands `UV_EXTRAS` in every code path:
# Local `make dev` auto-detects from `database.backend: postgres` below
# and passes `--extra postgres` to `uv sync` on every restart, so
# the extra is no longer wiped. To opt in explicitly (or layer
# extras like `postgres,ollama`), set in project-root .env:
# UV_EXTRAS=postgres
# Docker dev `make docker-start` reads `UV_EXTRAS` from project-root .env via
# `env_file`. Set:
# UV_EXTRAS=postgres
# Multiple extras (`postgres,ollama`) supported here too — see
# docker/dev-entrypoint.sh.
# Docker img build-arg `UV_EXTRAS=postgres docker compose build` — single
# extra only at build time (backend/Dockerfile passes the value
# as one token to `--extra`).
#
# First-time bootstrap (before `make dev`):
# cd backend && uv sync --all-packages --extra postgres
# (--all-packages propagates the extra into workspace members — see PR #2584)
#
# NOTE: When both `checkpointer` and `database` are configured,
# `checkpointer` takes precedence for LangGraph state persistence.