Share one cargo target dir per repo across worktrees #11

New issue

Open

opened 2026-08-09 19:48:02 +01:00 by vnprc-agent · 0 comments

vnprc-agent commented

2026-08-09 19:48:02 +01:00

Collaborator

Stop per-worktree cargo target dirs from filling the disk: share one build cache per repo so branch worktrees dedup artifacts and keep them warm across allod change cleanup.

Primary goals:

Shared per-repo target dir — devenv.nix sets env.CARGO_TARGET_DIR = "${config.env.HOME}/.cache/cargo-target/hashpool" (or equivalent), so every worktree of this repo builds into one cache instead of its own target/.
Convention for manual builds — the same env var exported when building this repo (or cdk/cdk-ehash checkouts) from a bare nix shell, so agent-driven builds outside devenv share the cache too; document in the README or a dev-docs note.
Bounded cleanup — one directory to measure and delete per repo; du -sh ~/.cache/cargo-target/* answers "what is eating the disk" in one line.

Current state and specifics

Every allod change worktree gets an independent target/ per workspace, and each rebuilds a nearly identical dependency graph. A recent multi-branch stretch accumulated ~90G of near-duplicate build artifacts on the 98G root disk (individual dirs measured at 20-31G for full workspace builds, 11-24G per hashpool worktree) and hit ENOSPC twice in two days; recovery each time was manually deleting ~/changes/*/target dirs. Cargo keys artifacts by toolchain/feature/dependency fingerprints, so builds under different rustc versions or cdk pins coexist in one dir without invalidating each other — the shared dir holds the union, which is far smaller than N copies. A second win: today the cache dies with the worktree at allod change cleanup, so every new change branch starts from a cold build; a shared dir keeps it warm.

Tradeoff to accept: cargo holds an exclusive build lock per target dir, so concurrent builds of the same repo serialize (waiting on the lock, visible as "Blocking waiting for file lock"). Per-repo sharing (rather than one global dir) confines that contention to same-repo builds and keeps cross-repo parallelism.

Validation

Two worktrees of this repo build sequentially into the shared dir with the second build substantially cache-warm; devenv up works unchanged; deleting a worktree leaves the cache; disk usage across a multi-worktree session stays bounded near single-build size.

Scope

In scope: hashpool devenv.nix + docs. The same convention applied to the cdk/cdk-ehash checkouts is a docs/practice change alongside, not separate tooling. Out of scope: sccache or other compiler-level caches (heavier machinery than this needs), and any allod change tooling changes.

Stop per-worktree cargo target dirs from filling the disk: share one build cache per repo so branch worktrees dedup artifacts and keep them warm across `allod change cleanup`. Primary goals: - **Shared per-repo target dir** — `devenv.nix` sets `env.CARGO_TARGET_DIR = "${config.env.HOME}/.cache/cargo-target/hashpool"` (or equivalent), so every worktree of this repo builds into one cache instead of its own `target/`. - **Convention for manual builds** — the same env var exported when building this repo (or cdk/cdk-ehash checkouts) from a bare nix shell, so agent-driven builds outside devenv share the cache too; document in the README or a dev-docs note. - **Bounded cleanup** — one directory to measure and delete per repo; `du -sh ~/.cache/cargo-target/*` answers "what is eating the disk" in one line. ### Current state and specifics Every `allod change` worktree gets an independent `target/` per workspace, and each rebuilds a nearly identical dependency graph. A recent multi-branch stretch accumulated ~90G of near-duplicate build artifacts on the 98G root disk (individual dirs measured at 20-31G for full workspace builds, 11-24G per hashpool worktree) and hit ENOSPC twice in two days; recovery each time was manually deleting `~/changes/*/target` dirs. Cargo keys artifacts by toolchain/feature/dependency fingerprints, so builds under different rustc versions or cdk pins coexist in one dir without invalidating each other — the shared dir holds the union, which is far smaller than N copies. A second win: today the cache dies with the worktree at `allod change cleanup`, so every new change branch starts from a cold build; a shared dir keeps it warm. Tradeoff to accept: cargo holds an exclusive build lock per target dir, so concurrent builds of the same repo serialize (waiting on the lock, visible as "Blocking waiting for file lock"). Per-repo sharing (rather than one global dir) confines that contention to same-repo builds and keeps cross-repo parallelism. ### Validation Two worktrees of this repo build sequentially into the shared dir with the second build substantially cache-warm; `devenv up` works unchanged; deleting a worktree leaves the cache; disk usage across a multi-worktree session stays bounded near single-build size. ### Scope In scope: hashpool `devenv.nix` + docs. The same convention applied to the cdk/cdk-ehash checkouts is a docs/practice change alongside, not separate tooling. Out of scope: sccache or other compiler-level caches (heavier machinery than this needs), and any `allod change` tooling changes.

vnprc-agent referenced this issue from a commit

2026-08-09 19:48:30 +01:00

hashpool: durable disk remedy tracked as shared target dir, Refs vnprc/hashpool#11