flake-update-cascade: process repositories in dependency order and propagate advanced pins #171

New issue

Closed

opened 2026-09-08 17:26:23 +01:00 by vnprc-agent · 0 comments

vnprc-agent commented

2026-09-08 17:26:23 +01:00

Contributor

Make one flake-update-cascade run bring every repository in the workspace to a consistent set of locks, instead of one run per level of the dependency chain.

Primary goals:

Dependency order — repositories are processed leaves first, so a repository re-pins an upstream only after that upstream's own lock commit exists.
Propagated updates — a pin of a workspace repository that is behind that repository's default branch is updated whether or not its input name was on the command line, so commits the run itself pushes, and lock PRs merged since the last run, reach every downstream lock in the same run.
Honest --pr runs — a run that cannot go further because the next commit sits on an unmerged PR ends by naming those PRs and stating that the same command re-run after merge continues the cascade.
Unchanged where nothing chains — on a workspace with no pins between its repositories, output, commits, and exit status are identical to today, and every existing suite stays green.
Lands in Go — the change follows the allod/tools#159 port and cutover; the port itself stays bug-for-bug, today's order included, so the Bash suites remain its oracle.

Current state

The tool cannot converge in one run for two independent reasons, and --pr mode adds a third.

Repositories are processed in the order workspace_collect_repos returns them (flake/flake-update-cascade:78, lib/workspace.sh:10), a directory walk. Under the <workspace>/allod/<repo> layout that is archetypes, deploy, inventory, nexus, profiles, secrets, tools, vm, close to reverse dependency order: archetypes and deploy are processed, committed, and pushed before nexus, secrets, and vm, which they pin. The public chain is vm → nexus → archetypes → deploy, so a vm commit made by the run is not seen by archetypes until the next run, and the archetypes commit that then results is not seen by deploy until the run after that.

The set of inputs to update is fixed at argument parsing (flake/flake-update-cascade:47) and only those names are resolved in each lock (get_update_paths, flake/flake-update-cascade:85). Nothing records that a repository processed earlier in the run received a new commit, so a later repository that pins it is not asked to move that pin. flake-status --upstream cannot pre-empt this either: it compares against the remote head, which only moves once the run has pushed.

In --pr mode a downstream pin is resolved on the default branch, so it cannot lock a commit that is still on a PR branch; every level of the chain needs its PR merged before the next level can move. That limit is inherent, and the tool does not report it.

The commit message names every input on the command line (INPUT_LABEL, flake/flake-update-cascade:149, used at :406 and :491) rather than the inputs that moved in that repository, so a commit on one repository names inputs that exist only in another.

Evidence: allod/deploy received four lock commits on 2026-09-08, bd12ed4, 204b739, 21ed97d, 633be25, each naming the inputs the previous wave had moved. The original design note put transitive updates out of scope and no revision since has added ordering, so this is a gap in the design rather than a regression.

Design

Graph. For each repository with a flake.lock, every directly pinned root input (a string edge, not a follows array) whose original.url normalises to the same repository as another workspace repository's origin remote is an edge from that repository to this one, labelled with the input name. Normalise to host, owner, and repository name; ignore scheme, user, port, and a .git suffix, so an ssh://…:2222/ pin matches an https:// remote. Pins of repositories absent from the workspace are external and take no part in ordering.

Order. Topological sort with ties broken by the current directory order. A cycle is a preflight error naming the cycle; nothing is touched. Preflight stays whole-workspace and atomic as today.

Update set per repository. The requested names resolved through the lock graph as today, plus every workspace-internal root pin whose locked rev differs from the target repository's remote-tracking head origin/<branch>, where <branch> is the pin's original.ref or the default branch when none is given; a rev-only pin with no ref is left alone. Preflight runs git fetch origin in every repository that is a pin target so the tracking refs are current, and a push made by this run advances the tracking ref of the repository pushed. Because a repository is processed after its upstreams, that head already includes anything this run pushed to them. One rule therefore covers both commits the run itself makes and lock PRs merged since the last run, with no per-run bookkeeping of what advanced. A pin whose rev matches its target is not touched, so a repository whose only stale inputs are external behaves exactly as today. This is the one visible change on an existing workspace: a run naming only an external input also moves internal pins that are behind.

Transitive nodes such as archetypes/vm move through the intermediate repository's own lock bump, as they do now; a transitive node a downstream lock owns ahead of the intermediate repository is the existing gotcha and stays out of scope.

Commit message. flake.lock: update <names>, where the names are the inputs that actually moved in that repository, in lock order.

--pr mode. The run creates or updates PRs in dependency order as today. A repository whose internal pin targets a repository that this run moved only on a PR branch is reported, not updated. The run ends with a summary listing those repositories, the PRs that block them, and the statement that the same command re-run after merge continues the cascade. Exit status stays 0 when every attempted update succeeded; the summary is information, not failure.

--dry-run. Prints the plan in dependency order. Propagation from commits this run would push cannot be computed without pushing, so a dry run shows the first wave and says so; propagation from already-merged commits is shown in full.

Validation

Extend tests/flake/flake-update-cascade/testlib.sh with a fixture of three chained repositories, leaf, mid, and top, whose locks carry original.url entries matching the mocked remote get-url origin, with the mock git serving a per-repository tracking head that advances on push.

Direct mode, one run: commits and pushes happen in the order leaf, mid, top; mid's pin of leaf and top's pin of mid equal the heads the run pushed; each commit message names only the input that moved in that repository.
Direct mode with top listed first in directory order: same result, proving the sort and not the walk decides.
--pr mode, one run: leaf gets a PR; mid and top are reported as waiting on it with its number; nothing is committed there; exit 0.
Already-merged wave: mid's pin behind leaf's tracking head with no new leaf commit; a run naming an unrelated input still moves mid's pin and then top's.
Cycle: two repositories pinning each other fail preflight naming both, and the command log shows no nix flake update.
Every existing suite unchanged: their fixtures have no inter-repository pins, and the stable tiebreak makes the new order equal to the old one.

Scope

In scope: the ordering, the update-set rule, the commit-message change, the --pr and --dry-run reporting, the tests above, and the flake/README.md and --help text that describe the tool as updating named inputs.

Sequencing: this follows allod/tools#159. The Go port lands bug-for-bug first, directory order included, and this change is the next Go PR after cutover. If allod/tools#159 has no implementation PR open when this issue is picked up, the same design applies to the Bash tool and the port then inherits it; that is the fallback, not the plan.

Tracked elsewhere: flake-status --upstream comparing to the default branch rather than the locked ref, allod/tools#170; transitive nodes a downstream lock owns ahead of the intermediate repository, the existing nix.md gotcha; pinning unmerged PR revisions by rev so a --pr cascade could chain without merges, deliberately not proposed because it makes every downstream PR depend on upstream merge order.

Make one `flake-update-cascade` run bring every repository in the workspace to a consistent set of locks, instead of one run per level of the dependency chain. Primary goals: - **Dependency order** — repositories are processed leaves first, so a repository re-pins an upstream only after that upstream's own lock commit exists. - **Propagated updates** — a pin of a workspace repository that is behind that repository's default branch is updated whether or not its input name was on the command line, so commits the run itself pushes, and lock PRs merged since the last run, reach every downstream lock in the same run. - **Honest `--pr` runs** — a run that cannot go further because the next commit sits on an unmerged PR ends by naming those PRs and stating that the same command re-run after merge continues the cascade. - **Unchanged where nothing chains** — on a workspace with no pins between its repositories, output, commits, and exit status are identical to today, and every existing suite stays green. - **Lands in Go** — the change follows the allod/tools#159 port and cutover; the port itself stays bug-for-bug, today's order included, so the Bash suites remain its oracle. ### Current state The tool cannot converge in one run for two independent reasons, and `--pr` mode adds a third. Repositories are processed in the order `workspace_collect_repos` returns them (`flake/flake-update-cascade:78`, `lib/workspace.sh:10`), a directory walk. Under the `<workspace>/allod/<repo>` layout that is `archetypes, deploy, inventory, nexus, profiles, secrets, tools, vm`, close to reverse dependency order: `archetypes` and `deploy` are processed, committed, and pushed before `nexus`, `secrets`, and `vm`, which they pin. The public chain is `vm → nexus → archetypes → deploy`, so a `vm` commit made by the run is not seen by `archetypes` until the next run, and the `archetypes` commit that then results is not seen by `deploy` until the run after that. The set of inputs to update is fixed at argument parsing (`flake/flake-update-cascade:47`) and only those names are resolved in each lock (`get_update_paths`, `flake/flake-update-cascade:85`). Nothing records that a repository processed earlier in the run received a new commit, so a later repository that pins it is not asked to move that pin. `flake-status --upstream` cannot pre-empt this either: it compares against the remote head, which only moves once the run has pushed. In `--pr` mode a downstream pin is resolved on the default branch, so it cannot lock a commit that is still on a PR branch; every level of the chain needs its PR merged before the next level can move. That limit is inherent, and the tool does not report it. The commit message names every input on the command line (`INPUT_LABEL`, `flake/flake-update-cascade:149`, used at `:406` and `:491`) rather than the inputs that moved in that repository, so a commit on one repository names inputs that exist only in another. Evidence: `allod/deploy` received four lock commits on 2026-09-08, `bd12ed4`, `204b739`, `21ed97d`, `633be25`, each naming the inputs the previous wave had moved. The original design note put transitive updates out of scope and no revision since has added ordering, so this is a gap in the design rather than a regression. ### Design Graph. For each repository with a `flake.lock`, every directly pinned root input (a string edge, not a `follows` array) whose `original.url` normalises to the same repository as another workspace repository's `origin` remote is an edge from that repository to this one, labelled with the input name. Normalise to host, owner, and repository name; ignore scheme, user, port, and a `.git` suffix, so an `ssh://…:2222/` pin matches an `https://` remote. Pins of repositories absent from the workspace are external and take no part in ordering. Order. Topological sort with ties broken by the current directory order. A cycle is a preflight error naming the cycle; nothing is touched. Preflight stays whole-workspace and atomic as today. Update set per repository. The requested names resolved through the lock graph as today, plus every workspace-internal root pin whose locked `rev` differs from the target repository's remote-tracking head `origin/<branch>`, where `<branch>` is the pin's `original.ref` or the default branch when none is given; a rev-only pin with no ref is left alone. Preflight runs `git fetch origin` in every repository that is a pin target so the tracking refs are current, and a push made by this run advances the tracking ref of the repository pushed. Because a repository is processed after its upstreams, that head already includes anything this run pushed to them. One rule therefore covers both commits the run itself makes and lock PRs merged since the last run, with no per-run bookkeeping of what advanced. A pin whose rev matches its target is not touched, so a repository whose only stale inputs are external behaves exactly as today. This is the one visible change on an existing workspace: a run naming only an external input also moves internal pins that are behind. Transitive nodes such as `archetypes/vm` move through the intermediate repository's own lock bump, as they do now; a transitive node a downstream lock owns ahead of the intermediate repository is the existing gotcha and stays out of scope. Commit message. `flake.lock: update <names>`, where the names are the inputs that actually moved in that repository, in lock order. `--pr` mode. The run creates or updates PRs in dependency order as today. A repository whose internal pin targets a repository that this run moved only on a PR branch is reported, not updated. The run ends with a summary listing those repositories, the PRs that block them, and the statement that the same command re-run after merge continues the cascade. Exit status stays 0 when every attempted update succeeded; the summary is information, not failure. `--dry-run`. Prints the plan in dependency order. Propagation from commits this run would push cannot be computed without pushing, so a dry run shows the first wave and says so; propagation from already-merged commits is shown in full. ### Validation Extend `tests/flake/flake-update-cascade/testlib.sh` with a fixture of three chained repositories, leaf, mid, and top, whose locks carry `original.url` entries matching the mocked `remote get-url origin`, with the mock `git` serving a per-repository tracking head that advances on `push`. - Direct mode, one run: commits and pushes happen in the order leaf, mid, top; mid's pin of leaf and top's pin of mid equal the heads the run pushed; each commit message names only the input that moved in that repository. - Direct mode with top listed first in directory order: same result, proving the sort and not the walk decides. - `--pr` mode, one run: leaf gets a PR; mid and top are reported as waiting on it with its number; nothing is committed there; exit 0. - Already-merged wave: mid's pin behind leaf's tracking head with no new leaf commit; a run naming an unrelated input still moves mid's pin and then top's. - Cycle: two repositories pinning each other fail preflight naming both, and the command log shows no `nix flake update`. - Every existing suite unchanged: their fixtures have no inter-repository pins, and the stable tiebreak makes the new order equal to the old one. ### Scope In scope: the ordering, the update-set rule, the commit-message change, the `--pr` and `--dry-run` reporting, the tests above, and the `flake/README.md` and `--help` text that describe the tool as updating named inputs. Sequencing: this follows allod/tools#159. The Go port lands bug-for-bug first, directory order included, and this change is the next Go PR after cutover. If allod/tools#159 has no implementation PR open when this issue is picked up, the same design applies to the Bash tool and the port then inherits it; that is the fallback, not the plan. Tracked elsewhere: `flake-status --upstream` comparing to the default branch rather than the locked ref, allod/tools#170; transitive nodes a downstream lock owns ahead of the intermediate repository, the existing `nix.md` gotcha; pinning unmerged PR revisions by `rev` so a `--pr` cascade could chain without merges, deliberately not proposed because it makes every downstream PR depend on upstream merge order.

vnprc-agent referenced this issue

2026-09-08 17:26:29 +01:00

Port and deploy flake-update-cascade as a Go binary #159

vnprc-agent referenced this issue from a commit

2026-09-08 17:27:07 +01:00

memory: note flake-update-cascade needs one run per dependency level until allod/tools#171