flake-update-cascade: process repositories in dependency order and propagate advanced pins #176
No reviewers
Labels
No labels
blocked
bug
decision
duplicate
enhancement
help wanted
invalid
landed?
question
ready-to-merge
stale
wontfix
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
allod/tools!176
Loading…
Reference in a new issue
No description provided.
Delete branch "agent/cascade-dependency-order"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
One
flake-update-cascaderun now brings every workspace repository to a consistent set of locks instead of one run per level of the chain. Repositories are processed leaves first, each after the workspace repositories it pins, and a pin of a workspace repository that is behind that repository's branch head is moved whether or not its input was on the command line, so a lock commit the run pushes tovmreachesarchetypesand thendeployin the same pass. What stays identical: a workspace with no pins between its repositories produces the same output, commits and exit status as before, and every existing suite is unchanged apart from the commit message, which now names the paths that moved in that repository rather than the command line. What proves it: a chained three-repository mock fixture in every mode, a hand-run fixture over real git and real nix with bare origins,nix flake check, a read-only review pass whose findings are fixed in the second commit, and two--dry-runs over the public chain in this workspace, all below. If a run does something wrong, the pre-flight is still atomic and a cycle or any pre-flight error touches nothing; a bad lock commit is one revert per repository, and--dry-runshows the first wave before any push.Also folded in: the paths a repository is updated on are read from its lock after the tool's own
git pull, so a checkout whose pull changed its input graph is judged on its current inputs instead of being reportedalready up to dateon paths that no longer exist (allod/tools#175).Closes allod/tools#171
Closes allod/tools#175
Design
The issue's design and validation sections are the spec; they predate the head-reading change in
090ceab, so what they call "the remote-tracking head aftergit fetch origin" is read here withgit ls-remoteat the pin's own URL, and a moved pin is written withnix flake lock --override-inputas every other moved input is.follows) whoseoriginaldeclares a git or GitHub branch and whose repository, compared by host and path with scheme, user, port, query and.gitsuffix ignored, is the origin of another workspace repository inside the push boundary (the forge, orallowed-external-remotes). A pin naming a revision, or carryingdir,hostorsubmodules, is neither ordered on nor propagated; a pin of a repository the workspace does not hold, or holds outside the push boundary, is external.x: dependency cycle: x → y → x, and nothing is touched; a cycle through skipped repositories, or a repository pinning itself, constrains nothing. The order and the cycle check come from the locks before the run's own pulls; everything else is decided on the lock after the pull.planLockcompares each against its branch head and keeps only what moved, so a pin at its head costs onels-remoteand nothing else, and a repository whose only stale inputs are external behaves as before. After a direct push the head every downstream pin reads is seeded from the checkout's HEAD, so the next repository down pins exactly what was pushed with no second round trip. A repository with no named input but with workspace pins is now examined (pulling.../already up to date) and pre-flighted where it was skipped before; that and "a run naming only an external input also moves internal pins that are behind" are the two visible changes on an existing workspace.--pr: a repository whose pin of a default branch targets a repository this run moved on a PR branch is pulled and then waits, reported aswaiting on PR #42 (allod/vm), with nothing written; the wait is transitive through a waiting repository; a pin of a release branch or a tag never waits; the run ends listing the waiting repositories and their PRs and says the same command re-run after merge continues; exit 0. A failedforge pr createfails the repository and leaves nothing downstream waiting; an existing PR has its title and body refreshed to what moved this time.--dry-run: the plan in dependency order; pins behind already-merged commits are shown in full; a pin of a commit this run would push is named on the repository's own lines and again at the end as not shown. Without--pr, a protected repository's update is shown, markeda direct run skips this repository, and promises nothing downstream.flake.lock: update <paths that moved here>in lock order, e.g.flake.lock: update allod-tools, vm/nixpkgs. The PR branch name still comes from the command line so re-runs find their branch.Review pass
A read-only review of the first commit (
c8ac5a8) produced ten findings, recorded in the PR comment. Seven are fixed inbdaf108: waiting decided on the pre-pull lock, propagation blind to the pin's branch, cycles among skipped repositories aborting every run, dry runs promising a wave behind a protected repository, the pushed head re-read from the remote, a failed PR creation reported as success, and an existing PR never retitled. Three are the issue's own design and are stated in the README: pre-flight now covering every pinning repository, one PR branch per command line, and a waiting repository deferring its own inputs one round per chain level.Risk
commit -mline changes for every run that names more than one input, and for propagated pins it names an input that was not on the command line. Anything parsing those messages sees the new shape; nothing in the workspace does.flake-update-cascade <external-input>now also moves stale workspace pins in the same commit. That is the design's stated intent, and it is what makes one run converge, but it widens what a routinenixpkgsbump can touch.remote get-url originfor every repository, including ones without a lock, and a repository with workspace pins is subject to the branch, dirty-tree and unpushed checks even when no named input reaches it. A dirtyarchetypestherefore blocks ahome-managerrun it used to skip through. That is the atomic pre-flight doing its job, since the run may commit there.--prmode before the first PR merges opens two PRs carrying the same propagated pin, because the branch is named after the command line.repoIdentityoverlapscmd/allod/patch.go:remoteIdentity, which rejects the file paths and query strings the chain fixture needs, so the two stay separate.nix.mdgotcha), pinning unmerged PR revisions to chain a--prcascade without merges, allod/tools#170, and anything aboutflake-status.Validation
nix flake checkonbdaf108:go-checks(gofmt, vet,go test ./...with and without thesitetag),cascade-suites(every mock-driven suite against the packaged program, now includingdependency-order.shandpost-pull-lock.sh),allod-parity. Every pre-existing suite passes unchanged exceptflake-update-cascade-multiple-inputs.sh, whose expected commit line is nowflake.lock: update allod-tools, vm/nixpkgs.New suites, from the issue's validation section plus one scenario per review finding:
tests/flake/flake-update-cascade/dependency-order.sh(62 checks): the chained leaf/mid/top fixture with a mock git whose per-repository tracking head advances on push. Direct mode pushes leaf, mid, top in that order with mid's pin equal to the leaf head the run pushed and top's to mid's, each commit naming only what moved there, and never asks the remote for a head this run pushed; the same with top first in directory order, proving the sort and not the walk decides;--prgives leaf a PR, refreshes its title, and reports mid and top aswaiting on PR #42 (leaf)with nothing committed there, exit 0, plus the closing summary; an already-merged wave (mid's pin behind leaf's head, no new leaf commit, a run naming an unrelated input) moves mid and then top; a dry run shows leaf, names the pins it cannot show, changes no lock; a release-branch pin is read but neither waits nor is promised a commit; a protected repository in a dry run is shown, named as skipped, and promises nothing downstream; a pin the pull brings in waits on the PR; a cycle among repositories inactive-pr-branchesdoes not fail the run; two repositories pinning each other fail pre-flight naming both with no nix or pull; a repository whose workspace pins are all current is examined and left alone while one with neither a named input nor a workspace pin is skipped as before.tests/flake/flake-update-cascade/post-pull-lock.sh(8 checks): a mock pull that rewrites the lock. When the pull turns a nestedvm/demointo a directdemo, nix is asked aboutdemoand never aboutvm/demo; when the pull removes the input, the repository is skipped after the pull with no head read, no nix call, and noalready up to date.tests/flake/flake-update-cascade/failures.shgains a failedforge pr create: status 1, the checkout returned to the default branch with the lock restored.tests/flake/flake-update-cascade-chain.sh, run by hand like the nixConfig suite: real git and real nix over three repositories with bare origins under a temporary directory, chained throughgit+file://inputs, one direct run naming only leaf's external input. Result onbdaf108: three origins each gained exactly one commit, in order, each pin equal to the commit the run had just pushed, every checkout clean; commitsflake.lock: update demo,flake.lock: update leaf, leaf/demo,flake.lock: update mid, mid/leaf/demo.RootPins.Live witnesses over this workspace with bdaf108's binary, git progress and pull chatter removed; direct and
--prmode were not run against real repositories, the mutating path's witness is the bare-origin fixture above.flake-update-cascade nixpkgs --dry-run --pr, 26 seconds, exit 0: dependency order and the first-wave notice for every pin that follows a commit the run would push.flake-update-cascade nixpkgs --dry-runwithout--pr, 38 seconds, exit 0: every repository here is inprotected-branches, so each one printsprotected branch (master) — a direct run skips this repository; re-run with --prabove its update, and no wave is announced. An earlier run of the first commit, before two lock PRs merged upstream, showed the already-merged wave in full:nexus: 65bafe4 → c5b9445in archetypes andarchetypes: 025bb21 → 77e9ce7in deploy, both taken alongside the named input.(
allod/memory,allod/strategyandallod/.profileprintno flake.lock, skippingand are elided from both.)Read-only review pass on
c8ac5a8(high effort, eight finder angles, 16 candidates verified). Ten findings survived; seven are fixed inbdaf108and three are the issue's own design, now stated in the README rather than changed.Fixed:
--prmode was decided on the pre-pull lock, so a workspace pin the pull brought in was neither waited on nor listed. Blockers now come from the lock after the pull; the waiting repository is pulled and nothing else is touched.--prmode. Only a pin of the target's default branch waits or is promised anything; every pin is still read and moved when behind.a direct run skips this repositoryand promises nothing downstream on its account.forge pr createfailing was reported as success with a placeholder that downstream repositories were then told to wait on. A failed creation fails the repository, restores the checkout, and leaves nothing waiting.Design, disclosed in the README:
agent/flake-update-<input>so re-runs find their PR; running two different input names before the first PR merges opens two PRs carrying the same propagated pin.Cut as cleanup rather than defects:
repoIdentityoverlapscmd/allod/patch.go:remoteIdentity, which rejects the file paths and query strings the chain fixture needs, so the two stay separate;remoteIsAllowedscans its file twice per repository.nix flake checkpasses onbdaf108; the real-git chain suite and a--dry-runover this workspace were re-run on it.