Move the check suite out of flake.nix #93

New issue

Closed

opened 2026-09-21 22:59:38 +01:00 by allod-agent · 0 comments

allod-agent commented

2026-09-21 22:59:38 +01:00

Member

Move the check suite out of flake.nix into checks/, so the framework flake is the composition and each check is a function of what it reads.

Primary goals:

flake.nix is composition again — it is 3751 lines today and the checks output starts at line 822, so roughly three quarters of the file is test apparatus sitting in the file that builds every machine.
Every check a function of its inputs — each takes machineConfigurations, the builders, pkgs and system as arguments, which is newly possible because no check reads a data input by name any more.
Diffs that name what changed — a change to one check stops rendering as a change to the flake, and two checks stop being edited in the same 2900-line region.

Current state and specifics

All nineteen checks are declared inline in the checks output (flake.nix:822 onward). Six of them delegate part of their body to a file under checks/ — public-host.nix (flake.nix:3002), static-site.nix (:3014), pi-integration.nix (:3110), pi-provider-lifecycle.nix (:3232), memory-traces.nix (:3281), ssh-settings.nix (:3575) — but the part that stayed behind in each case is the fixture construction and the input wiring, which is the part that makes the region hard to read and hard to review.

allod/archetypes#88 is what makes this cheap now. Before it, a check reached into machines, devIdentities and mergedProfileDefinitions through closure over the flake's top-level let, so a check could not be moved without dragging that scope with it. Every check now names what it reads: the composed machines, or fixture values it builds itself. The six existing helper files show the shape the other thirteen should take.

Two constraints to carry across. testing.md records that two checks with the same derivation name shadow each other silently when check sets are merged, so names must stay unique and unchanged — a fork that excluded a check by name should not have to track a rename. And the suite's peak evaluation memory is the reason check.sh runs one nix process per check; moving declarations between files does not change that, but a refactor that accidentally forces more fixtures in one process would.

Scope

In scope: the thirteen checks with no helper file yet, the fixture construction that stayed behind in the six that have one, and whatever shared helpers the move makes worth extracting.

Explicitly not in scope: changing what any check asserts. This is a move, and the gate is that ./check.sh, ./fork-safety.sh and every machine's derivation path are unchanged by it.

Tracked elsewhere: the builder signatures (allod/archetypes#92), and whether the fork witness becomes a second composition (allod/archetypes#94), which this unblocks.

Refs allod/archetypes#88

Move the check suite out of `flake.nix` into `checks/`, so the framework flake is the composition and each check is a function of what it reads. Primary goals: - **`flake.nix` is composition again** — it is 3751 lines today and the `checks` output starts at line 822, so roughly three quarters of the file is test apparatus sitting in the file that builds every machine. - **Every check a function of its inputs** — each takes `machineConfigurations`, the builders, `pkgs` and `system` as arguments, which is newly possible because no check reads a data input by name any more. - **Diffs that name what changed** — a change to one check stops rendering as a change to the flake, and two checks stop being edited in the same 2900-line region. ### Current state and specifics All nineteen checks are declared inline in the `checks` output (`flake.nix:822` onward). Six of them delegate part of their body to a file under `checks/` — `public-host.nix` (`flake.nix:3002`), `static-site.nix` (`:3014`), `pi-integration.nix` (`:3110`), `pi-provider-lifecycle.nix` (`:3232`), `memory-traces.nix` (`:3281`), `ssh-settings.nix` (`:3575`) — but the part that stayed behind in each case is the fixture construction and the input wiring, which is the part that makes the region hard to read and hard to review. allod/archetypes#88 is what makes this cheap now. Before it, a check reached into `machines`, `devIdentities` and `mergedProfileDefinitions` through closure over the flake's top-level `let`, so a check could not be moved without dragging that scope with it. Every check now names what it reads: the composed machines, or fixture values it builds itself. The six existing helper files show the shape the other thirteen should take. Two constraints to carry across. `testing.md` records that two checks with the same derivation name shadow each other silently when check sets are merged, so names must stay unique and unchanged — a fork that excluded a check by name should not have to track a rename. And the suite's peak evaluation memory is the reason `check.sh` runs one `nix` process per check; moving declarations between files does not change that, but a refactor that accidentally forces more fixtures in one process would. ### Scope In scope: the thirteen checks with no helper file yet, the fixture construction that stayed behind in the six that have one, and whatever shared helpers the move makes worth extracting. Explicitly not in scope: changing what any check asserts. This is a move, and the gate is that `./check.sh`, `./fork-safety.sh` and every machine's derivation path are unchanged by it. Tracked elsewhere: the builder signatures (allod/archetypes#92), and whether the fork witness becomes a second composition (allod/archetypes#94), which this unblocks. Refs allod/archetypes#88

allod-agent referenced this issue

2026-09-21 22:59:55 +01:00

Reconsider the fork witness as a second composition #94

allod-agent referenced this issue from a commit

2026-09-22 00:56:48 +01:00

Move the check suite out of flake.nix into checks/

allod-agent referenced this issue from a pull request that will close it,

2026-09-22 01:01:04 +01:00

Move the check suite out of flake.nix into checks/ #96