Make the runtime-selection, vm-facts drift, and dev-forge-opt-out checks fork-safe #36

New issue

Open

opened 2026-08-02 23:09:22 +01:00 by vnprc-agent · 1 comment

vnprc-agent commented

2026-08-02 23:09:22 +01:00

Bring runtime-module-selection, vm-facts-negative, and dev-forge-opt-out up to the same fork-safety standard allod/archetypes#11 asks of pi-integration: all three fail against a deployment fork that supplies its own profiles/secrets/inventory inputs through the documented follows redirects, so a fork consuming the check suite has to exclude them and loses their coverage.

Primary goals:

Fixtures must not assume the public data layer — a check that drives a builder or captures expected values from public master data breaks on any fork whose identities, profile data, or inventory legitimately differ.
Negative fixtures must not depend on an unused enum member — a drift fixture that needs a valid runtime literal no real machine uses becomes unsatisfiable the day a fleet uses every member, which is exactly the state the microvm arc drives fleets toward.
Keep the coverage — each check pins a real property; the fix is fork-safe fixtures (synthetic machines/identities, data-derived expectations, or presence-gating), not deletion.

Per-check evidence, measured against a fork at archetypes `8aa0fce`

runtime-module-selection builds its selection fixtures from whichever dev machine inventory declares first, driving mkDevVm without the profileData layer that machineConfigurations composes. On a fork whose secrets identity assigns that machine a different username than the public secrets do, the fixture merges two home.username definitions and dies before asserting anything:

error: The option `home-manager.users.<user>.home.username' has conflicting definition values:
- In `<fork profiles>/flake.nix': "<fork username>"
- In `<public secrets>/nixos/common.nix': "<public username>"

It also hard-requires the inventory input to expose checks.<system>.runtime-fact-mutations, which only the public inventory carries.

vm-facts-negative (as of the runtime-axis extension) requires a valid runtime literal that differs from every runtime in the composed vmFacts. With a two-value libvirt/microvm enum and a fleet that has machines on both, no such literal exists and the check refuses by its own design:

error: vm-facts-negative: fixture runtime "microvm" is in use by a real machine; it must differ from every runtime in vmFacts, or a factFor returning that literal would satisfy both this check and vm-facts-coherence's inventory diff

dev-forge-opt-out asserts the composed dev machine generates exactly the two age secrets captured from public master, byte-for-byte in file/path/owner/group/mode. A fork that uses the framework's own token-registry opt-out (the mechanism allod/archetypes#17 added) to withhold one credential fails the capture comparison:

error: dev-forge-opt-out: with-access allod-dev no longer generates today's two secrets with today's file/path/owner/group/mode

Scope

In scope: making these three checks pass, or self-skip with a named reason, against a fork supplying divergent profiles/secrets/inventory data. Out of scope: the fork's own exclusion list (each fork carries that until this lands) and pi-integration, which allod/archetypes#11 already tracks.

Found during the microvm runtime adoption arc (allod/strategy#20), where a fork excluding these checks is what makes its composed check set green.

Bring `runtime-module-selection`, `vm-facts-negative`, and `dev-forge-opt-out` up to the same fork-safety standard allod/archetypes#11 asks of `pi-integration`: all three fail against a deployment fork that supplies its own `profiles`/`secrets`/`inventory` inputs through the documented follows redirects, so a fork consuming the check suite has to exclude them and loses their coverage. Primary goals: - **Fixtures must not assume the public data layer** — a check that drives a builder or captures expected values from public `master` data breaks on any fork whose identities, profile data, or inventory legitimately differ. - **Negative fixtures must not depend on an unused enum member** — a drift fixture that needs a valid runtime literal no real machine uses becomes unsatisfiable the day a fleet uses every member, which is exactly the state the microvm arc drives fleets toward. - **Keep the coverage** — each check pins a real property; the fix is fork-safe fixtures (synthetic machines/identities, data-derived expectations, or presence-gating), not deletion. ### Per-check evidence, measured against a fork at archetypes `8aa0fce` `runtime-module-selection` builds its selection fixtures from whichever dev machine inventory declares first, driving `mkDevVm` without the `profileData` layer that `machineConfigurations` composes. On a fork whose secrets identity assigns that machine a different username than the public secrets do, the fixture merges two `home.username` definitions and dies before asserting anything: ``` error: The option `home-manager.users.<user>.home.username' has conflicting definition values: - In `<fork profiles>/flake.nix': "<fork username>" - In `<public secrets>/nixos/common.nix': "<public username>" ``` It also hard-requires the inventory input to expose `checks.<system>.runtime-fact-mutations`, which only the public inventory carries. `vm-facts-negative` (as of the runtime-axis extension) requires a valid runtime literal that differs from every runtime in the composed `vmFacts`. With a two-value `libvirt`/`microvm` enum and a fleet that has machines on both, no such literal exists and the check refuses by its own design: ``` error: vm-facts-negative: fixture runtime "microvm" is in use by a real machine; it must differ from every runtime in vmFacts, or a factFor returning that literal would satisfy both this check and vm-facts-coherence's inventory diff ``` `dev-forge-opt-out` asserts the composed dev machine generates exactly the two age secrets captured from public `master`, byte-for-byte in file/path/owner/group/mode. A fork that uses the framework's own token-registry opt-out (the mechanism allod/archetypes#17 added) to withhold one credential fails the capture comparison: ``` error: dev-forge-opt-out: with-access allod-dev no longer generates today's two secrets with today's file/path/owner/group/mode ``` ### Scope In scope: making these three checks pass, or self-skip with a named reason, against a fork supplying divergent `profiles`/`secrets`/`inventory` data. Out of scope: the fork's own exclusion list (each fork carries that until this lands) and pi-integration, which allod/archetypes#11 already tracks. Found during the microvm runtime adoption arc (allod/strategy#20), where a fork excluding these checks is what makes its composed check set green.

vnprc-agent commented

2026-08-22 04:33:19 +01:00

Author

A fourth check now has the same defect: pi-provider-lifecycle, added with the managed Pi provider arc.

Its fixture hard-codes machineName = "allod-dev" and drives mkDevVm directly, without the profileData layer machineConfigurations composes. That is the same shape as runtime-module-selection: on a fork whose profileData supplies that machine a different runtime identity, the fixture merges two home.username definitions and dies before asserting anything.

… while evaluating the option `home-manager.users.<user>.home.activation':
… while evaluating the option `home-manager.users.<user>.home.username':
error: The option `home-manager.users.<user>.home.username' has conflicting definition values:
- In `<fork profiles>/flake.nix': "<fork username>"
- In `<public secrets>/nixos/common.nix': "<public username>"

Measured against archetypes c41a294 with a fork consuming the check suite through the documented profiles/secrets/inventory follows redirects. Every other framework check in that fork's re-exported suite passes at this pin except the ones already listed here and microvm-host-join, which is the same fixture class again.

The cost is higher for this one than for the three above, because the fork has no fork-safe sibling and no real machine to fall back on: a deployment that manages no Pi provider yet has nothing that exercises the reconciler, so excluding the check drops the coverage entirely rather than relocating it. The fixture builds a machine only to reach a home-activation script and an age secret, so composing the fixture identity through the same layer production uses — or building a synthetic dev machine rather than reusing a real machine name — would keep the witness and cost nothing.

A fourth check now has the same defect: `pi-provider-lifecycle`, added with the managed Pi provider arc. Its fixture hard-codes `machineName = "allod-dev"` and drives `mkDevVm` directly, without the `profileData` layer `machineConfigurations` composes. That is the same shape as `runtime-module-selection`: on a fork whose `profileData` supplies that machine a different runtime identity, the fixture merges two `home.username` definitions and dies before asserting anything. ``` … while evaluating the option `home-manager.users.<user>.home.activation': … while evaluating the option `home-manager.users.<user>.home.username': error: The option `home-manager.users.<user>.home.username' has conflicting definition values: - In `<fork profiles>/flake.nix': "<fork username>" - In `<public secrets>/nixos/common.nix': "<public username>" ``` Measured against archetypes `c41a294` with a fork consuming the check suite through the documented `profiles`/`secrets`/`inventory` follows redirects. Every other framework check in that fork's re-exported suite passes at this pin except the ones already listed here and `microvm-host-join`, which is the same fixture class again. The cost is higher for this one than for the three above, because the fork has no fork-safe sibling and no real machine to fall back on: a deployment that manages no Pi provider yet has nothing that exercises the reconciler, so excluding the check drops the coverage entirely rather than relocating it. The fixture builds a machine only to reach a home-activation script and an age secret, so composing the fixture identity through the same layer production uses — or building a synthetic dev machine rather than reusing a real machine name — would keep the witness and cost nothing.