Deliver microvm guest credentials from host memory #37

Merged
vnprc merged 4 commits from agent/microvm-guest-credentials into master 2026-08-06 20:03:10 +01:00
Member

A machine on the microVM runtime now has its private keys and tokens handed to it by the host at every boot instead of decrypting them itself. It holds no age identity, writes no credential onto anything that survives a reboot, and cannot quietly invent a new SSH identity when its host key fails to arrive. Nothing about a libvirt machine changes — all four example machines build to byte-identical derivations across this commit, and no machine moves to the microVM runtime here.

What proves it: a two-node boot test shows a guest presenting exactly the key the host delivered with nothing written to disk, and a guest given no key leaving sshd failed with no key generated. The service that writes the other credentials is executed against canary credentials and shown refusing every malformed one. The whole repo gate passes.

If this is wrong, revert the commit. Nothing here runs on the host, no deployed state is touched, and no machine selects the new runtime yet.

Three things below are judgment calls a reviewer should either accept or send back: sshd bypasses the credential-writing service, the option is named differently from the parent plan, and a machine that declares GitHub credential targets is now refused rather than built.


What this implements

Refs allod/archetypes#29 — the guest half of allod/strategy#20 contracts 7, 8a, 9, 10, 11, 12 and 14, all in allod/archetypes.

  • One runtime root. allod.archetypes.microvm.guestCredentialRoot (/run/allod/credentials by default), validated as a normalized absolute string strictly below /run/allod. Every consumer path is derived from it and no literal is restated.
  • A closed, derived credential-name set. The SSH host key always; the Forge HTTPS credential, API token and SSH key only when the machine declares them, so the forgeAccess opt-out keeps meaning one thing on both runtimes. Exported as allod.archetypes.microvm.credentialNames for the host integration (contract 8b) to build microvm.credentialFiles from — this change sets no values.
  • A materializer boot service, not an activation script: it loads the credentials with systemd, validates each one's exact shape, and writes each consumer file atomically with a declared owner and mode. It carries no "missing is expected" branch. nix-daemon.service requires it, so a failed materializer fails its consumer visibly instead of starting one with no credentials.
  • Runtime-rooted consumers: sshd's HostKey, the Nix daemon's netrc, root's and the account's git credential stores, the Forge API token (both FORGE_TOKEN_FILE and FORGEJO_TOKEN_FILE), and the Forge SSH IdentityFile in the generated Home Manager output.
  • No age in a microvm guest: no age.secrets, no age.identityPaths, no modules/netrc.nix, no GitHub credential module.
  • Assertions over the merged configuration for every contract rule, each with a paired sabotage pinned to the diagnostic it names.

Decisions a reviewer should accept or send back

1. sshd reads its credential directly; it does not go through the materializer. The issue's handoff comment calls this an explicit fork and asks for a deliberate decision. Taken: services.openssh.hostKeys = [ ], sshd-keygen masked, LoadCredential=ssh-host-key on sshd.service, HostKey /run/credentials/sshd.service/ssh-host-key.

It writes strictly less plaintext than materializing a copy would — systemd's own credential directory is 0400 root:root inside a 0500 directory on unswappable ramfs and goes away with the unit — and it deletes the ordering hazard the handoff names rather than working around it (sshd.service is After=sshd-keygen.service by default, and a materializer ordered before sshd must not inherit that).

The cost, stated plainly: the issue's primary goal says every credential-derived plaintext file lives under the guest credential root, and this one does not. That reading cannot be literal — the materializer's own LoadCredential creates the same kind of directory, and so would any unit-scoped credential — but it is a goal-level deviation and it is recorded on the issue rather than left silent. A second cost: ssh-host-key is then the one credential no framework code shape-validates. A reviewer suggested loading it into the materializer for validation without writing a destination; declined, because that adds a second full-boot-lifetime plaintext copy of the machine's identity key to buy a nicer message than sshd's own no hostkeys available -- exiting plus a failed unit.

2. The option is allod.archetypes.microvm.guestCredentialRoot, not allod.microvm.guestCredentialRoot. The plan and the issue write the latter. This repo already owns allod.archetypes.microvm.volumeImageRoot and allod.archetypes.microvm.hostInterfaceId under a namespace whose recorded reason is that it names the repo that owns it. A third spelling in the same feature area would be worse than either choice. Recorded on the issue so the plan can absorb it rather than be silently contradicted.

3. A microvm machine that declares GitHub credential targets fails evaluation. Contract 9 says to point them at runtime files; there is no route to do that — their registry deployed_path values are durable, and per-target names would collide with contract 7's closed 28-character name set. Refusing beats building a machine without a credential it says it has. The public template declares none, so nothing in the example fleet changes. This is a contract inversion, recorded on the issue as interim.

Beyond the issue's scope, deliberately

  • A booted witness for contract 11 (checks/microvm-sshd-boot.nix). The issue says nothing here boots a guest; its handoff comment retracts that and prices the experiment. Two nodes prove both halves directly, and the nodes import the same production fragment a real machine composes rather than a restated copy. allod/nexus already runs pkgs.testers.runNixOSTest in its gate.
  • Three existing checks made runtime-aware. credential-profiles' registry expectations describe the agenix deployment and are now scoped to libvirt machines; netrc-activation selects a libvirt dev machine by fact instead of by name and throws rather than going quiet when there is none; dev-forge-opt-out pins its fixtures to libvirt. Without these, the slice that first selects a microvm machine would find four red checks and have to work out which failures were real.
  • modules/github-credentials.nix takes its targets as an argument. One composition seam both runtimes read, and the only way a fixture can exercise decision 3's refusal.

Left for a later slice, stated rather than hidden

The issue asks for "a scan of the microvm guest closure for age identities and private key material". This gate never builds a machine — check.sh instantiates toplevel.drvPath and stops — and building a dev microvm guest closure is a whole NixOS system including the nixpkgs-unstable agent CLIs. What lands instead is a scan of the generated artifacts that would put anything in that closure: the combined activation script, every rendered unit, every environment.etc text and the account's Home Manager activation, checked for age --decrypt, an agenix activation entry, private-key material, and any reference to /etc/ssh/<machine>. The whole-closure scan belongs with the nested-boot slice, which builds a guest anyway. Closing allod/archetypes#29 is the reviewer's call once they accept that carry-over, which is why this PR only references it.

Risk

R3 High. Generated lifecycle behaviour — a boot service, systemd credentials, sshd's identity, and a secret-delivery boundary — but no deployed state is touched, no machine selects the runtime, every contract rule is exercised in fixtures or a boot, and rollback is a straight revert against an untouched libvirt path.

The worst credible failure after this validation is a real microvm guest that evaluates and boots but leaves a consumer unauthenticated, because the option value is right while the program reading it wants something else. What authenticates a private git+https flake input is the specific thing to scrutinise, and it is now measured rather than argued: at the pinned Nix 2.31.5 those inputs are fetched by shelling out to git — the failure is git's own fatal: could not read Username, not a libgit2 message — and adding only a credential helper to the calling user's git config turns that into fatal: Authentication failed, so the helper is consulted. The credential stores this change wires are therefore the authentication route for both root and the account, and the two per-user .netrc copies contract 9 forbids can be dropped rather than moved. The runtime netrc keeps /etc/nix/netrc's exact root:users 0640 for a different reason: netrc-file feeds Nix's own curl transfers — caches, tarballs, the registry — which the unprivileged client makes as well as the daemon.

Most useful human scrutiny: the rendered sshd_config, nix.conf, /etc/gitconfig, /etc/pam/environment and the materializer unit, then the negative fixtures.

Validation

./check.sh                     # 19 steps, all pass
  • Libvirt byte-identity. nixosConfigurations.{allod-dev,privacy-1,nexus,installer}.config.system.build.toplevel.drvPath compared against the parent commit: all four unchanged. This is the whole of the "the libvirt path is untouched" claim for the public template fleet, and it is not the fleet that matters — a deployment composes these modules against its own machine data. A verification request for the private fleet follows on the issue.
  • checks.x86_64-linux.microvm-sshd-boot. Two booted nodes. The delivered one starts sshd, presents a key whose fingerprint equals the delivered one, has no /etc/ssh/ssh_host_*, and holds the credential at 0400 in a 0500 directory. The one given nothing reaches multi-user.target with sshd.service inactive, no /etc/ssh/ssh_host_*, sshd-keygen.service masked, and no hostkeys available in its journal.
  • checks.x86_64-linux.microvm-credentials. Runs the production materializer generator with fixture inputs: every consumer file is checked byte-for-byte, mode and owner included, and the SSH key's exact trailing newline. Twelve refusals are shown, each pinned to its own diagnostic — missing, empty, not a regular file, two credential lines, a malformed credential URL, a token with a quote, a two-line token, a token carrying a carriage return, a public key where a private one belongs, a truncated key, a key with no trailing newline, and running outside a unit. Then 26 evaluation sabotages, each pinned to the assertion it names, and 7 generator rejections proved to fail but not pinned — they are lexical throws, and tryEval reports only that evaluation failed, and rendered-artifact reads of sshd_config, nix.conf, /etc/gitconfig, the account's git and ssh config, /etc/pam/environment and the materializer unit. sshd_config and nix.conf carry a paired sabotage of their own, because their failure mode is invisible to an option read; the other five are second witnesses for a value a module assertion already pins with its own sabotage. The nix.conf and /etc/gitconfig reads are diffs against the libvirt guest with the changed line removed, so a mkForce that ever drops something else goes red instead of passing a positive needle.

Rollback

Revert the commit. The libvirt modules, agenix wiring and netrc activation are unchanged throughout, no machine selects the microvm runtime, and nothing in this change runs on a host or touches a real credential.

A machine on the microVM runtime now has its private keys and tokens handed to it by the host at every boot instead of decrypting them itself. It holds no age identity, writes no credential onto anything that survives a reboot, and cannot quietly invent a new SSH identity when its host key fails to arrive. Nothing about a libvirt machine changes — all four example machines build to byte-identical derivations across this commit, and no machine moves to the microVM runtime here. What proves it: a two-node boot test shows a guest presenting exactly the key the host delivered with nothing written to disk, and a guest given no key leaving sshd failed with no key generated. The service that writes the other credentials is executed against canary credentials and shown refusing every malformed one. The whole repo gate passes. If this is wrong, revert the commit. Nothing here runs on the host, no deployed state is touched, and no machine selects the new runtime yet. Three things below are judgment calls a reviewer should either accept or send back: sshd bypasses the credential-writing service, the option is named differently from the parent plan, and a machine that declares GitHub credential targets is now refused rather than built. --- ## What this implements `Refs allod/archetypes#29` — the guest half of `allod/strategy#20` contracts 7, 8a, 9, 10, 11, 12 and 14, all in `allod/archetypes`. - **One runtime root.** `allod.archetypes.microvm.guestCredentialRoot` (`/run/allod/credentials` by default), validated as a normalized absolute string strictly below `/run/allod`. Every consumer path is derived from it and no literal is restated. - **A closed, derived credential-name set.** The SSH host key always; the Forge HTTPS credential, API token and SSH key only when the machine declares them, so the `forgeAccess` opt-out keeps meaning one thing on both runtimes. Exported as `allod.archetypes.microvm.credentialNames` for the host integration (contract 8b) to build `microvm.credentialFiles` from — this change sets no values. - **A materializer boot service**, not an activation script: it loads the credentials with systemd, validates each one's exact shape, and writes each consumer file atomically with a declared owner and mode. It carries no "missing is expected" branch. `nix-daemon.service` requires it, so a failed materializer fails its consumer visibly instead of starting one with no credentials. - **Runtime-rooted consumers**: sshd's `HostKey`, the Nix daemon's netrc, root's and the account's git credential stores, the Forge API token (both `FORGE_TOKEN_FILE` and `FORGEJO_TOKEN_FILE`), and the Forge SSH `IdentityFile` in the generated Home Manager output. - **No age in a microvm guest**: no `age.secrets`, no `age.identityPaths`, no `modules/netrc.nix`, no GitHub credential module. - **Assertions over the merged configuration** for every contract rule, each with a paired sabotage pinned to the diagnostic it names. ## Decisions a reviewer should accept or send back **1. sshd reads its credential directly; it does not go through the materializer.** The issue's handoff comment calls this an explicit fork and asks for a deliberate decision. Taken: `services.openssh.hostKeys = [ ]`, `sshd-keygen` masked, `LoadCredential=ssh-host-key` on `sshd.service`, `HostKey /run/credentials/sshd.service/ssh-host-key`. It writes strictly less plaintext than materializing a copy would — systemd's own credential directory is `0400 root:root` inside a `0500` directory on unswappable ramfs and goes away with the unit — and it deletes the ordering hazard the handoff names rather than working around it (`sshd.service` is `After=sshd-keygen.service` by default, and a materializer ordered before sshd must not inherit that). The cost, stated plainly: the issue's primary goal says *every* credential-derived plaintext file lives under the guest credential root, and this one does not. That reading cannot be literal — the materializer's own `LoadCredential` creates the same kind of directory, and so would any unit-scoped credential — but it is a goal-level deviation and it is recorded on the issue rather than left silent. A second cost: `ssh-host-key` is then the one credential no framework code shape-validates. A reviewer suggested loading it into the materializer for validation without writing a destination; declined, because that adds a second full-boot-lifetime plaintext copy of the machine's *identity* key to buy a nicer message than sshd's own `no hostkeys available -- exiting` plus a failed unit. **2. The option is `allod.archetypes.microvm.guestCredentialRoot`, not `allod.microvm.guestCredentialRoot`.** The plan and the issue write the latter. This repo already owns `allod.archetypes.microvm.volumeImageRoot` and `allod.archetypes.microvm.hostInterfaceId` under a namespace whose recorded reason is that it names the repo that owns it. A third spelling in the same feature area would be worse than either choice. Recorded on the issue so the plan can absorb it rather than be silently contradicted. **3. A microvm machine that declares GitHub credential targets fails evaluation.** Contract 9 says to point them at runtime files; there is no route to do that — their registry `deployed_path` values are durable, and per-target names would collide with contract 7's closed 28-character name set. Refusing beats building a machine without a credential it says it has. The public template declares none, so nothing in the example fleet changes. This is a contract inversion, recorded on the issue as interim. ## Beyond the issue's scope, deliberately - **A booted witness for contract 11** (`checks/microvm-sshd-boot.nix`). The issue says nothing here boots a guest; its handoff comment retracts that and prices the experiment. Two nodes prove both halves directly, and the nodes import the same production fragment a real machine composes rather than a restated copy. `allod/nexus` already runs `pkgs.testers.runNixOSTest` in its gate. - **Three existing checks made runtime-aware.** `credential-profiles`' registry expectations describe the agenix deployment and are now scoped to libvirt machines; `netrc-activation` selects a libvirt dev machine by fact instead of by name and throws rather than going quiet when there is none; `dev-forge-opt-out` pins its fixtures to libvirt. Without these, the slice that first selects a microvm machine would find four red checks and have to work out which failures were real. - **`modules/github-credentials.nix` takes its targets as an argument.** One composition seam both runtimes read, and the only way a fixture can exercise decision 3's refusal. ## Left for a later slice, stated rather than hidden The issue asks for "a scan of the microvm guest closure for age identities and private key material". This gate never builds a machine — `check.sh` instantiates `toplevel.drvPath` and stops — and building a dev microvm guest closure is a whole NixOS system including the `nixpkgs-unstable` agent CLIs. What lands instead is a scan of the generated artifacts that would put anything in that closure: the combined activation script, every rendered unit, every `environment.etc` text and the account's Home Manager activation, checked for `age --decrypt`, an agenix activation entry, private-key material, and any reference to `/etc/ssh/<machine>`. The whole-closure scan belongs with the nested-boot slice, which builds a guest anyway. Closing allod/archetypes#29 is the reviewer's call once they accept that carry-over, which is why this PR only references it. ## Risk **R3 High.** Generated lifecycle behaviour — a boot service, systemd credentials, sshd's identity, and a secret-delivery boundary — but no deployed state is touched, no machine selects the runtime, every contract rule is exercised in fixtures or a boot, and rollback is a straight revert against an untouched libvirt path. The worst credible failure after this validation is a real microvm guest that evaluates and boots but leaves a consumer unauthenticated, because the option value is right while the program reading it wants something else. What authenticates a private `git+https` flake input is the specific thing to scrutinise, and it is now measured rather than argued: at the pinned Nix 2.31.5 those inputs are fetched by shelling out to `git` — the failure is git's own `fatal: could not read Username`, not a libgit2 message — and adding only a credential helper to the calling user's git config turns that into `fatal: Authentication failed`, so the helper is consulted. The credential stores this change wires are therefore the authentication route for both root and the account, and the two per-user `.netrc` copies contract 9 forbids can be dropped rather than moved. The runtime netrc keeps `/etc/nix/netrc`'s exact `root:users 0640` for a different reason: `netrc-file` feeds Nix's own curl transfers — caches, tarballs, the registry — which the unprivileged client makes as well as the daemon. Most useful human scrutiny: the rendered `sshd_config`, `nix.conf`, `/etc/gitconfig`, `/etc/pam/environment` and the materializer unit, then the negative fixtures. ## Validation ``` ./check.sh # 19 steps, all pass ``` - **Libvirt byte-identity.** `nixosConfigurations.{allod-dev,privacy-1,nexus,installer}.config.system.build.toplevel.drvPath` compared against the parent commit: all four unchanged. This is the whole of the "the libvirt path is untouched" claim for the public template fleet, and it is not the fleet that matters — a deployment composes these modules against its own machine data. A verification request for the private fleet follows on the issue. - **`checks.x86_64-linux.microvm-sshd-boot`.** Two booted nodes. The delivered one starts sshd, presents a key whose fingerprint equals the delivered one, has no `/etc/ssh/ssh_host_*`, and holds the credential at `0400` in a `0500` directory. The one given nothing reaches `multi-user.target` with `sshd.service` inactive, no `/etc/ssh/ssh_host_*`, `sshd-keygen.service` masked, and `no hostkeys available` in its journal. - **`checks.x86_64-linux.microvm-credentials`.** Runs the production materializer generator with fixture inputs: every consumer file is checked byte-for-byte, mode and owner included, and the SSH key's exact trailing newline. Twelve refusals are shown, each pinned to its own diagnostic — missing, empty, not a regular file, two credential lines, a malformed credential URL, a token with a quote, a two-line token, a token carrying a carriage return, a public key where a private one belongs, a truncated key, a key with no trailing newline, and running outside a unit. Then 26 evaluation sabotages, each pinned to the assertion it names, and 7 generator rejections proved to fail but not pinned — they are lexical throws, and `tryEval` reports only that evaluation failed, and rendered-artifact reads of `sshd_config`, `nix.conf`, `/etc/gitconfig`, the account's git and ssh config, `/etc/pam/environment` and the materializer unit. `sshd_config` and `nix.conf` carry a paired sabotage of their own, because their failure mode is invisible to an option read; the other five are second witnesses for a value a module assertion already pins with its own sabotage. The `nix.conf` and `/etc/gitconfig` reads are diffs against the libvirt guest with the changed line removed, so a `mkForce` that ever drops something else goes red instead of passing a positive needle. ## Rollback Revert the commit. The libvirt modules, agenix wiring and netrc activation are unchanged throughout, no machine selects the microvm runtime, and nothing in this change runs on a host or touches a real credential.
A selected microvm archetype now receives its SSH host key, Forge HTTPS
credential, Forge API token and Forge SSH key from the host at every boot
instead of decrypting them itself. It composes no age secrets and no age
identity, writes every credential-derived file under one tmpfs runtime root,
and cannot regenerate its SSH host key: sshd reads the delivered key straight
out of systemd's credential directory and sshd-keygen is masked, so a boot
without the credential leaves sshd failed rather than presenting a new
identity.

The libvirt path is untouched. Every libvirt machine composes exactly the
modules it did before, in the same order, and all four example machines build
to byte-identical derivations across this commit. No machine selects the
microvm runtime here.

Two new checks. One runs the materializer against canary credentials and
requires it to refuse every malformed one; the other boots two nodes and shows
sshd presenting exactly the delivered key with nothing on disk, and failing
with no key generated when nothing is delivered.

Refs allod/archetypes#29
Refs allod/strategy#20
A verification pass re-ran every claim in the PR body against the tree and
found four defects.

The materializer's ordering and the Nix daemon's requirement on it were stated
by the module and read by a grep over the rendered unit, but asserted nowhere,
so the grep had nothing to pair against and either could have been dropped
with every remaining check green. Both are assertions over the merged unit
definitions now, each with a sabotage.

The vacuity guard on the age and private-key absence scan could not fail: it
grepped a two-line INI for an agenix needle and swallowed the miss. The libvirt
guest's same generated surfaces are rendered now and every needle must be
present there, so a renamed activation entry turns the absence proof red.
Private key material has no positive control and says so.

The gitconfig diff filter blanked every line containing the substring helper,
hiding a smuggled second helper line from the diff; it is anchored to the key
now.

The PR body claimed a paired sabotage for every rendered artifact when two of
seven had one, and claimed the boot test proves no host key anywhere when it
reads /etc/ssh. Both corrected there.

Refs allod/archetypes#29
Author
Member

Review pass: one false claim in this body, one check that could not fail, two properties nothing asserted

A read-only verification pass re-ran every claim in the PR body against the tree rather than reading it. Four findings, all now fixed in the follow-up commit; the body is corrected too.

1. The body claimed every rendered artifact had a paired sabotage. It did not — 2 of 7. Only sshd_config and nix.conf had one. /etc/gitconfig, the account's git and ssh config, /etc/pam/environment and the materializer unit did not. Four of those five are covered by a module assertion with its own sabotage, so the rendered read is a second witness rather than the only one, and that is what the body should have said. The materializer unit was the real gap: nothing asserted its ordering at all, so the Before= grep was the only evidence and had nothing to pair against. Likewise nothing read Requires= off nix-daemon.service, which the body claims as the mechanism that makes a failed materializer visible.

Fixed by giving both properties a module assertion over the merged unit definitions, each with a sabotage: emptying the materializer's before list, and emptying nix-daemon's requires. The rendered greps stay as the independent statement — the assertion catches a profile emptying the list, the grep catches the list being wrong in the first place.

2. The vacuity guard on the age/private-key absence scan could not fail. It grepped libvirtArtifacts.gitconfig — a two-line INI — for agenixInstall, and swallowed the miss with || true. So the four absence needles (age --decrypt, agenixInstall, BEGIN OPENSSH PRIVATE KEY, /etc/ssh/<machine>) were unwitnessed: a renamed agenix activation entry would have left the absence proof green over nothing.

Fixed by rendering the libvirt guest's same generated surfaces and requiring each needle to be present there. BEGIN OPENSSH PRIVATE KEY is excluded from that pairing and now says why in the check: the libvirt guest carries ciphertext and an activation that decrypts it, never a plaintext key, so there is no positive control for that one and it is an unpaired absence scan.

3. "no host key anywhere" in the boot test was overstated. The assertion is ls /etc/ssh/ssh_host_*. A key written outside /etc/ssh would not be caught. In practice sshd-keygen can only write the paths in services.openssh.hostKeys, which is empty and asserted so, and the test separately requires the unit to be masked — but the body's wording claimed more than the test does. Corrected to name the directory.

4. strip_helper in the /etc/gitconfig diff blanked every line containing the substring helper. A smuggled second helper line would have been invisible to the diff. It is anchored to the key now. The bare-store grep alongside it already covered that case, so this is tightening rather than a repair.

Also confirmed by execution rather than assertion: the four machine drvPaths are byte-identical between 7e83fbc and this branch; ./check.sh really is 19 steps and green; all four boot-test subtests ran; the ten materializer refusals, 24 evaluation sabotages and 6 generator rejections are really there; the runtime netrc really keeps root:users 0640; and the module really sets no microvm.credentialFiles values.

A measurement that replaces the biggest argument in this change

The riskiest claim here was that dropping the per-user and root .netrc copies does not break an unprivileged nix flake update against a private git+https flake input. That was reasoning, not evidence, and allod/memory nix.md currently says the opposite — that git+https goes through libgit2 and needs a netrc per calling user.

Measured at the pinned Nix 2.31.5. A git+https input with no credentials available fails with fatal: could not read Username for 'https://forge.anarch.diy' — git's own message, so Nix is shelling out to git, not using libgit2's own transport. Configuring only a credential helper in the calling user's git config and retrying changes the failure to fatal: Authentication failed, i.e. the helper was consulted and its credential was tried. So the credential store this change gives the account is a working authentication route for exactly the path nix flake update takes, and the account's netrc is not needed for it.

Someone already has a correction to that nix.md entry in flight on agent/nix-git-https-credentials, reaching the same conclusion from a PATH shim, so nothing is being changed here.

## Review pass: one false claim in this body, one check that could not fail, two properties nothing asserted A read-only verification pass re-ran every claim in the PR body against the tree rather than reading it. Four findings, all now fixed in the follow-up commit; the body is corrected too. **1. The body claimed every rendered artifact had a paired sabotage. It did not — 2 of 7.** Only `sshd_config` and `nix.conf` had one. `/etc/gitconfig`, the account's git and ssh config, `/etc/pam/environment` and the materializer unit did not. Four of those five are covered by a module assertion with its own sabotage, so the rendered read is a second witness rather than the only one, and that is what the body should have said. The materializer unit was the real gap: **nothing asserted its ordering at all**, so the `Before=` grep was the only evidence and had nothing to pair against. Likewise nothing read `Requires=` off `nix-daemon.service`, which the body claims as the mechanism that makes a failed materializer visible. Fixed by giving both properties a module assertion over the merged unit definitions, each with a sabotage: emptying the materializer's `before` list, and emptying `nix-daemon`'s `requires`. The rendered greps stay as the independent statement — the assertion catches a profile emptying the list, the grep catches the list being wrong in the first place. **2. The vacuity guard on the age/private-key absence scan could not fail.** It grepped `libvirtArtifacts.gitconfig` — a two-line INI — for `agenixInstall`, and swallowed the miss with `|| true`. So the four absence needles (`age --decrypt`, `agenixInstall`, `BEGIN OPENSSH PRIVATE KEY`, `/etc/ssh/<machine>`) were unwitnessed: a renamed agenix activation entry would have left the absence proof green over nothing. Fixed by rendering the libvirt guest's *same* generated surfaces and requiring each needle to be present there. `BEGIN OPENSSH PRIVATE KEY` is excluded from that pairing and now says why in the check: the libvirt guest carries ciphertext and an activation that decrypts it, never a plaintext key, so there is no positive control for that one and it is an unpaired absence scan. **3. "no host key anywhere" in the boot test was overstated.** The assertion is `ls /etc/ssh/ssh_host_*`. A key written outside `/etc/ssh` would not be caught. In practice `sshd-keygen` can only write the paths in `services.openssh.hostKeys`, which is empty and asserted so, and the test separately requires the unit to be masked — but the body's wording claimed more than the test does. Corrected to name the directory. **4. `strip_helper` in the `/etc/gitconfig` diff blanked every line containing the substring `helper`.** A smuggled second helper line would have been invisible to the diff. It is anchored to the key now. The bare-`store` grep alongside it already covered that case, so this is tightening rather than a repair. Also confirmed by execution rather than assertion: the four machine `drvPath`s are byte-identical between `7e83fbc` and this branch; `./check.sh` really is 19 steps and green; all four boot-test subtests ran; the ten materializer refusals, 24 evaluation sabotages and 6 generator rejections are really there; the runtime netrc really keeps `root:users 0640`; and the module really sets no `microvm.credentialFiles` values. ## A measurement that replaces the biggest argument in this change The riskiest claim here was that dropping the per-user and root `.netrc` copies does not break an unprivileged `nix flake update` against a private `git+https` flake input. That was reasoning, not evidence, and `allod/memory` `nix.md` currently says the opposite — that `git+https` goes through libgit2 and needs a netrc per calling user. Measured at the pinned Nix 2.31.5. A `git+https` input with no credentials available fails with `fatal: could not read Username for 'https://forge.anarch.diy'` — git's own message, so Nix is shelling out to `git`, not using libgit2's own transport. Configuring only a credential helper in the calling user's git config and retrying changes the failure to `fatal: Authentication failed`, i.e. the helper was consulted and its credential was tried. So the credential store this change gives the account is a working authentication route for exactly the path `nix flake update` takes, and the account's netrc is not needed for it. Someone already has a correction to that `nix.md` entry in flight on `agent/nix-git-https-credentials`, reaching the same conclusion from a PATH shim, so nothing is being changed here.
An adversarial review pass found that a machine with nothing to materialize
composed a materializer anyway, with an empty LoadCredential. systemd sets
CREDENTIALS_DIRECTORY only for a unit that declares at least one credential,
so the script's own "this must run as a unit" guard fired on every boot of
such a guest -- and through the Nix daemon's requirement on the materializer,
left that guest unable to run nix at all. The check asserted the shape and
never ran it. A machine that materializes nothing now composes no materializer,
the privacy assertion requires the unit to be absent, and the empty file table
is exercised against the generator rather than reasoned about.

Five smaller defects from the same pass.

The account's credential store lived in the root-owned runtime root, but
git-credential-store creates a lock file beside the credential file on every
successful authenticated HTTPS operation, so the account could not write it and
every authenticated fetch would have printed a lock error while appearing to
succeed. The account's three files live in a directory it owns now; the
materializer grew a declared directory table for it and refuses a destination
naming an undeclared directory.

The netrc rationale named the wrong mechanism in both the module and the PR
body: netrc-file feeds Nix's own curl transfers, and a git+https flake input is
fetched by shelling out to git, which authenticates through a credential helper.
Measured and corrected in place.

The API token validator stripped carriage returns before checking while the
installer wrote the raw bytes, and the private key validator matched a footer on
a file with no trailing newline that ssh rejects. Both validate the raw bytes
now, each with a refusal fixture.

credential-profiles went quiet rather than red when its runtime filters emptied
it, and treated the hypervisor's absent runtime fact as not-libvirt even though
mkHypervisor composes the netrc activation itself. Both corrected.

The HostKey guard was case-sensitive against a case-insensitive parser, at the
assertion and at the rendered-config count.

Refs allod/archetypes#29
Author
Member

Second review pass: one blocker, and it would have stranded every privacy microvm

An adversarial pass over the diff — verified against the pinned nixpkgs, live systemd and live git rather than by reading — found a blocker plus five smaller defects. All are fixed in the follow-up commit.

Blocker: a privacy microvm could not have run nix at all

A privacy machine materializes nothing, so its file table is empty and the unit rendered LoadCredential with no entries. systemd sets $CREDENTIALS_DIRECTORY only for a unit that declares at least one credential — verified live — so the materializer's own "this must run as a systemd unit" guard fired on every boot, before the empty-table branch it deliberately carried could run. Because nix-daemon.service requires the materializer, and every microvm guest enables the daemon through the writable store overlay, the first privacy microvm would have booted with a failed unit and no working nix.

The gate proved the broken shape was composed — the check asserted LoadCredential == [] — and never executed it. That is the exact failure mode this repo's check policy exists to prevent, and it is worth stating plainly: the check was written to read the shape rather than run it.

Fixed by not composing a materializer at all for a machine with nothing to materialize, which is the honest shape rather than a workaround. The privacy assertion now requires the unit to be absent and the Nix daemon to require nothing, and the empty file table is exercised behaviourally against the generator so the branch is run rather than reasoned about.

The account's git printed fatal: on every authenticated fetch

git-credential-store runs its store action on every successful authenticated HTTPS operation and creates a lock file beside the credential file. The account's store sat in the root-owned 0711 credential root, so the account could not create it: verified with the shipped git 2.51.2, the helper dies fatal: unable to get credential storage lock ... Permission denied while the parent git exits 0. Every nix flake update and git fetch against the private forge would have succeeded while printing an alarming line — new on microvm only, and on a machine whose whole job is running agents that read command output.

The module's own rationale for splitting root's and the account's stores described exactly this writer path and then left it broken for the account's own file. Fixed by giving the account a directory it owns (<root>/<username>/, 0700), which is where its three files now live. The materializer grew a declared directory table for it, the generator refuses a destination naming an undeclared directory, and the behavioural fixture now creates a lock file in that directory as part of the happy path.

The netrc justification was wrong-mechanism, in the PR body and in the module

The body and modules/microvm-credentials.nix claimed the runtime netrc is what authenticates the account's git+https flake-input fetches, and that git consults credential helpers before curl looks at a netrc. Both wrong. netrc-file feeds only Nix's internal curl — caches, tarballs, the registry — and git never reads it; and curl applies ~/.netrc on the first request while helpers run only after a 401.

The configuration is right for a different reason, now measured and recorded: git+https inputs are fetched by shelling out to git (the failure is git's own could not read Username, not a libgit2 message), and adding only a credential helper turns that into Authentication failed, so the credential stores this change wires are what authenticate them. The netrc keeps root:users 0640 because both the daemon and an unprivileged client make Nix's own curl transfers. Comment and body corrected to say that.

Two validators accepted bytes their consumers reject

The API-token validator stripped every carriage return before checking while the installer wrote the raw bytes, so a token with a CR passed at boot and failed at the first API call — precisely what the validator's comment says it exists to prevent. And tail -n 1 matched the OpenSSH footer even on a file with no final newline, which ssh rejects — the check's own comment documents that requirement one screen away. Both validate the raw bytes now, and both have a refusal fixture.

credential-profiles would have gone quiet, and its scoping was wrong for the hypervisor

Two problems in the runtime filtering added by the first commit. When the last agenix-deployed machine moves to microvm the filters empty, the mutation proof short-circuits, and the whole registry surface passes over nothing — silently, which is the state the same commit's netrc-activation edit throws on. There is a guard now that refuses an empty measurement while the registry still names machines this repo builds.

Second, a hypervisor is deliberately given no runtime fact by inventory, and mkHypervisor composes modules/netrc.nix itself, so it deploys through agenix exactly as a libvirt guest does. The filter read a missing fact as "not libvirt" and silently dropped it. It defaults to libvirt now, and the comments that said otherwise are corrected. Latent in the public template, which names no hypervisor registry entry.

Smaller

The HostKey guard was case-sensitive against a case-insensitive parser, at both layers — hostkey /etc/smuggled would have passed the assertion and the rendered-config count and been loaded by sshd. Both lowercase now.

Two assertions cannot be reached from any input a deployment can write (credentialNameErrors over this file's own literals, and the user-scoped-without-account guard). Neither is deleted: one is the only statement of the fw_cfg name budget anywhere in the stack. Both now say in the code that they carry no sabotage and why, rather than sitting among assertions that do.

The PR body claimed the six generator rejections were "pinned to the throw they name". They are lexical throws, so tryEval reports only that evaluation failed — the same limit runtime-module-selection records for its own negatives. Body corrected; the check says so too.

What the pass attacked and did not break

Recorded so the reviewer knows where the confidence comes from: sshd's credential path against the nixpkgs pin and the measured prototype; FORGE_TOKEN_FILE/FORGEJO_TOKEN_FILE against the two tools that actually read them; the ssh IdentityFile override against both the profiles hard-code and home-shared's mkDefault; that nothing in tools or profiles reads a .netrc, so the two dropped copies have no orphaned consumer; that modules/github-credentials.nix's signature change has no importer outside this flake; that nexus already names the credential ssh-host-key; and that the fw_cfg delivery in the boot check matches the pinned runner's rendering byte for byte.

## Second review pass: one blocker, and it would have stranded every privacy microvm An adversarial pass over the diff — verified against the pinned nixpkgs, live systemd and live git rather than by reading — found a blocker plus five smaller defects. All are fixed in the follow-up commit. ### Blocker: a privacy microvm could not have run nix at all A privacy machine materializes nothing, so its file table is empty and the unit rendered `LoadCredential` with no entries. **systemd sets `$CREDENTIALS_DIRECTORY` only for a unit that declares at least one credential** — verified live — so the materializer's own "this must run as a systemd unit" guard fired on every boot, before the empty-table branch it deliberately carried could run. Because `nix-daemon.service` requires the materializer, and every microvm guest enables the daemon through the writable store overlay, the first privacy microvm would have booted with a failed unit and no working `nix`. The gate proved the broken shape was composed — the check asserted `LoadCredential == []` — and never executed it. That is the exact failure mode this repo's check policy exists to prevent, and it is worth stating plainly: the check was written to read the shape rather than run it. Fixed by not composing a materializer at all for a machine with nothing to materialize, which is the honest shape rather than a workaround. The privacy assertion now requires the unit to be *absent* and the Nix daemon to require nothing, and the empty file table is exercised behaviourally against the generator so the branch is run rather than reasoned about. ### The account's git printed `fatal:` on every authenticated fetch `git-credential-store` runs its `store` action on every successful authenticated HTTPS operation and creates a lock file **beside** the credential file. The account's store sat in the root-owned `0711` credential root, so the account could not create it: verified with the shipped git 2.51.2, the helper dies `fatal: unable to get credential storage lock ... Permission denied` while the parent git exits 0. Every `nix flake update` and `git fetch` against the private forge would have succeeded while printing an alarming line — new on microvm only, and on a machine whose whole job is running agents that read command output. The module's own rationale for splitting root's and the account's stores described exactly this writer path and then left it broken for the account's own file. Fixed by giving the account a directory it owns (`<root>/<username>/`, `0700`), which is where its three files now live. The materializer grew a declared directory table for it, the generator refuses a destination naming an undeclared directory, and the behavioural fixture now creates a lock file in that directory as part of the happy path. ### The netrc justification was wrong-mechanism, in the PR body and in the module The body and `modules/microvm-credentials.nix` claimed the runtime netrc is what authenticates the account's `git+https` flake-input fetches, and that git consults credential helpers before curl looks at a netrc. Both wrong. `netrc-file` feeds only Nix's internal curl — caches, tarballs, the registry — and git never reads it; and curl applies `~/.netrc` on the *first* request while helpers run only after a 401. The configuration is right for a different reason, now measured and recorded: `git+https` inputs are fetched by shelling out to `git` (the failure is git's own `could not read Username`, not a libgit2 message), and adding only a credential helper turns that into `Authentication failed`, so the credential stores this change wires are what authenticate them. The netrc keeps `root:users 0640` because both the daemon and an unprivileged client make Nix's own curl transfers. Comment and body corrected to say that. ### Two validators accepted bytes their consumers reject The API-token validator stripped every carriage return before checking while the installer wrote the raw bytes, so a token with a CR passed at boot and failed at the first API call — precisely what the validator's comment says it exists to prevent. And `tail -n 1` matched the OpenSSH footer even on a file with no final newline, which ssh rejects — the check's own comment documents that requirement one screen away. Both validate the raw bytes now, and both have a refusal fixture. ### `credential-profiles` would have gone quiet, and its scoping was wrong for the hypervisor Two problems in the runtime filtering added by the first commit. When the last agenix-deployed machine moves to microvm the filters empty, the mutation proof short-circuits, and the whole registry surface passes over nothing — silently, which is the state the same commit's `netrc-activation` edit throws on. There is a guard now that refuses an empty measurement while the registry still names machines this repo builds. Second, a hypervisor is deliberately given no runtime fact by inventory, and `mkHypervisor` composes `modules/netrc.nix` itself, so it deploys through agenix exactly as a libvirt guest does. The filter read a missing fact as "not libvirt" and silently dropped it. It defaults to `libvirt` now, and the comments that said otherwise are corrected. Latent in the public template, which names no hypervisor registry entry. ### Smaller The `HostKey` guard was case-sensitive against a case-insensitive parser, at both layers — `hostkey /etc/smuggled` would have passed the assertion and the rendered-config count and been loaded by sshd. Both lowercase now. Two assertions cannot be reached from any input a deployment can write (`credentialNameErrors` over this file's own literals, and the user-scoped-without-account guard). Neither is deleted: one is the only statement of the fw_cfg name budget anywhere in the stack. Both now say in the code that they carry no sabotage and why, rather than sitting among assertions that do. The PR body claimed the six generator rejections were "pinned to the throw they name". They are lexical throws, so `tryEval` reports only that evaluation failed — the same limit `runtime-module-selection` records for its own negatives. Body corrected; the check says so too. ### What the pass attacked and did not break Recorded so the reviewer knows where the confidence comes from: sshd's credential path against the nixpkgs pin and the measured prototype; `FORGE_TOKEN_FILE`/`FORGEJO_TOKEN_FILE` against the two tools that actually read them; the ssh `IdentityFile` override against both the profiles hard-code and home-shared's `mkDefault`; that nothing in `tools` or `profiles` reads a `.netrc`, so the two dropped copies have no orphaned consumer; that `modules/github-credentials.nix`'s signature change has no importer outside this flake; that `nexus` already names the credential `ssh-host-key`; and that the fw_cfg delivery in the boot check matches the pinned runner's rendering byte for byte.
The comment claimed a repo-wide naming convention and cited allod/nexus as an
example of it. Nexus is the counterexample: it owns a bare nexus.*, with no
allod prefix, while allod/vm and allod/archetypes both use allod.<repo>.*.

The choice stands on the narrower ground the volumes slice actually recorded --
matching the two options immediately beside it in the same tree on the same
configuration. Why nexus differs is reconstruction, and now says so.

Refs allod/archetypes#29
vnprc approved these changes 2026-08-06 20:03:03 +01:00
vnprc merged commit 8982381e55 into master 2026-08-06 20:03:10 +01:00
vnprc deleted branch agent/microvm-guest-credentials 2026-08-06 20:03:10 +01:00
Sign in to join this conversation.
No description provided.