pi-provider assumes the hypervisor host key lives in machine-host-keys.json #29

New issue

Closed

opened 2026-08-22 05:31:24 +01:00 by vnprc-agent · 2 comments

vnprc-agent commented

2026-08-22 05:31:24 +01:00

pi-provider cannot encrypt for a deployment that keeps its hypervisor host key outside machine-host-keys.json.

collect_recipients seeds its machine list with the literal nexus and then requires a record for every machine in $MACHINE_HOST_KEYS, which defaults to ${IDENTITY_CONFIG}/machine-host-keys.json:

local -a machines=(nexus) keys=()
...
jq -e --arg machine "$machine" '.[$machine] | type == "object"' "$MACHINE_HOST_KEYS" >/dev/null ||
  die "no host recipient record for '${machine}'"

The public template carries the hypervisor in that one file, so this holds there. A deployment whose secrets repo keeps VM host keys in machine-host-keys.json and the hypervisor's own key in a separate registry — a reasonable split, since the two rotate through different commands with different gates — hits no host recipient record for 'nexus' on the first add, before any prompt.

The MACHINE_HOST_KEYS environment variable is a usable workaround: point it at a merged file assembled outside the tool. That works because the script only ever reads it. But it means every operator with this shape maintains a derived file by hand, out of band from the registries that own the data, and a stale one silently encrypts to the wrong recipient set.

Two smaller problems sit beside it:

The hypervisor name is a literal. A fleet whose hypervisor is not called nexus gets the same failure with a more confusing message. The name is already available — secrets.lib.nexusIdentity.hostname on the Nix side, and inventory knows which machine is type == "hypervisor".

The same PROFILES_CONFIG/IDENTITY_CONFIG/INVENTORY defaults resolve to the public template checkouts (allod/profiles, allod/secrets, allod/inventory). A deployment that redirects those inputs must override all three by hand or the tool stages private provider metadata and Age ciphertext into public working trees. It fails closed in practice — the public inventory does not declare the private targets — but the diagnostic points at the target, not at the resolution, and the failure is one edit away from being a boundary incident rather than an error message.

Suggestion

Derive the recipient machine set and the hypervisor name from the same sources the Nix contract uses, rather than from one file plus a literal. If a single merged host-key view is the right interface, secrets is the repo that should expose it, so the tool and mkPiCredentialContract agree by construction instead of by an operator remembering to regenerate a file.

`pi-provider` cannot encrypt for a deployment that keeps its hypervisor host key outside `machine-host-keys.json`. `collect_recipients` seeds its machine list with the literal `nexus` and then requires a record for every machine in `$MACHINE_HOST_KEYS`, which defaults to `${IDENTITY_CONFIG}/machine-host-keys.json`: ```sh local -a machines=(nexus) keys=() ... jq -e --arg machine "$machine" '.[$machine] | type == "object"' "$MACHINE_HOST_KEYS" >/dev/null || die "no host recipient record for '${machine}'" ``` The public template carries the hypervisor in that one file, so this holds there. A deployment whose secrets repo keeps VM host keys in `machine-host-keys.json` and the hypervisor's own key in a separate registry — a reasonable split, since the two rotate through different commands with different gates — hits `no host recipient record for 'nexus'` on the first `add`, before any prompt. The `MACHINE_HOST_KEYS` environment variable is a usable workaround: point it at a merged file assembled outside the tool. That works because the script only ever reads it. But it means every operator with this shape maintains a derived file by hand, out of band from the registries that own the data, and a stale one silently encrypts to the wrong recipient set. Two smaller problems sit beside it: The hypervisor name is a literal. A fleet whose hypervisor is not called `nexus` gets the same failure with a more confusing message. The name is already available — `secrets.lib.nexusIdentity.hostname` on the Nix side, and inventory knows which machine is `type == "hypervisor"`. The same `PROFILES_CONFIG`/`IDENTITY_CONFIG`/`INVENTORY` defaults resolve to the *public* template checkouts (`allod/profiles`, `allod/secrets`, `allod/inventory`). A deployment that redirects those inputs must override all three by hand or the tool stages private provider metadata and Age ciphertext into public working trees. It fails closed in practice — the public inventory does not declare the private targets — but the diagnostic points at the target, not at the resolution, and the failure is one edit away from being a boundary incident rather than an error message. ### Suggestion Derive the recipient machine set and the hypervisor name from the same sources the Nix contract uses, rather than from one file plus a literal. If a single merged host-key view is the right interface, `secrets` is the repo that should expose it, so the tool and `mkPiCredentialContract` agree by construction instead of by an operator remembering to regenerate a file.

vnprc-agent commented

2026-08-22 06:16:26 +01:00

Author

Two more undocumented assumptions in the same family, both hit while driving pi-provider against a real fork. Adding them here rather than opening separate issues, since the fix is the same idea: the tool's contract with the redirected repos is written down only inside the script.

Check attribute names are part of the contract

post_install_validation builds two checks by hardcoded attribute name:

nix build "path:${PROFILES_CONFIG}#checks.<system>.pi-provider-catalog" --no-link
nix build "path:${IDENTITY_CONFIG}#checks.<system>.pi-credential-registry" --no-link

A redirected repo that names its equivalent check anything else — or has none — takes the bearer at the prompt, encrypts it, then fails and rolls back:

error: flake 'path:/…/profiles' does not provide attribute '…checks.x86_64-linux.pi-provider-catalog'
Error: operation failed; restored the recorded pre-run paths

The rollback is correct and nothing is written, but nothing announces the requirement beforehand. A --dry-run does not surface it either, because the dry run stops before this stage. Checking for both attributes during preflight, beside the registry and inventory checks, would move the failure ahead of the prompt.

The public template's own check makes `add` self-defeating

allod/secrets' pi-credential-registry asserts the committed registry is empty:

assert lib.assertMsg (piCredentialContract.registry == {}) …

post_install_validation builds that check after writing a registry entry. On the template itself, the first successful add therefore fails its own validation and rolls back. A fork only avoids this by writing a different check under the same name.

The tool does not write `secrets.nix`

add writes the registry and the ciphertext but not the agenix recipient declaration. The credential contract compares declared recipients against derived ones, so the repo stops evaluating immediately after a successful add:

error: pi-credential-registry: declared Pi recipients drift from the credential registry

The fix is a one-line entry, but the required list is order-sensitive — the contract compares publicKeys as a list, so it must be nexus keys followed by targets in sorted order. That ordering is not documented anywhere, and getting it wrong reproduces the same error with no hint that only the order is wrong.

This also interacts with the check-name requirement above: a fork's pi-credential-registry check cannot force the committed contract, because the tool builds it in exactly the window where the registry is written and secrets.nix is not yet updated. So the one check the tool insists on is the one that cannot gate the data it just wrote.

Either the tool should write the recipient entry, or the derivation should be exposed so a repo can compute it rather than transcribe it.

Two more undocumented assumptions in the same family, both hit while driving `pi-provider` against a real fork. Adding them here rather than opening separate issues, since the fix is the same idea: the tool's contract with the redirected repos is written down only inside the script. ### Check attribute names are part of the contract `post_install_validation` builds two checks by hardcoded attribute name: ```sh nix build "path:${PROFILES_CONFIG}#checks.<system>.pi-provider-catalog" --no-link nix build "path:${IDENTITY_CONFIG}#checks.<system>.pi-credential-registry" --no-link ``` A redirected repo that names its equivalent check anything else — or has none — takes the bearer at the prompt, encrypts it, then fails and rolls back: ``` error: flake 'path:/…/profiles' does not provide attribute '…checks.x86_64-linux.pi-provider-catalog' Error: operation failed; restored the recorded pre-run paths ``` The rollback is correct and nothing is written, but nothing announces the requirement beforehand. A `--dry-run` does not surface it either, because the dry run stops before this stage. Checking for both attributes during preflight, beside the registry and inventory checks, would move the failure ahead of the prompt. ### The public template's own check makes `add` self-defeating `allod/secrets`' `pi-credential-registry` asserts the committed registry is empty: ```nix assert lib.assertMsg (piCredentialContract.registry == {}) … ``` `post_install_validation` builds that check *after* writing a registry entry. On the template itself, the first successful `add` therefore fails its own validation and rolls back. A fork only avoids this by writing a different check under the same name. ### The tool does not write `secrets.nix` `add` writes the registry and the ciphertext but not the agenix recipient declaration. The credential contract compares declared recipients against derived ones, so the repo stops evaluating immediately after a successful add: ``` error: pi-credential-registry: declared Pi recipients drift from the credential registry ``` The fix is a one-line entry, but the required list is order-sensitive — the contract compares `publicKeys` as a list, so it must be nexus keys followed by targets in sorted order. That ordering is not documented anywhere, and getting it wrong reproduces the same error with no hint that only the order is wrong. This also interacts with the check-name requirement above: a fork's `pi-credential-registry` check cannot force the committed contract, because the tool builds it in exactly the window where the registry is written and `secrets.nix` is not yet updated. So the one check the tool insists on is the one that cannot gate the data it just wrote. Either the tool should write the recipient entry, or the derivation should be exposed so a repo can compute it rather than transcribe it.

vnprc-agent commented

2026-08-24 04:22:48 +01:00

Author

Reproduced during a named-token rollout after rebuilding the host with merged Nexus revision 0a9b30e.

The documented plain command fails before bearer input:

$ pi-provider token add <credential> standby
Error: provider registry not found: /home/operator/work/allod/profiles/pi-providers.json

The deployment inventory redirects its private profiles checkout to $HOME/work/profiles, but pi-provider resolves the allod/profiles alias instead. The host configuration supplies INVENTORY; the installed command has no wrapper or declarative inputs supplying PROFILES_CONFIG, IDENTITY_CONFIG, or the deployment’s split hypervisor recipient record. This confirms that the current workaround requires both explicit repository overrides and an externally merged host-key view, and that the documented plain CLI is not usable for this deployment shape.

Reproduced during a named-token rollout after rebuilding the host with merged Nexus revision `0a9b30e`. The documented plain command fails before bearer input: ```text $ pi-provider token add <credential> standby Error: provider registry not found: /home/operator/work/allod/profiles/pi-providers.json ``` The deployment inventory redirects its private `profiles` checkout to `$HOME/work/profiles`, but `pi-provider` resolves the `allod/profiles` alias instead. The host configuration supplies `INVENTORY`; the installed command has no wrapper or declarative inputs supplying `PROFILES_CONFIG`, `IDENTITY_CONFIG`, or the deployment’s split hypervisor recipient record. This confirms that the current workaround requires both explicit repository overrides and an externally merged host-key view, and that the documented plain CLI is not usable for this deployment shape.