allod secret: declare, create, rotate, and migrate on credential value templates #208

Merged

vnprc merged 3 commits from agent/secret-value-templates into master

2026-09-12 02:24:16 +01:00

allod-agent commented

2026-09-12 01:59:51 +01:00

Member

Nothing changes on any machine until nexus moves its allod-tools pin; dev VMs never gain the host-only commands because they compile only behind the secret tag. On the host, allod secret stops carrying a copy of the credential vocabulary: declare writes a credential's non-secret half as a value.template (or no value for a plain secret) with an optional value.encode checked against the secrets flake's lib.credentialEncodings export and one verify command per target; create and rotate render the candidate secret through one shared renderer and never decrypt an existing value; the new migrate <name> translates one legacy container entry to the declared shape exactly once, proving the decrypted bytes round-trip through the proposed template before touching the registry. rotate keeps the whole reviewed landing gate from allod/tools#207 (clean branch, in-memory encryption, atomic writes, checks, branch re-check before commit, printed deploy, verify, and revocation steps even when the push fails) and refuses a legacy entry by naming migrate. The proof is the tagged and untagged test suites over fake age, nix, git, and rclone, with golden bytes for every rendering and every registry edit. This PR lands last, after the archetypes and secrets PRs. If it is wrong, revert it and move the pin back; rotate-token still rotates every legacy entry and no ciphertext this PR wrote exists until the owner runs a command.

What changes

allod secret declare (untagged): --format and the probe-type flags (--verify <probe>, --verify-repo-url, --verify-context, --verify-user) are gone. A credential's value is declared with --value-template-file <path> or --value-template-stdin (mutually exclusive, bytes stored untrimmed, exactly one {secret}), optionally --value-encode <name> checked against the checkout's lib.credentialEncodings, and --verify <command> is one non-empty line per target. The registry encoder no longer HTML-escapes, so < in a command stays readable.
create looks the credential up in the registry, refuses a legacy or mixed-shape entry by naming migrate, validates the declared value against the exported encodings, and renders the candidate secret through the same renderer rotate uses.
rotate is allod/tools#207's reviewed command with the format switch replaced by one rendering per group member from the one value read, and the enum validation replaced by the new contract. It never decrypts, refuses a legacy selected credential or group member by naming migrate, checks rclone is on PATH before reading the value when any member encodes, keeps the dry run, the branch re-checks, atomic writes, restore-all-then-report, group summary, deploy steps (including the rotate-token refresh-local-auth step), verification, and revocation gate. Verification prints the bare command for the local host and ssh '<system>' '<command>' with POSIX single-quote escaping otherwise.
New migrate <name> (tagged): checks everything it can without plaintext first (branch, clean tree, registry lookup, still-legacy, not mixed, not a local_auth_refresh source, one of the two containers, every target's verify object convertible, no unmodelled registry field), then decrypts once into memory, parses only the canonical bytes rotate-token writes, proves the extracted secret renders back to the original bytes and appears nowhere else in the template, splices only that credential object's bytes in forgejo-token-groups.json, runs the checks, commits, pushes. The rclone container's migrated declaration carries encode: "rclone-obscure"; the ciphertext is never rewritten. Verify objects become the inner commands rotate-token printed (git-ls-remote, forge-token-verify, site-check, tailscale-status); the legacy target user is baked into the command and dropped.
Shared secret_value.go: the renderer, value validation, the one encoder implementation (stdin only, output held to one base64url token), verify-string reading, and the registry lookup.
landCommit regains the branch re-verification immediately before git add and the report-push variant rotate uses to print its steps before a push failure is fatal.
Tests: the three test files torn out by the previous attempt are restored from the merged baseline and adapted; obsolete format-parser tests are dropped and the plan's acceptance coverage added: union decoding (all-new, all-legacy, mixed refused), golden rendering by both create and rotate for plain, URL, and encoded multiline values with and without a trailing newline, every new refusal, migration goldens and refusals with a whole-checkout scan proving plaintext reached no file, remote verification with embedded single quotes, and the untagged surface (declare present; create, rekey, migrate, rotate unknown).
docs/allod-secret.md, README.md, and the flake.nix tag comment describe the new command set and the verbatim-secret rule.
History: three commits, each green under every tag set. The first is allod/tools#207's three commits squashed and adapted to the merged declare dispatcher; the second is the value-template change; the third fixes the review findings. None is stacked on an open PR.

Risk

R3 in the plan (host-only decryption and authoritative repository writes). Residual after the tests: migrate's strict legacy parsers refuse a real ciphertext whose bytes are not the canonical shape rotate-token wrote, which is a refusal, not a wrong write; and a verbatim secret piped with a trailing newline lands inside a URL template, which the netrc activation and git credential store then reject loudly at the next rebuild. The first live run of each command should be --dry-run where it exists and be verified with the printed command before any old value is revoked. Rollback: revert this PR and the pin; a landed rotation or migration is a commit on a branch nobody merged.

Validation

All run by the overseer in the worktree on the committed tree (834ee26), each command's last line and exit status:

command	last line	exit
`nix develop -c gofmt -l .`	no output	0
`nix develop -c go vet ./...`	no output	0
`nix develop -c go test ./...`	`ok forge.anarch.diy/allod/tools/internal/gitremote 0.170s`	0
`nix develop -c go test -tags site ./...`	`ok forge.anarch.diy/allod/tools/internal/gitremote 0.140s`	0
`nix develop -c go test -tags secret ./...`	`ok forge.anarch.diy/allod/tools/internal/gitremote 0.145s`	0
`nix develop -c go test -tags site,secret ./...`	`ok forge.anarch.diy/allod/tools/internal/gitremote 0.168s`	0
`nix build --no-link .#checks.x86_64-linux.go-checks`	`building '/nix/store/vky3ixr643aqgly5599i6djfrcg8di1n-allod-tools-go-checks.drv'...`	0
`git diff --check`	no output	0

The worker also ran the seven nix develop commands on the first commit alone (1bb3406) and on the second (931ab6d) and reported them green; the overseer ran the full table above on 931ab6d as well before the review round.

Cold reviews of this diff before the third commit, each reading a .git-free snapshot and the diff against master: through the pi runner on provider nullsink, gpt-5.6-terra at thinking high (1 blocker, 3 high, 1 medium: encoder stderr echoed into a fatal error, migrate decrypting before checking the encoder export, unquoted legacy user/repo_url/deployed_path becoming shell text in a verify command, same-group duplicate entries passing the one-entry check, invalid UTF-8 templates silently altered by the JSON encoder) and, because gpt-5.6-sol at xhigh refused with a router balance error and codex was out of quota, a second pass by claude-fable-5-1 as a fresh Claude Code subagent (5 gaps, 1 simplify, 1 question: create reading stdin before proving rclone is on PATH, unknown verify fields and user dropped silently, the written template never validated, rotate misdirecting plain legacy entries to migrate, stale create help, non-atomic create/rekey writes). Every finding was verified against the source and fixed in the third commit (834ee26), each with a test; the golden for the baked user also caught the verify command being HTML-escaped on write, now fixed. The second pass is same-family with the fix author, recorded here as a deviation forced by runner availability. The fixes were authored by a Claude model, so the GPT pass is cross-family. A coverage audit of the previous attempt's deleted tests (70 tests deleted, none re-homed) drove the restoration; the audit and the worker brief are recorded in the overseer's session notes.

Known limit, deliberately left: age encrypt and decrypt errors still quote age's own stderr, the same class as the encoder finding. age does not echo its input, so no test can make it leak here; tightening it is a one-line follow-up if the owner wants the same rule for every subprocess handed a secret.

Not run: a live create, rotate, or migrate against real age, nix, and rclone on the host, because no agent machine holds an age identity. The first live run of each should be --dry-run where it exists, and be verified with the printed command before any old value is revoked.

Closes allod/secrets#23
Closes allod/tools#197
Refs allod/tools#182, allod/tools#207

Nothing changes on any machine until nexus moves its `allod-tools` pin; dev VMs never gain the host-only commands because they compile only behind the `secret` tag. On the host, `allod secret` stops carrying a copy of the credential vocabulary: `declare` writes a credential's non-secret half as a `value.template` (or no `value` for a plain secret) with an optional `value.encode` checked against the secrets flake's `lib.credentialEncodings` export and one `verify` command per target; `create` and `rotate` render the candidate secret through one shared renderer and never decrypt an existing value; the new `migrate <name>` translates one legacy container entry to the declared shape exactly once, proving the decrypted bytes round-trip through the proposed template before touching the registry. `rotate` keeps the whole reviewed landing gate from allod/tools#207 (clean branch, in-memory encryption, atomic writes, checks, branch re-check before commit, printed deploy, verify, and revocation steps even when the push fails) and refuses a legacy entry by naming `migrate`. The proof is the tagged and untagged test suites over fake `age`, `nix`, `git`, and `rclone`, with golden bytes for every rendering and every registry edit. This PR lands last, after the archetypes and secrets PRs. If it is wrong, revert it and move the pin back; `rotate-token` still rotates every legacy entry and no ciphertext this PR wrote exists until the owner runs a command. ## What changes - `allod secret declare` (untagged): `--format` and the probe-type flags (`--verify <probe>`, `--verify-repo-url`, `--verify-context`, `--verify-user`) are gone. A credential's value is declared with `--value-template-file <path>` or `--value-template-stdin` (mutually exclusive, bytes stored untrimmed, exactly one `{secret}`), optionally `--value-encode <name>` checked against the checkout's `lib.credentialEncodings`, and `--verify <command>` is one non-empty line per target. The registry encoder no longer HTML-escapes, so `<` in a command stays readable. - `create` looks the credential up in the registry, refuses a legacy or mixed-shape entry by naming `migrate`, validates the declared value against the exported encodings, and renders the candidate secret through the same renderer `rotate` uses. - `rotate` is allod/tools#207's reviewed command with the format switch replaced by one rendering per group member from the one value read, and the enum validation replaced by the new contract. It never decrypts, refuses a legacy selected credential or group member by naming `migrate`, checks `rclone` is on `PATH` before reading the value when any member encodes, keeps the dry run, the branch re-checks, atomic writes, restore-all-then-report, group summary, deploy steps (including the `rotate-token refresh-local-auth` step), verification, and revocation gate. Verification prints the bare command for the local host and `ssh '<system>' '<command>'` with POSIX single-quote escaping otherwise. - New `migrate <name>` (tagged): checks everything it can without plaintext first (branch, clean tree, registry lookup, still-legacy, not mixed, not a `local_auth_refresh` source, one of the two containers, every target's verify object convertible, no unmodelled registry field), then decrypts once into memory, parses only the canonical bytes rotate-token writes, proves the extracted secret renders back to the original bytes and appears nowhere else in the template, splices only that credential object's bytes in `forgejo-token-groups.json`, runs the checks, commits, pushes. The rclone container's migrated declaration carries `encode: "rclone-obscure"`; the ciphertext is never rewritten. Verify objects become the inner commands rotate-token printed (`git-ls-remote`, `forge-token-verify`, `site-check`, `tailscale-status`); the legacy target `user` is baked into the command and dropped. - Shared `secret_value.go`: the renderer, value validation, the one encoder implementation (stdin only, output held to one base64url token), verify-string reading, and the registry lookup. - `landCommit` regains the branch re-verification immediately before `git add` and the report-push variant `rotate` uses to print its steps before a push failure is fatal. - Tests: the three test files torn out by the previous attempt are restored from the merged baseline and adapted; obsolete format-parser tests are dropped and the plan's acceptance coverage added: union decoding (all-new, all-legacy, mixed refused), golden rendering by both `create` and `rotate` for plain, URL, and encoded multiline values with and without a trailing newline, every new refusal, migration goldens and refusals with a whole-checkout scan proving plaintext reached no file, remote verification with embedded single quotes, and the untagged surface (`declare` present; `create`, `rekey`, `migrate`, `rotate` unknown). - `docs/allod-secret.md`, `README.md`, and the `flake.nix` tag comment describe the new command set and the verbatim-secret rule. - History: three commits, each green under every tag set. The first is allod/tools#207's three commits squashed and adapted to the merged `declare` dispatcher; the second is the value-template change; the third fixes the review findings. None is stacked on an open PR. ## Risk R3 in the plan (host-only decryption and authoritative repository writes). Residual after the tests: `migrate`'s strict legacy parsers refuse a real ciphertext whose bytes are not the canonical shape rotate-token wrote, which is a refusal, not a wrong write; and a verbatim secret piped with a trailing newline lands inside a URL template, which the netrc activation and git credential store then reject loudly at the next rebuild. The first live run of each command should be `--dry-run` where it exists and be verified with the printed command before any old value is revoked. Rollback: revert this PR and the pin; a landed rotation or migration is a commit on a branch nobody merged. ## Validation All run by the overseer in the worktree on the committed tree (`834ee26`), each command's last line and exit status: | command | last line | exit | |---|---|---| | `nix develop -c gofmt -l .` | no output | 0 | | `nix develop -c go vet ./...` | no output | 0 | | `nix develop -c go test ./...` | `ok forge.anarch.diy/allod/tools/internal/gitremote 0.170s` | 0 | | `nix develop -c go test -tags site ./...` | `ok forge.anarch.diy/allod/tools/internal/gitremote 0.140s` | 0 | | `nix develop -c go test -tags secret ./...` | `ok forge.anarch.diy/allod/tools/internal/gitremote 0.145s` | 0 | | `nix develop -c go test -tags site,secret ./...` | `ok forge.anarch.diy/allod/tools/internal/gitremote 0.168s` | 0 | | `nix build --no-link .#checks.x86_64-linux.go-checks` | `building '/nix/store/vky3ixr643aqgly5599i6djfrcg8di1n-allod-tools-go-checks.drv'...` | 0 | | `git diff --check` | no output | 0 | The worker also ran the seven `nix develop` commands on the first commit alone (`1bb3406`) and on the second (`931ab6d`) and reported them green; the overseer ran the full table above on `931ab6d` as well before the review round. Cold reviews of this diff before the third commit, each reading a `.git`-free snapshot and the diff against master: through the `pi` runner on provider `nullsink`, `gpt-5.6-terra` at thinking `high` (1 blocker, 3 high, 1 medium: encoder stderr echoed into a fatal error, `migrate` decrypting before checking the encoder export, unquoted legacy `user`/`repo_url`/`deployed_path` becoming shell text in a verify command, same-group duplicate entries passing the one-entry check, invalid UTF-8 templates silently altered by the JSON encoder) and, because `gpt-5.6-sol` at `xhigh` refused with a router balance error and codex was out of quota, a second pass by `claude-fable-5-1` as a fresh Claude Code subagent (5 gaps, 1 simplify, 1 question: `create` reading stdin before proving rclone is on PATH, unknown verify fields and `user` dropped silently, the written template never validated, `rotate` misdirecting plain legacy entries to `migrate`, stale create help, non-atomic create/rekey writes). Every finding was verified against the source and fixed in the third commit (`834ee26`), each with a test; the golden for the baked `user` also caught the verify command being HTML-escaped on write, now fixed. The second pass is same-family with the fix author, recorded here as a deviation forced by runner availability. The fixes were authored by a Claude model, so the GPT pass is cross-family. A coverage audit of the previous attempt's deleted tests (70 tests deleted, none re-homed) drove the restoration; the audit and the worker brief are recorded in the overseer's session notes. Known limit, deliberately left: `age` encrypt and decrypt errors still quote `age`'s own stderr, the same class as the encoder finding. `age` does not echo its input, so no test can make it leak here; tightening it is a one-line follow-up if the owner wants the same rule for every subprocess handed a secret. Not run: a live `create`, `rotate`, or `migrate` against real `age`, `nix`, and `rclone` on the host, because no agent machine holds an age identity. The first live run of each should be `--dry-run` where it exists, and be verified with the printed command before any old value is revoked. Closes allod/secrets#23 Closes allod/tools#197 Refs allod/tools#182, allod/tools#207

allod-agent added 3 commits

2026-09-12 01:59:51 +01:00

allod secret rotate: replace a credential's value across its rotation group 1bb340680e

Adds the third landing shape beside create (first value) and rekey (new
recipients): rotate re-encrypts every credential in a registry group from
one value read on stdin, carries the non-secret fields of the two
structured formats forward, runs the repository's checks, commits, pushes,
and prints the deploy, verify, and revocation steps the registry describes.

This is the reviewed work from the superseded rotate branch, adapted to the
secret namespace's merged command table: create, rekey, and rotate register
themselves as entries in secretCommands rather than through a second
namespace registration and a second copy of the shared checkout helpers.
Nothing changes on any machine until nexus's allod-tools pin moves, and the
command compiles only behind the secret tag.

Refs allod/secrets#23

allod secret: declare, create, rotate, and migrate on credential value templates 931ab6d179

Replaces the three hand-named credential formats and the verify-probe enum
with one declared shape. A credential's non-secret text is a template around
one {secret} placeholder, an optional value.encode names a transformation
applied to the secret before substitution, and each target's verify is one
command string printed verbatim. Adding a kind of credential is now a
registry edit rather than a matching code change in three repositories.

declare takes --value-template-file or --value-template-stdin (bytes stored
without trimming), --value-encode, and --verify <command> in place of
--format and the structured verify flags. create and rotate render through
one shared renderer, so both produce the same bytes for the same
declaration, and rotate no longer decrypts anything: the non-secret half is
declared, not recovered. The encoder list comes from the checkout's own
lib.credentialEncodings; this build carries an implementation per exported
name and refuses one it cannot perform.

Adds the host-only secret-tagged migrate, which translates one legacy
container credential exactly once. It decrypts in memory after every check
that does not need the plaintext, accepts only the canonical bytes
rotate-token writes, proves the extracted secret substituted back reproduces
those bytes and occurs nowhere else in the template, and rewrites only that
credential's registry entry; the ciphertext is untouched. It refuses a
local_auth_refresh source until that consumer is migrated.

During coexistence a credential is wholly legacy or wholly new; the mix is
refused. rotate names migrate for a legacy container, and for a legacy
member of the group it was asked to rotate.

Refs allod/secrets#23

allod secret: refuse encoder stderr, unquoted legacy fields, duplicate entries, and invalid UTF-8 834ee2643b

A failing 'rclone obscure -' now reports its exit status alone, never its
own output, which could carry the candidate secret. 'migrate' proves the
checkout exports the encoder it would declare before it decrypts, holds
every legacy field that reaches a verify command to the grammar 'declare'
holds the same field to, decodes the legacy verify object with unknown
fields refused, keeps a 'user' only where forge-token-verify consumed it,
and validates the declaration it is about to write. A credential listed
twice in one group is refused by create, rotate, and migrate rather than
counted as one. A value template that is not UTF-8 is refused instead of
silently stored with replacement characters.

'create' proves rclone is on PATH before it reads the value, the legacy
refusals name a registry edit rather than 'migrate' for a format migrate
cannot open, and every command now writes through the one atomic helper.

Refs allod/secrets#23

vnprc force-pushed agent/secret-value-templates from 834ee2643b to f48399c5e5

2026-09-12 02:17:34 +01:00

Compare