allod/strategy

Watch

Fork 0

Explore keeping secrets outside agent-controlled environments #13

New issue

Open

opened 2026-07-10 23:50:23 +01:00 by allod-agent · 1 comment

allod-agent commented

2026-07-10 23:50:23 +01:00

Member

User story: So that I can let agents use authorized services without handing them durable secrets, Allod needs an architecture path for brokering service access outside agent-controlled environments.

Part of the "Agent isolation & security boundary" arc.

Context

Allod already reduces agent exposure by separating public template repos from private identity and credential repos, constraining the allod-dev VM to public checkouts, and treating host-side secret work as a human gate. That is a strong baseline, but agents still need to use external services, and the default way to do that is often to place a usable credential somewhere inside the agent-controlled environment.

This issue is primarily inspired by Netclode's secret-proxy architecture:

Netclode is useful here as a comparison point because its design keeps provider API keys outside the sandbox and mediates access through services outside the agent VM. This issue is not proposing that Allod copy Netclode's Kubernetes, MITM proxy, or service layout wholesale. The architectural goal is broader: agents should be able to use authorized services without possessing the underlying long-lived secrets.

Related comparison context:

microvm.nix: https://github.com/microvm-nix/microvm.nix
Michael Stapelberg's coding-agent MicroVM setup: https://michael.stapelberg.ch/posts/2026-02-01-coding-agent-microvm-nix/

Those projects reinforce the value of isolating agent compute, but this issue is specifically about reducing secret material inside that compute boundary.

Goal

Explore an Allod architecture where a compromised agent VM has less durable credential material to steal or reuse. Agents may still need to trigger Git, Forgejo, model-provider, package-registry, deployment, or other service operations, but the long-lived credentials should ideally remain in a host-controlled, service-controlled, or otherwise narrower trust domain.

Questions to explore

Which credentials currently need to enter an agent-controlled VM, and which are only there because there is no brokered access path yet?
What should count as the secret boundary in Allod: the Nix store closure, the VM filesystem, process environment, Home Manager activation output, Forgejo identity, or a future host-side service?
Can Allod distinguish long-lived credentials from short-lived or session-scoped authority in a way that fits the existing NixOS and age/agenix model?
What audit signal would prove that a given service credential did not enter the agent environment, logs, shell history, or persistent workspace?
How should failures behave when a brokered service refuses a request: clear denial, human approval path, capability escalation request, or something else?

Non-goals for now

Choosing a concrete proxy, token exchange, socket, network, or control-plane implementation.
Replacing age/agenix or the existing private secrets authority.
Broadening what agents can do before the authority model is understood.
Treating Netclode's exact design as the target architecture.

**User story:** So that I can let agents use authorized services without handing them durable secrets, Allod needs an architecture path for brokering service access outside agent-controlled environments. _Part of the "Agent isolation & security boundary" arc._ --- ## Context Allod already reduces agent exposure by separating public template repos from private identity and credential repos, constraining the `allod-dev` VM to public checkouts, and treating host-side secret work as a human gate. That is a strong baseline, but agents still need to use external services, and the default way to do that is often to place a usable credential somewhere inside the agent-controlled environment. This issue is primarily inspired by Netclode's secret-proxy architecture: - https://github.com/angristan/netclode - https://github.com/angristan/netclode/blob/master/docs/secret-proxy.md - https://stanislas.blog/2026/02/netclode-self-hosted-cloud-coding-agent/ Netclode is useful here as a comparison point because its design keeps provider API keys outside the sandbox and mediates access through services outside the agent VM. This issue is not proposing that Allod copy Netclode's Kubernetes, MITM proxy, or service layout wholesale. The architectural goal is broader: agents should be able to use authorized services without possessing the underlying long-lived secrets. Related comparison context: - microvm.nix: https://github.com/microvm-nix/microvm.nix - Michael Stapelberg's coding-agent MicroVM setup: https://michael.stapelberg.ch/posts/2026-02-01-coding-agent-microvm-nix/ Those projects reinforce the value of isolating agent compute, but this issue is specifically about reducing secret material inside that compute boundary. ## Goal Explore an Allod architecture where a compromised agent VM has less durable credential material to steal or reuse. Agents may still need to trigger Git, Forgejo, model-provider, package-registry, deployment, or other service operations, but the long-lived credentials should ideally remain in a host-controlled, service-controlled, or otherwise narrower trust domain. ## Questions to explore - Which credentials currently need to enter an agent-controlled VM, and which are only there because there is no brokered access path yet? - What should count as the secret boundary in Allod: the Nix store closure, the VM filesystem, process environment, Home Manager activation output, Forgejo identity, or a future host-side service? - Can Allod distinguish long-lived credentials from short-lived or session-scoped authority in a way that fits the existing NixOS and age/agenix model? - What audit signal would prove that a given service credential did not enter the agent environment, logs, shell history, or persistent workspace? - How should failures behave when a brokered service refuses a request: clear denial, human approval path, capability escalation request, or something else? ## Non-goals for now - Choosing a concrete proxy, token exchange, socket, network, or control-plane implementation. - Replacing age/agenix or the existing private `secrets` authority. - Broadening what agents can do before the authority model is understood. - Treating Netclode's exact design as the target architecture.

allod-agent commented

2026-07-28 19:14:00 +01:00

Author

Member

The injection blocker has a named upstream answer, so this issue is not gated on it

A separate concern — that moving to microvm.nix has no way to deliver a VM's SSH host key, since there is no installer and a key cannot be baked into a world-readable Nix store path — has an existing mechanism: microvm.credentialFiles in microvm.nix, which emits -fw_cfg name=opt/io.systemd.credentials/<name>,file=<path> and surfaces the value in the guest at /run/credentials/@system/<name> on a noswap tmpfs. Only the path enters the Nix store, not the secret. Constraints worth recording now: the credential name is capped at 28 characters, the option value must be a quoted absolute string rather than a Nix path literal or Nix copies the secret into the store, and the transport is QEMU-only — every other microvm.nix runner throws, including cloud-hypervisor.

Whether PID 1 actually receives these on the nixpkgs 25.11 line is unverified and fails silently if it does not, because CONFIG_FW_CFG_SYSFS is built as a module. That is being spiked separately. The point for this issue is only that the mechanism exists and is not a prerequisite here.

The secret boundary question, answered by capability rather than preference

This issue asks what should count as the secret boundary: the Nix store closure, the VM filesystem, process environment, Home Manager activation output, Forgejo identity, or a future host-side service. The dev VM's own configuration settles part of it. vm/modules/qemu-guest.nix puts the VM user in wheel with wheelNeedsPassword = false, and archetypes/modules/ai-agents.nix launches agents with permission prompts disabled. Verified live in a dev VM: the Forgejo PR token and the forge SSH key are readable by the agent's own user with no sudo at all, and /etc/ssh/<vm> (0600 root), the DMI tables (0400 root), and /run/credentials/* (0500 root) are each one sudo away. systemd also cannot scrub an imported credential afterwards, since sysfs is kernel-owned, so it remains root-readable for the whole boot.

So no in-guest mechanism can be justified on the agent being unable to read the secret — that is precisely the cooperation-dependent boundary principle 4 forbids. Anything claiming otherwise is a false boundary a future reviewer might re-derive, which is why it is worth writing down here.

What actually discriminates between designs is therefore host-side and authority-side:

Axis	Question	Owner
Nix store cleartext	does the secret ever enter a store path?	this decision
Host process table	does it appear in argv or `ps`?	this decision
Durable host disk	does it land anywhere but a ramfs?	this decision
New host-side surface	does it add host code parsing guest input?	this decision
Durable authority of the payload	how long is the stolen material useful?	this issue

The first four are mechanism questions with clear answers. The fifth is the one this issue owns, and it is the only one that survives the agent being root.

`microvm.vsock.cid` is the single-host substitute for Netclode's identity primitive

Netclode's secret-proxy authenticates the caller with a Kubernetes projected ServiceAccount token, audience-scoped and refreshed every five minutes, validated by TokenReview. That primitive is the k8s API server, which is why the design does not port directly to a single-host stack.

microvm.vsock.cid looks like the replacement: the CID is assigned by the host at VM start and is not settable by the guest, so a broker listening on vsock knows which VM is calling without the VM presenting any credential at all. That would satisfy this issue's "brokered access without durable secrets" goal using a primitive the host already controls, with no proxy identity material inside the VM. Flagging it as unverified — that the CID cannot be forged or spoofed from inside the guest needs checking against the vhost-vsock driver before anything is designed on top of it.

Two corrections to how the reference designs should be read

Netclode does not broker git. Its secret-proxy keeps model-provider API keys out of the sandbox through header-only substitution with a placeholder value, and explicitly never rewrites request bodies. But the GitHub credential is a GitHub App installation token handed to the sandbox as GITHUB_TOKEN, mitigated by repo scoping and a one-hour TTL — by narrowing, not by absence. So brokering git or Forgejo push would put Allod ahead of the reference design rather than level with it, and should be scoped as its own harder problem rather than assumed to come along with the proxy pattern.

Stapelberg's microVM setup is an isolation template, not a secrets template. Its entire credential mechanism is a read-write virtiofs share of a host directory, with the same host source for every VM. That is the option Allod must reject outright, since principle 7 says "no shared folders" verbatim and microvm.nix's microvm-virtiofsd@ unit runs with no User= — a root-privileged host FUSE parser driven by an untrusted guest, against principles 4 and 6. The topology, the tmpfs root, and storeOnDisk are reusable; the credential and network story is not.

Privacy VMs are already an existence proof

mkPrivacyVm in archetypes/flake.nix declares zero age.secrets, and forge_key = null makes the forge-key bootstrap a no-op. A privacy VM's host key decrypts nothing — it exists only for SSH host authentication. So the "agent VM with no durable service credentials inside it" end state already exists in the stack for one archetype; the question this issue asks is what it takes to reach it for dev VMs.

One clarification on causation

It is tempting to conclude that brokering is what would let a VM stop holding a long-lived age identity. It is not. agenix runs in the guest because provisioning is one-shot: the guest must re-derive its secrets at every activation with no help available. Under microvm.nix the host is present at every start and can hand over final plaintext, at which point an in-guest age layer is redundant because the delivery channel is already exactly as trusted as the key it would deliver.

That means per-boot delivery retires the in-guest age identity with or without this issue, and this issue's contribution is orthogonal and more durable: it removes the authority of the material, not merely its persistence. Both are worth having, and they should not be sequenced as if one depends on the other.

On the audit-signal question

This issue asks what would prove a given credential did not enter the agent environment. A compromised guest attests nothing about itself, so the provable claim is host-side and structural: the declared system, as generated, contains no path that places the credential there — principle 13's "believe generated artifacts, not source review."

Concretely that is an assertion that the credential is absent from config.age.secrets, that its deployed path appears nowhere in the generated activate script or unit set, and that no private-key or age-key material appears anywhere in the closure. Per principle 11's "validation is itself validated," the check needs a fixture that does deploy the credential, with an assertion that the check trips on it. The repo already has that pattern in two places — the mutated-registry checks and the vmFacts negative check in archetypes/flake.nix both build sabotaged inputs and assert each mutation errors — so this would follow an established shape rather than invent one.

Runtime evidence stays weaker and host-side by necessity: the broker logged a request from CID N and never emitted the secret. That is the honest ceiling, and worth stating in the design so it is not mistaken for guest-side attestation.

Findings from investigating microvm.nix secret injection, which turned out to bear directly on several of the questions above. Recording them here so this issue is not blocked on work it does not actually depend on. ## The injection blocker has a named upstream answer, so this issue is not gated on it A separate concern — that moving to microvm.nix has no way to deliver a VM's SSH host key, since there is no installer and a key cannot be baked into a world-readable Nix store path — has an existing mechanism: `microvm.credentialFiles` in microvm.nix, which emits `-fw_cfg name=opt/io.systemd.credentials/<name>,file=<path>` and surfaces the value in the guest at `/run/credentials/@system/<name>` on a `noswap` tmpfs. Only the path enters the Nix store, not the secret. Constraints worth recording now: the credential name is capped at 28 characters, the option value must be a quoted absolute string rather than a Nix path literal or Nix copies the secret into the store, and the transport is QEMU-only — every other microvm.nix runner throws, including cloud-hypervisor. Whether PID 1 actually receives these on the nixpkgs 25.11 line is unverified and fails silently if it does not, because `CONFIG_FW_CFG_SYSFS` is built as a module. That is being spiked separately. The point for this issue is only that the mechanism exists and is not a prerequisite here. ## The secret boundary question, answered by capability rather than preference This issue asks what should count as the secret boundary: the Nix store closure, the VM filesystem, process environment, Home Manager activation output, Forgejo identity, or a future host-side service. The dev VM's own configuration settles part of it. `vm/modules/qemu-guest.nix` puts the VM user in `wheel` with `wheelNeedsPassword = false`, and `archetypes/modules/ai-agents.nix` launches agents with permission prompts disabled. Verified live in a dev VM: the Forgejo PR token and the forge SSH key are readable by the agent's own user with no `sudo` at all, and `/etc/ssh/<vm>` (0600 root), the DMI tables (0400 root), and `/run/credentials/*` (0500 root) are each one `sudo` away. systemd also cannot scrub an imported credential afterwards, since sysfs is kernel-owned, so it remains root-readable for the whole boot. So **no in-guest mechanism can be justified on the agent being unable to read the secret** — that is precisely the cooperation-dependent boundary principle 4 forbids. Anything claiming otherwise is a false boundary a future reviewer might re-derive, which is why it is worth writing down here. What actually discriminates between designs is therefore host-side and authority-side: | Axis | Question | Owner | |---|---|---| | Nix store cleartext | does the secret ever enter a store path? | this decision | | Host process table | does it appear in argv or `ps`? | this decision | | Durable host disk | does it land anywhere but a ramfs? | this decision | | New host-side surface | does it add host code parsing guest input? | this decision | | Durable authority of the payload | how long is the stolen material useful? | **this issue** | The first four are mechanism questions with clear answers. The fifth is the one this issue owns, and it is the only one that survives the agent being root. ## `microvm.vsock.cid` is the single-host substitute for Netclode's identity primitive Netclode's secret-proxy authenticates the caller with a Kubernetes projected ServiceAccount token, audience-scoped and refreshed every five minutes, validated by TokenReview. That primitive *is* the k8s API server, which is why the design does not port directly to a single-host stack. `microvm.vsock.cid` looks like the replacement: the CID is assigned by the host at VM start and is not settable by the guest, so a broker listening on vsock knows which VM is calling without the VM presenting any credential at all. That would satisfy this issue's "brokered access without durable secrets" goal using a primitive the host already controls, with no proxy identity material inside the VM. Flagging it as unverified — that the CID cannot be forged or spoofed from inside the guest needs checking against the vhost-vsock driver before anything is designed on top of it. ## Two corrections to how the reference designs should be read **Netclode does not broker git.** Its secret-proxy keeps model-provider API keys out of the sandbox through header-only substitution with a placeholder value, and explicitly never rewrites request bodies. But the GitHub credential is a GitHub App installation token handed to the sandbox as `GITHUB_TOKEN`, mitigated by repo scoping and a one-hour TTL — by *narrowing*, not by *absence*. So brokering git or Forgejo push would put Allod ahead of the reference design rather than level with it, and should be scoped as its own harder problem rather than assumed to come along with the proxy pattern. **Stapelberg's microVM setup is an isolation template, not a secrets template.** Its entire credential mechanism is a read-write virtiofs share of a host directory, with the same host source for every VM. That is the option Allod must reject outright, since principle 7 says "no shared folders" verbatim and microvm.nix's `microvm-virtiofsd@` unit runs with no `User=` — a root-privileged host FUSE parser driven by an untrusted guest, against principles 4 and 6. The topology, the tmpfs root, and `storeOnDisk` are reusable; the credential and network story is not. ## Privacy VMs are already an existence proof `mkPrivacyVm` in `archetypes/flake.nix` declares zero `age.secrets`, and `forge_key = null` makes the forge-key bootstrap a no-op. A privacy VM's host key decrypts nothing — it exists only for SSH host authentication. So the "agent VM with no durable service credentials inside it" end state already exists in the stack for one archetype; the question this issue asks is what it takes to reach it for dev VMs. ## One clarification on causation It is tempting to conclude that brokering is what would let a VM stop holding a long-lived age identity. It is not. agenix runs in the guest because provisioning is one-shot: the guest must re-derive its secrets at every activation with no help available. Under microvm.nix the host is present at *every start* and can hand over final plaintext, at which point an in-guest age layer is redundant because the delivery channel is already exactly as trusted as the key it would deliver. That means per-boot delivery retires the in-guest age identity **with or without this issue**, and this issue's contribution is orthogonal and more durable: it removes the *authority* of the material, not merely its persistence. Both are worth having, and they should not be sequenced as if one depends on the other. ## On the audit-signal question This issue asks what would prove a given credential did not enter the agent environment. A compromised guest attests nothing about itself, so the provable claim is host-side and structural: **the declared system, as generated, contains no path that places the credential there** — principle 13's "believe generated artifacts, not source review." Concretely that is an assertion that the credential is absent from `config.age.secrets`, that its deployed path appears nowhere in the generated `activate` script or unit set, and that no private-key or age-key material appears anywhere in the closure. Per principle 11's "validation is itself validated," the check needs a fixture that *does* deploy the credential, with an assertion that the check trips on it. The repo already has that pattern in two places — the mutated-registry checks and the `vmFacts` negative check in `archetypes/flake.nix` both build sabotaged inputs and assert each mutation errors — so this would follow an established shape rather than invent one. Runtime evidence stays weaker and host-side by necessity: the broker logged a request from CID *N* and never emitted the secret. That is the honest ceiling, and worth stating in the design so it is not mistaken for guest-side attestation.

allod-agent referenced this issue

2026-07-28 20:25:41 +01:00

Adopt microvm.nix as a VM runtime in the framework #20

allod-agent referenced this issue

2026-07-28 20:27:08 +01:00

Add public dev plan for microvm.nix framework adoption #21