new-vm --replace: graceful shutdown by default, hard kill behind a flag #72

New issue

Open

opened 2026-10-01 15:06:09 +01:00 by vnprc-agent · 0 comments

vnprc-agent commented

2026-10-01 15:06:09 +01:00

Make replacing a running guest safe for a stateful service VM: --replace shuts the guest down cleanly and waits, and the immediate virsh destroy it does today moves behind an explicit flag.

Primary goals:

Graceful default — --replace sends virsh shutdown, waits for the domain to reach shut off, then undefines and recreates as today.
Bounded wait, loud failure — a guest still running at the ceiling fails the run and names the force flag; the script never falls back to destroy on its own.
Hard kill on request — a flag, named per cli-design.md (--force or --hard), keeps today's immediate virsh destroy.
Unconditional — the rule holds whether or not the machine declares a data disk; a stateless guest pays a few seconds.

Current state and specifics

scripts/new-vm:210-212 on --replace: virsh destroy (silenced) then virsh undefine --nvram --storage vda; scripts/new-vm:217-218 does the same for a stale domain; scripts/new-vm:137 suggests the same destroy; undefine pair by hand when the domain exists without --replace. provision-vm passes --replace through to new-vm and reattaches a declared data disk after the install (scripts/provision-vm:150-157), so the disk survives but whatever was mid-write on it when the guest was killed does not.

For a guest running a pruned Bitcoin node that is the difference between a resumed sync and a full redownload: a hard kill mid-sync can leave the chainstate inconsistent, and a pruned node cannot reindex because the blocks it would need are gone. The node's unit allows ten minutes to flush on stop, so the wait ceiling must exceed that; a default of fifteen minutes, overridable by a flag or environment variable, covers it.

No guest-side change is needed: NixOS guests power off on the ACPI button virsh shutdown sends.

Validation

With a guest running, new-vm --replace <name> logs the shutdown, the domain reaches shut off before the undefine, and the data disk's filesystem mounts clean on the new machine.
With a guest that ignores the shutdown (a test domain with the power button masked), the run fails at the ceiling with a message naming the force flag and leaves the domain running.
With the force flag, behaviour is byte-for-byte today's.
tests/new-vm.sh covers the three paths.

Scope

In scope: scripts/new-vm, its usage text, tests/new-vm.sh, and the by-hand suggestion at scripts/new-vm:137. The same treatment for rebuild-vm, if it has a kill path, is a follow-up once this lands.

Make replacing a running guest safe for a stateful service VM: `--replace` shuts the guest down cleanly and waits, and the immediate `virsh destroy` it does today moves behind an explicit flag. Primary goals: - **Graceful default** — `--replace` sends `virsh shutdown`, waits for the domain to reach `shut off`, then undefines and recreates as today. - **Bounded wait, loud failure** — a guest still running at the ceiling fails the run and names the force flag; the script never falls back to `destroy` on its own. - **Hard kill on request** — a flag, named per `cli-design.md` (`--force` or `--hard`), keeps today's immediate `virsh destroy`. - **Unconditional** — the rule holds whether or not the machine declares a data disk; a stateless guest pays a few seconds. ### Current state and specifics `scripts/new-vm:210-212` on `--replace`: `virsh destroy` (silenced) then `virsh undefine --nvram --storage vda`; `scripts/new-vm:217-218` does the same for a stale domain; `scripts/new-vm:137` suggests the same `destroy; undefine` pair by hand when the domain exists without `--replace`. `provision-vm` passes `--replace` through to `new-vm` and reattaches a declared data disk after the install (`scripts/provision-vm:150-157`), so the disk survives but whatever was mid-write on it when the guest was killed does not. For a guest running a pruned Bitcoin node that is the difference between a resumed sync and a full redownload: a hard kill mid-sync can leave the chainstate inconsistent, and a pruned node cannot reindex because the blocks it would need are gone. The node's unit allows ten minutes to flush on stop, so the wait ceiling must exceed that; a default of fifteen minutes, overridable by a flag or environment variable, covers it. No guest-side change is needed: NixOS guests power off on the ACPI button `virsh shutdown` sends. ### Validation - With a guest running, `new-vm --replace <name>` logs the shutdown, the domain reaches `shut off` before the undefine, and the data disk's filesystem mounts clean on the new machine. - With a guest that ignores the shutdown (a test domain with the power button masked), the run fails at the ceiling with a message naming the force flag and leaves the domain running. - With the force flag, behaviour is byte-for-byte today's. - `tests/new-vm.sh` covers the three paths. ### Scope In scope: `scripts/new-vm`, its usage text, `tests/new-vm.sh`, and the by-hand suggestion at `scripts/new-vm:137`. The same treatment for `rebuild-vm`, if it has a kill path, is a follow-up once this lands.

vnprc-agent referenced this issue

2026-10-05 01:55:35 +01:00

provision-vm: log the old guest out of the tailnet before the wipe, so an ephemeral reprovision keeps its plain name #76