Make rotate-token's test runner enforce every assertion #48

New issue

Closed

opened 2026-09-10 19:10:23 +01:00 by allod-agent · 0 comments

allod-agent commented

2026-09-10 19:10:23 +01:00

Member

Make every assertion in tests/rotate-token.sh count, so the suite can fail on the properties it names instead of only on each test's last line (the defect in allod/nexus#46).

Primary goals:

An enforcing runner — run_test runs each test in a context where a failed command ends the test: a subshell that sets -e itself, with the status read outside any if or ||, so bash's rule that suspends -e inside conditions no longer reaches the test body.
Every test honest — each test that fails under the enforcing runner is read to decide whether its assertion was wrong about the script or the script is wrong about the assertion; the test is corrected where it was the defect, and a script defect is fixed only when it is small and the test's intent is unambiguous, otherwise reported.
Unobservable checks removed — read -p prints no prompt when stdin is not a terminal, so no assertion on prompt text can pass or fail for a reason; those assertions go, or are reshaped to observe the effect of the prompt instead.
Provable — sabotaging one non-final assertion in any test fails the suite, which today it cannot.

Current state and specifics

run_test (tests/rotate-token.sh:11-21) is if "$@"; then PASS else FAIL; exit 1. Under an enforcing runner on master the sixth test, "in-place continuity dry-run orders refresh before network steps", fails, and the suite stops there, so the number of affected tests is unknown until the runner is fixed. The six tests added in allod/nexus#47 end every assertion with || return 1 and are already enforced; they stay as they are.

The suite runs in the provisioning-contract flake check, so the fixed runner and every corrected test must pass there as well as locally.

Scope

In: run_test, the tests whose assertions were vacuous, and scripts/rotate-token only where a corrected assertion exposes a small, unambiguous script defect.

Out: new coverage, restructuring the suite, and any script change whose intent the failing test does not settle; those are reported in the PR for separate issues.

Make every assertion in `tests/rotate-token.sh` count, so the suite can fail on the properties it names instead of only on each test's last line (the defect in allod/nexus#46). Primary goals: - **An enforcing runner** — `run_test` runs each test in a context where a failed command ends the test: a subshell that sets `-e` itself, with the status read outside any `if` or `||`, so bash's rule that suspends `-e` inside conditions no longer reaches the test body. - **Every test honest** — each test that fails under the enforcing runner is read to decide whether its assertion was wrong about the script or the script is wrong about the assertion; the test is corrected where it was the defect, and a script defect is fixed only when it is small and the test's intent is unambiguous, otherwise reported. - **Unobservable checks removed** — `read -p` prints no prompt when stdin is not a terminal, so no assertion on prompt text can pass or fail for a reason; those assertions go, or are reshaped to observe the effect of the prompt instead. - **Provable** — sabotaging one non-final assertion in any test fails the suite, which today it cannot. ### Current state and specifics `run_test` (`tests/rotate-token.sh:11-21`) is `if "$@"; then PASS else FAIL; exit 1`. Under an enforcing runner on `master` the sixth test, "in-place continuity dry-run orders refresh before network steps", fails, and the suite stops there, so the number of affected tests is unknown until the runner is fixed. The six tests added in allod/nexus#47 end every assertion with `|| return 1` and are already enforced; they stay as they are. The suite runs in the `provisioning-contract` flake check, so the fixed runner and every corrected test must pass there as well as locally. ### Scope In: `run_test`, the tests whose assertions were vacuous, and `scripts/rotate-token` only where a corrected assertion exposes a small, unambiguous script defect. Out: new coverage, restructuring the suite, and any script change whose intent the failing test does not settle; those are reported in the PR for separate issues.

allod-agent referenced this issue from a commit

2026-09-10 19:24:35 +01:00

tests/rotate-token: make run_test enforce every assertion in a test body

allod-agent referenced this issue from a commit

2026-09-10 19:25:04 +01:00

tests/rotate-token: fix the tests the enforcing runner exposed as vacuous

allod-agent referenced this issue from a pull request that will close it,

2026-09-10 19:28:42 +01:00

Make rotate-token's test runner enforce every assertion #49