Give tests/pi-provider.sh one function per case #65

Merged

vnprc merged 1 commit from agent/pi-provider-test-cases into master

2026-09-22 03:38:39 +01:00

allod-agent commented

2026-09-22 02:23:06 +01:00

Member

tests/pi-provider.sh now has one function per case, run from the same runner its sibling tests use, so a failing scenario is named rather than being a line number into a fifteen-hundred-line straight-line script. Every assertion is preserved: the multiset of all 440 assertion calls is identical before and after, and the suite passes with the same 336 assertions as master. Nothing outside this one test file changes. If it is wrong, revert the one commit.

What changed

33 scenario functions plus a final leak-scan case, 34 run_test calls, each scenario opening with its own fixture setup. The prose comment that introduced each scenario is its function's header.
setup_fixture (327 lines) is split into itself and write_fixture_command_stubs, the eight sabotage-capable command stubs, written with the same content and mode.
The runner is copied from tests/nexus-host-key.sh and tests/refresh-local-auth.sh: the case runs in a subshell that sets -e itself and its status is read as a plain statement, never from an if condition, per shell.md.
One mechanism had to change because each case now runs in its own subshell: the pass count, the synthetic bearers and the bearer command outputs that the old flat script accumulated in shell variables are appended to log files in a bookkeeping directory kept outside $TEST_ROOT, and the leak-scan case, registered last, reads them from there. Inside $TEST_ROOT the scan flagged its own log as a leak, which is why it lives outside.
Two scenarios stay as single long functions (about 285 and 400 lines) because they are sequential narratives whose later steps assert against state the earlier steps produced; splitting them means re-threading tokens, digest snapshots and commit points across helpers, which is a follow-up if wanted, not a move.

Risk

R2. One test file; no production script or module changes. The failure modes are an assertion that now checks a freshly generated value where it used to check one an earlier scenario produced, and the on-disk bookkeeping letting the leak scan pass vacuously; the review below was asked to enumerate both. Rollback is a straight revert.

Validation

Assertion multiset: every assert_*, pass and fail call's source text extracted from both versions, whitespace-normalized and sorted: identical, 440 calls. The integrator independently compared per-helper call counts on master and the branch: assert_contains 97, assert_missing 12, assert_json 40, assert_ciphertext 23, assert_undecryptable 4, pass 85, fail 119, all equal.
bash -n and shellcheck -x --source-path=SCRIPTDIR, exactly as the flake check invokes them: clean, re-run by the integrator.
bash tests/pi-provider.sh from the worktree root, re-run by the integrator: exit 0, no FAIL line, pi-provider tests passed: 336 assertions, the same count as master.
Sabotage: a false inserted mid-case between a pass and the case's final assertion is reported FAIL by the runner and stops the suite with exit 1; removed, the file is byte-identical to the committed version.
Not run: the provisioning-contract flake check that wraps every test in tests/, since it builds the provisioning scripts package; the same lint and the same test invocation were run directly with the same tools from the VM's profile.
One cold cross-family review of the diff (gpt-5.6-terra on codex, high effort), asked specifically about cross-case state, the bookkeeping directory, the runner's set -e handling, the two long cases and the moved stubs: no findings (transcript of 732 KB; an empty review was ruled out before it was read as approval).

Closes allod/nexus#64

`tests/pi-provider.sh` now has one function per case, run from the same runner its sibling tests use, so a failing scenario is named rather than being a line number into a fifteen-hundred-line straight-line script. Every assertion is preserved: the multiset of all 440 assertion calls is identical before and after, and the suite passes with the same 336 assertions as master. Nothing outside this one test file changes. If it is wrong, revert the one commit. ## What changed - 33 scenario functions plus a final leak-scan case, 34 `run_test` calls, each scenario opening with its own fixture setup. The prose comment that introduced each scenario is its function's header. - `setup_fixture` (327 lines) is split into itself and `write_fixture_command_stubs`, the eight sabotage-capable command stubs, written with the same content and mode. - The runner is copied from `tests/nexus-host-key.sh` and `tests/refresh-local-auth.sh`: the case runs in a subshell that sets `-e` itself and its status is read as a plain statement, never from an `if` condition, per `shell.md`. - One mechanism had to change because each case now runs in its own subshell: the pass count, the synthetic bearers and the bearer command outputs that the old flat script accumulated in shell variables are appended to log files in a bookkeeping directory kept outside `$TEST_ROOT`, and the leak-scan case, registered last, reads them from there. Inside `$TEST_ROOT` the scan flagged its own log as a leak, which is why it lives outside. - Two scenarios stay as single long functions (about 285 and 400 lines) because they are sequential narratives whose later steps assert against state the earlier steps produced; splitting them means re-threading tokens, digest snapshots and commit points across helpers, which is a follow-up if wanted, not a move. ## Risk R2. One test file; no production script or module changes. The failure modes are an assertion that now checks a freshly generated value where it used to check one an earlier scenario produced, and the on-disk bookkeeping letting the leak scan pass vacuously; the review below was asked to enumerate both. Rollback is a straight revert. ## Validation - Assertion multiset: every `assert_*`, `pass` and `fail` call's source text extracted from both versions, whitespace-normalized and sorted: identical, 440 calls. The integrator independently compared per-helper call counts on master and the branch: `assert_contains` 97, `assert_missing` 12, `assert_json` 40, `assert_ciphertext` 23, `assert_undecryptable` 4, `pass` 85, `fail` 119, all equal. - `bash -n` and `shellcheck -x --source-path=SCRIPTDIR`, exactly as the flake check invokes them: clean, re-run by the integrator. - `bash tests/pi-provider.sh` from the worktree root, re-run by the integrator: exit 0, no `FAIL` line, `pi-provider tests passed: 336 assertions`, the same count as master. - Sabotage: a `false` inserted mid-case between a `pass` and the case's final assertion is reported `FAIL` by the runner and stops the suite with exit 1; removed, the file is byte-identical to the committed version. - Not run: the `provisioning-contract` flake check that wraps every test in `tests/`, since it builds the provisioning scripts package; the same lint and the same test invocation were run directly with the same tools from the VM's profile. - One cold cross-family review of the diff (`gpt-5.6-terra` on codex, high effort), asked specifically about cross-case state, the bookkeeping directory, the runner's `set -e` handling, the two long cases and the moved stubs: no findings (transcript of 732 KB; an empty review was ruled out before it was read as approval). Closes allod/nexus#64

allod-agent added 1 commit

2026-09-22 02:23:06 +01:00

tests/pi-provider.sh: one function per case, with a run_test runner c1a0878394

Restructure the file to match tests/nexus-host-key.sh and
tests/refresh-local-auth.sh: every scenario after the fixture helpers is now
a named function called from a run_test runner at the bottom, in a subshell
that sets -e itself and reads its status as a plain statement (shell.md).
setup_fixture is split into itself plus write_fixture_command_stubs, which
writes the eight sabotage-capable command stubs every fixture shadows.

Two scenarios are long sequential narratives that build up and consume
specific mutable state across dozens of steps on one fixture
(credential_lifecycle_add_share_token_management_retarget_and_retirement and
authenticated_discovery_covers_add_refresh_confirmation_and_field_validation);
each run_test case runs in its own subshell, so state built by one case
cannot be resumed by another, and these two stayed single functions rather
than being split into many small precondition-chaining helpers, to keep the
move mechanical and low-risk. Everything else became its own independent
case with its own fresh fixture.

The final leak-scan (no synthetic bearer appears anywhere) and the closing
assertion count previously relied on shell arrays (BEARERS,
BEARER_COMMAND_OUTPUTS) and a PASS_COUNT variable mutated across the whole
flat script; under the new per-case-subshell model those mutations would be
discarded when each case's subshell exits. Both now accumulate on disk under
a bookkeeping directory kept outside $TEST_ROOT (so the leak-scan's own logs
are never mistaken for a leak), and the leak-scan case is registered last.

Every assert_contains/assert_missing/assert_json/assert_ciphertext/
assert_undecryptable/pass/fail call and its arguments is unchanged; verified
by extracting and diffing the sorted multiset of every such call before and
after (empty diff).

Refs allod/nexus#64