allod memory backpass: run the skill as a program, not a prompt #276

New issue

Closed

opened 2026-10-19 03:17:05 +01:00 by vnprc-agent · 0 comments

vnprc-agent commented

2026-10-19 03:17:05 +01:00

Contributor

Make the memory backpass a program, allod memory backpass, so a nightly timer runs code and spends a model only on the two prompts, instead of asking a headless model to follow the skill's ten steps with a shell. Part of allod/strategy#55.

Primary goals:

Deterministic driver — pending-trace listing, prompt filling, model calls, quote check, fold, ledger lines, ledger prunes, PR body and state-file update are code; the skill text at allod/memory skills/memory-backpass/SKILL.md stays the spec and the command cites its step numbers.
Two model seats, nothing else — one bulk-analysis call per trace with one retry and the roster's fallback, one synthesis call; every prompt begins with  and is self-contained, because a harness that loads its own instruction files has refused a bare prompt as an injection.
Safe to run unattended — idempotent on rerun (the hash marker makes a failed or grown trace pending again), --report-only, a non-zero exit when any trace failed, never a push to a default branch; the PR is the artifact.
The funnel is the alarm — print traces analyzed, items returned, items dropped for a missing quote, corroborations and edits; a drop rate far above the few-percent baseline means corrupted input, not a bad model.

Current state and specifics

The skill has run three times by hand, each time as a session following the prose and writing throwaway scripts. The planned timer (allod/archetypes modules/memory-backpass.nix, in progress) starts pi -p with a prompt that says to read the skill and run it. The 2026-10-18 hand run took a full session for 43 traces at about three minutes each with four calls in flight, and the scripts it left behind are the shape of this command:

step in the skill	what the code does
1	number every `memory.md` bullet (`PUB-nnn`, `PRV-nnn`), id every open ledger line as `L-` plus the first 8 hex of its SHA-256, list merged `memory-backpass:` PRs with an unchecked relay line
2	fill `analysis-prompt.md` by literal substitution, call the bulk model under a hard timeout, gate on parsed content with `positive`, `negative`, `gaps` arrays, strip a Markdown fence, retry once, then the fallback seat
3	whitespace-fold quote and trace, keep substrings only, count the drops
4 to 6	one ledger line per surviving item and session, corroborations appended to the existing line, prunes of pre-run entries by age or coverage
7 to 9	private worktree via `allod change begin`, synthesis call, budget hook, `record`, `submit`, label `relay`; public edits as plain commits in a second worktree, left unpushed

Three defects the hand run hit that the code must not repeat: a bash ${var//pattern/replacement} filler turned every & in a trace into the matched placeholder (patsub_replacement is on by default in bash 5.2), which showed up only as a 36% quote-drop rate; a pkill -f whose pattern also appeared in the calling shell's command line killed the run; and the state file recorded paths, so a trace that kept growing after it was read stayed marked done (fixed in the skill: sha256sum lines, see the relay in the private memory repository's backpass PR of 2026-10-18).

Model choice stays with the deployment's agent-roster skill; the command takes the bulk and synthesis seats as options or environment and names them in the report, and never substitutes a seat silently.

Scope

In: the subcommand, a docs/allod-memory.md page, tests on fixture traces and memory trees with a fake model (a script the command is pointed at) covering the funnel, the quote check, the one-line-per-rule-and-session rule and the hash marker. Out: the timer module itself (allod/strategy#55 slice 2 shrinks to calling this command), the memory-prune skill (a later allod memory prune in the same shape), and trace retention in memory-traces.

Make the memory backpass a program, `allod memory backpass`, so a nightly timer runs code and spends a model only on the two prompts, instead of asking a headless model to follow the skill's ten steps with a shell. Part of allod/strategy#55. Primary goals: - **Deterministic driver** — pending-trace listing, prompt filling, model calls, quote check, fold, ledger lines, ledger prunes, PR body and state-file update are code; the skill text at `allod/memory` `skills/memory-backpass/SKILL.md` stays the spec and the command cites its step numbers. - **Two model seats, nothing else** — one bulk-analysis call per trace with one retry and the roster's fallback, one synthesis call; every prompt begins with `` and is self-contained, because a harness that loads its own instruction files has refused a bare prompt as an injection. - **Safe to run unattended** — idempotent on rerun (the hash marker makes a failed or grown trace pending again), `--report-only`, a non-zero exit when any trace failed, never a push to a default branch; the PR is the artifact. - **The funnel is the alarm** — print traces analyzed, items returned, items dropped for a missing quote, corroborations and edits; a drop rate far above the few-percent baseline means corrupted input, not a bad model. ### Current state and specifics The skill has run three times by hand, each time as a session following the prose and writing throwaway scripts. The planned timer (`allod/archetypes` `modules/memory-backpass.nix`, in progress) starts `pi -p` with a prompt that says to read the skill and run it. The 2026-10-18 hand run took a full session for 43 traces at about three minutes each with four calls in flight, and the scripts it left behind are the shape of this command: | step in the skill | what the code does | |---|---| | 1 | number every `memory.md` bullet (`PUB-nnn`, `PRV-nnn`), id every open ledger line as `L-` plus the first 8 hex of its SHA-256, list merged `memory-backpass:` PRs with an unchecked relay line | | 2 | fill `analysis-prompt.md` by literal substitution, call the bulk model under a hard timeout, gate on parsed content with `positive`, `negative`, `gaps` arrays, strip a Markdown fence, retry once, then the fallback seat | | 3 | whitespace-fold quote and trace, keep substrings only, count the drops | | 4 to 6 | one ledger line per surviving item and session, corroborations appended to the existing line, prunes of pre-run entries by age or coverage | | 7 to 9 | private worktree via `allod change begin`, synthesis call, budget hook, `record`, `submit`, label `relay`; public edits as plain commits in a second worktree, left unpushed | Three defects the hand run hit that the code must not repeat: a bash `${var//pattern/replacement}` filler turned every `&` in a trace into the matched placeholder (`patsub_replacement` is on by default in bash 5.2), which showed up only as a 36% quote-drop rate; a `pkill -f` whose pattern also appeared in the calling shell's command line killed the run; and the state file recorded paths, so a trace that kept growing after it was read stayed marked done (fixed in the skill: `sha256sum` lines, see the relay in the private memory repository's backpass PR of 2026-10-18). Model choice stays with the deployment's `agent-roster` skill; the command takes the bulk and synthesis seats as options or environment and names them in the report, and never substitutes a seat silently. ### Scope In: the subcommand, a `docs/allod-memory.md` page, tests on fixture traces and memory trees with a fake model (a script the command is pointed at) covering the funnel, the quote check, the one-line-per-rule-and-session rule and the hash marker. Out: the timer module itself (allod/strategy#55 slice 2 shrinks to calling this command), the `memory-prune` skill (a later `allod memory prune` in the same shape), and trace retention in `memory-traces`.