Reassess the plan-review convergence heuristic after its first full run

vnprc-agent commented

2026-07-04 23:58:06 +01:00

Contributor

The plan-review convergence heuristic in dev-plans.md and templates/plan-review-prompt.md — stop the plan-text review when either "review-introduced findings outnumber original-plan findings for two consecutive passes" or "two consecutive passes produce no BLOCKER and no original-plan GAP" — is in its first run now. That run is in progress, not finished: the plan-text review has had three passes and has not converged (it needs two consecutive clean passes and has had none), and nothing has been implemented yet. This issue records an open question the run has already surfaced, to be decided once the run finishes.

Interim signal (3 passes so far, not converged). Pass 1 (claude-fable-5): 1 BLOCKER + 5 GAP, all original-plan. Pass 2 (claude-opus-4-8): 3 GAP (2 original-plan, 1 review-introduced). Pass 3 (gpt-5): 1 GAP, original-plan. Two things stand out so far: original-plan defects were still surfacing at pass 3, and they surfaced from a fresh full re-walk the reviewer performed beyond its scoped-diff remit — not from the scoped diff itself. This is an early read, not a conclusion; the run has not reached convergence.

Open question. When a review reaches two clean convergence passes, does a single dedicated full sweep by a fresh, fix-stable model — primed specifically for original-plan defects — earn a place before handoff to implementation review? The gate can declare a pass "clean" on a scoped-diff verification, and a scoped pass need not re-walk the whole original plan, so "clean" can mean "the last fix's diff was clean," not "the whole plan is clean." So far the reviewers have re-walked anyway, so the gap has not bitten; whether that holds through convergence is what this run tests.

Decision trigger. After this first run finishes — the review converges and the plan is implemented — and its cycle is assessed. Decide with:

Did the two clean convergence passes actually re-walk the full original plan, or only verify the last fix's scoped diff?
Did any original-plan defect surface during implementation that the converged review had declared absent?
Yes on either → add a convergence-confirmation full sweep to the template (surface as Convergence confirmed by: <model>). No on both → the rotating re-walks already cover it; leave the heuristic as-is.

The plan-review convergence heuristic in `dev-plans.md` and `templates/plan-review-prompt.md` — stop the plan-text review when either "review-introduced findings outnumber original-plan findings for two consecutive passes" or "two consecutive passes produce no BLOCKER and no original-plan GAP" — is in its first run now. That run is **in progress, not finished**: the plan-text review has had three passes and has not converged (it needs two consecutive clean passes and has had none), and nothing has been implemented yet. This issue records an open question the run has already surfaced, to be decided once the run finishes. **Interim signal (3 passes so far, not converged).** Pass 1 (claude-fable-5): 1 BLOCKER + 5 GAP, all original-plan. Pass 2 (claude-opus-4-8): 3 GAP (2 original-plan, 1 review-introduced). Pass 3 (gpt-5): 1 GAP, original-plan. Two things stand out so far: original-plan defects were still surfacing at pass 3, and they surfaced from a fresh full re-walk the reviewer performed beyond its scoped-diff remit — not from the scoped diff itself. This is an early read, not a conclusion; the run has not reached convergence. **Open question.** When a review reaches two clean convergence passes, does a single dedicated full sweep by a fresh, fix-stable model — primed specifically for original-plan defects — earn a place before handoff to implementation review? The gate can declare a pass "clean" on a scoped-diff verification, and a scoped pass need not re-walk the whole original plan, so "clean" can mean "the last fix's diff was clean," not "the whole plan is clean." So far the reviewers have re-walked anyway, so the gap has not bitten; whether that holds through convergence is what this run tests. **Decision trigger.** After this first run finishes — the review converges and the plan is implemented — and its cycle is assessed. Decide with: - Did the two clean convergence passes actually re-walk the full original plan, or only verify the last fix's scoped diff? - Did any original-plan defect surface during implementation that the converged review had declared absent? - Yes on either → add a convergence-confirmation full sweep to the template (surface as `Convergence confirmed by: <model>`). No on both → the rotating re-walks already cover it; leave the heuristic as-is.

vnprc commented

2026-07-05 19:55:54 +01:00

Owner

This is now complete. The notes/allod/rotation-script-consolidation-dev-plan.md passed review by human decree after 4 passes. The final pass flagged a new "gap" that was reclassified as a flawed, vague requirement in allod/memory instructions. See allod/memory issue #12.

This is now complete. The `notes/allod/rotation-script-consolidation-dev-plan.md` passed review by human decree after 4 passes. The final pass flagged a new "gap" that was reclassified as a flawed, vague requirement in `allod/memory` instructions. See `allod/memory` issue https://forge.anarch.diy/allod/memory/issues/12.

vnprc-agent commented

2026-07-16 19:37:25 +01:00

Author

Contributor

Closing per maintainer: the plan-review convergence concern has not recurred in practice and may be masked by the volume of cross-references these issues now accrue. Reopen with a concrete case if it resurfaces.

Closed as not planned.

Closing per maintainer: the plan-review convergence concern has not recurred in practice and may be masked by the volume of cross-references these issues now accrue. Reopen with a concrete case if it resurfaces. Closed as not planned.

vnprc-agent closed this issue

2026-07-16 19:37:25 +01:00

Rows
Columns

Reassess the plan-review convergence heuristic after its first full run #11