Hephaestus Skills/ Docs / verify-loop GitHub

Docs / Skills / verify-loop

verify-loop

Core/verify-loopStop / StopFailure

Use before calling a code change done, before the last commit or opening a PR/MR, after a subagent reports success, and when asked "verify loop", "verify before PR", "check before PR", "verify trước khi mở PR", "code xong chưa", "đã chắc chưa". Also when adding a project check: "thêm check", "lần sau nhớ kiểm tra", "lỗi này lặp lại hoài". Not for checking a deployed environment or one screen in a browser (e2e-verify).

Before calling a change done or opening a PR: every gate in order on one head, loop back on red, one PASS/FAIL table

Also in this skill: references/ci.md scripts/checks.py

Every gate already has an owner skill; changes still ship broken because a gate ran on an older head, was skipped without a reason, or never existed for the one rule this project keeps breaking. This skill owns the order of the gates, the loop back after a red one, and the one table that proves the final head passed; each gate's method stays with its owner.

0. A claim needs fresh evidence

Before writing "passes", "fixed", "done", or any word that implies it, run the command that proves it in this turn and read its exit code and output.

ClaimEvidenceNot evidence
Tests passThe test command's output with 0 failures, on this headAn earlier run, "should pass"
Build or typecheck passesThat command exits 0Lint passing, an editor with no red
Bug fixedThe reporter's sequence passes, and the test fails with the fix revertedCode changed
Requirement metSection 2 gate 7, every row with evidenceTests passing
A subagent finishedIts diff read and its checks rerun by youIts report

1. Scope the run

  • <int> per worktree-flow section 0; the range is origin/<int>...HEAD plus uncommitted files.
  • Classify the change from git diff --name-only origin/<int>...HEAD: docs or config only, copy only, no behavior change (rename, move, extract), code, user-facing (routes, components, copy, or config such as a redirect, route or flag), and whether it hits the security-pass triggers (its section 0) or the flow-trace triggers (AGENTS.md). That classification picks the gate set: a small-change gate set (section 2) or every gate.
  • A caller skill can hand a gate to someone else (jira-bug-hunter defers gate 6 until its PR is merged and deployed to dev): mark it DEFER with who runs it, and that runner runs the whole gate, security-pass and flow-trace included. No caller can rule a gate out; a user-facing change gets gate 6 whatever the bug's tier.
  • Project checks: python3 <this skill>/scripts/checks.py match origin/<int> lists the checks in .agents/checks/ whose paths the change touches.

2. Gates, in order

Run them top to bottom on one head. A red gate stops the run: fix the cause (root-cause when it is not obvious), commit, and restart at gate 2 per the rerun table below, never at the gate that failed.

  1. Base: rebased on the latest origin/<int> (worktree-flow section 4 step 1), so nothing after this changes the head except fixes.
  2. Toolchain: build when the project-init block of AGENTS.md lists one, typecheck, lint, and tests of the touched packages, with that block's commands, plus the Verify section of each feature-build tech file the work loaded. The block's E2E suite belongs to gate 6, not here. Read the exit code and the failure count, not the tail of the log.
  3. Project script checks: python3 <this skill>/scripts/checks.py run origin/<int> exits 0; keep its "Project checks for the reviewer" lines for gate 5.
  4. Proof: every behavior the change adds or fixes has a test that fails without it, proven with the revert-only-the-change commands of root-cause section 5. The failure must be the assertion: when the revert leaves only import or compile errors (new code), break the new code in one small way instead, per that section's mutation list. A migration's proof runs on your own database (worktree-flow section 2), never on a shared one.
  5. Review: the fresh-review loop, with the gate 3 reviewer lines pasted into its prompt. Run security-pass and flow-trace when section 1 found their triggers, even if the reviewer did not flag them, and when the reviewer flags them.
  6. Behavior: user-facing change: e2e-verify, with its evidence file; otherwise SKIP not user-facing.
  7. Requirements: one row per acceptance criterion of the ticket, spec, or request (the feature-build step 1 list; for a bug, the reporter's expected result and every scope item the comments added), each with its evidence: a test name, an E2E step, or open. An open row is red unless the user already agreed to leave it out; a row whose evidence is a DEFER gate is DEFER too.

Caps and never weaken

  • The owners keep their caps and what happens at them (3 review rounds, 3 E2E FAIL cycles then root-cause, 3 failed fixes then stop). Gates 2, 3, and 7 have no owner cap: the same failure after 3 fixes follows root-cause section 3.
  • Never make a gate green by weakening it: no skipped or retried tests, raised timeouts, lint disables, or deleted checks, unless the user asks for that by name.

Small-change gate sets

These gate sets are independent of the feature-build lanes: a lane S one-line tweak that changes behavior still gets every gate. Section 1 picks a gate set only from facts in the diff; when any fact is in doubt, the change gets every gate. A risk trigger means a security-pass section 0 trigger or a flow-trace trigger from AGENTS.md; every gate set requires none. Gates 1, 2, 3, and 7 run in every gate set, and the report still has every row.

Gate setThe diff touches onlyGate 4Gate 5Gate 6
Docs or configDocs or config filesSKIP docs/config onlySKIP docs/config onlyRuns when section 1 found it user-facing (a redirect, route or flag config), else SKIP not user-facing
CopyUser-visible text (string literals, translation files), nothing else on those linesSKIP copy-only, proven in behavior (root-cause section 5 bans asserting on copy)RunsRuns; its evidence is the step that shows the new text
No behavior changeCode whose behavior stays the same (rename, move, extract), with no test expectation changedSKIP no behavior added or fixed, gate 2 tests cover itRuns, with "no behavior change" stated in the promptRuns when section 1 found it user-facing, else SKIP not user-facing

The security and flow rows read SKIP no trigger in every gate set. A reviewer finding of a behavior change, or a fix commit that breaks a gate set's facts, moves the change out of that gate set, and the gates it now needs run in full.

Rerun after a fix commit

"Touched" means git diff --name-only <head the gate last passed on>..HEAD, which also covers a rebase. A gate that has not yet passed on this branch runs in full.

GateReruns
1 BaseNot rerun; rechecked when the PR/MR opens (section 3)
2 ToolchainAlways, in full
3 Script checksAlways, in full
4 ProofOnly the behaviors with a touched file, plus every behavior the fix adds or fixes
5 Reviewfresh-review on the delta (its own later-round range), counted against its cap; security-pass reruns unless its section 0 skip rule applies, and also when the delta meets a trigger it did not meet before; flow-trace reruns when the delta touches a file on a traced flow or meets a flow trigger
6 BehaviorOnly what the e2e-verify re-run rule (its Output section) names
7 RequirementsAlways: every row rechecked, evidence pointing at the head it ran on

A gate that does not rerun keeps its evidence with the head it ran on and "no touched file since" (the behavior row in section 3).

3. Report

One table for the final head, pasted into the PR/MR body's verification part (worktree-flow section 4 step 5):

verify-loop  head <short sha>  base origin/<int> <short sha>
gate          status  evidence
base          PASS    rebased, 0 commits behind
toolchain     PASS    pnpm typecheck (exit 0); pnpm lint (exit 0); pnpm --filter web test (212 passed)
script checks PASS    2 run: no-console, migration-order
proof         PASS    apps/web/billing.test.ts "keeps paid invoice unlocked" fails with the fix reverted
review        PASS    fresh-review 2 rounds, risk low
security      SKIP    no security-pass trigger
flow          PASS    flow-trace docs/flow-traces/invoice.md
behavior      PASS    e2e/reports/WEST-12-2026-10-19T09:12:00Z.json at a1b2c3d, no AC path touched since
requirements  PASS    AC1 test, AC2 test, AC3 e2e step 4

Statuses: PASS, FAIL, SKIP, or DEFER, each with its evidence or reason. A SKIP on a gate the change needs is a FAIL; a DEFER names who runs that gate and where its result will be. The head in the table must equal git rev-parse HEAD when the PR/MR opens, and the base row is rechecked then: after git fetch origin, git rev-list --count HEAD..origin/<int> prints 0, otherwise rebase and restart at gate 2 per the rerun table (section 2).

Once the PR/MR is open, its CI on that head is the last gate: gh pr checks <n> --watch --fail-fast --required, or glab ci status --live on GitLab, until every required check has finished. A pending check is not a pass, and a red check is a red gate: fix the cause, commit, push, restart at gate 2 per the rerun table, then update the table in the PR/MR body (gh pr edit <n> --body-file <file>, glab mr update <n> --description "$(cat <file>)", or without glab the user pastes it, per worktree-flow section 4). A repo with no CI on PRs/MRs says so in the report; section 5 is where to propose it.

4. Add a project check

When the user corrects the same kind of mistake a second time, or a bug escaped that a rule on the diff would have caught, propose a check, and add it in the same branch when the user agrees.

  • python3 <this skill>/scripts/checks.py new <id> prints the template; its --help owns the fields.
  • script when a command can decide it (a grep, a schema query, a codegen diff); review when it needs judgment on the diff.
  • why names the incident or correction and its date, so the check can be retired when its reason is gone.
  • Prove a script check like a test: it fails on the commit that had the mistake and passes on the fix.
  • checks.py list validates every check; run it after each edit.

5. Every PR, in CI

Running the same gates on every PR is the project's choice; references/ci.md has the workflow and what it costs. Propose it once per project when its project checks settle, and add it only when the user agrees.