Checklists

A checklist is a plain list of conditions the agent walks through before it reports done, written in the runbook next to the steps. The agent ticks through the list, fixes what fails, rechecks, and only then writes its report.

At a glance

Checklist

Conditions a reader can confirm by looking but a command cannot easily assert: the summary names every flagged receipt, the report links to its sources.

Run by

The agent

By inspecting its own output

When

Before its report

In the agent’s evaluate step

Fix-and-recheck

Yes

Up to three rounds in the templates

If not reported

skipped

Fails the verdict only under strict evaluation

Written as: Checkbox lines under ## Checklist.
In the report: A checklist entry per item, written by the agent. The result is the agent's word.

Example

## Checklist

- [ ] Overall rubric average is at least 4.0
- [ ] summary.md has the rubric scores and the iteration history
- [ ] No placeholder text remains

Writing a checklist

Put the items under a ## Checklist heading, one checkbox per line. A qualifier on the heading, such as ## Checklist (manual), is allowed, and so is the templates' numbering (## Step 8: Checklist).

Only checkbox lines count (- [ ] or - [x]). Prose between them, and anything inside a fenced block, is not an item.
Each item's id is its text in lower case with every run of other characters turned into a dash: No placeholder text remains becomes no-placeholder-text-remains. Two items with the same id are told apart as -2, -3.
A run's {{name}} inputs are filled into the items before their ids are taken, so the ids match what the agent read.

How a checklist runs

The checklist is the agent's own evaluate step. When an item fails, the agent goes back, fixes the output, and rechecks the list. The loop is bounded: the templates allow three rounds, after which the agent keeps its best result, notes what still fails, and writes its report.

How items are reported

Jetty tells the agent the id of every item, and the agent records one checklist entry per id in its report, with pass or fail and a one-line message. Jetty does not re-check an item. A failed item fails the run's verdict; an item the report leaves out is recorded as skipped. See how the report comes together.

Tips

Make each item small and observable. “The CSV has one row per input image” can be confirmed; “the output is good” cannot.
Keep item text stable. Editing an item changes its id, so runs before and after the edit record it as different items.
Move what a command can see to a Jetty check. A file exists, a pattern is absent: as a Jetty check, the agent cannot misreport it.