Authoring your first runbook

The best way to write your first runbook is with an agent. You brief an agent, it drafts the file, and Jetty runs it. The part that stays yours is deciding what “done” means and reading what the first run did. This guide covers those four moves, then one pattern worth knowing early: a runbook that runs other runbooks.

Four moves to a first runbook: you write the brief, an agent drafts the runbook, Jetty runs it and scores it, and the run tells you the one thing to change before the next run.

1. Brief it like a person

Pick a small job you already know how to judge. Say three things in plain words: the job, the input that will change from run to run, and what a finished result looks like.

Text

Compare the pricing pages of Acme, Globex and Initech.
Input: the three company names. I'll want to swap them later.
Done: one markdown table, a row per plan, with the monthly price
and the page each price came from. If a plan has no public price,
say "contact sales". Don't guess.

Give the brief to Pelly on the Author Runbooks screen in the web app, or to your own coding agent with the create-runbook skill. Both are set up in How to start with Jetty, and both hand back the same thing: one markdown file.

2. Read the draft

A first draft for that brief looks like this:

RUNBOOK.md

---
version: "1.0.0"
agent: claude-code
model: claude-sonnet-5
---
# Pricing page comparison

## Objective
Compare the public pricing of the companies in {{companies}} so a sales
lead can read the result in two minutes.

## Parameters
| Parameter | Template Variable | Default | Description |
|-----------|-------------------|---------|-------------|
| Companies | `{{companies}}` | *(required)* | Comma-separated company names |

## Steps
1. Find the pricing page of each company in {{companies}}.
2. Record every plan: its name, its monthly price, and the page it came from.
3. Write {{results_dir}}/comparison.md as one markdown table, a row per plan.
   A plan with no public price says "contact sales".

## Code Checks

### comparison-written — comparison.md exists and is not empty
```bash
test -s {{results_dir}}/comparison.md
```

## Checklist
- [ ] Every company in {{companies}} has at least one row
- [ ] Every price has its source URL beside it
- [ ] No price was guessed

## Required output files
- {{results_dir}}/comparison.md

Before you run it, read it for four things:

The steps describe work. “Record every plan and the page it came from” can be followed.
Every input is named in the steps. The agent reads the body of the file. A parameter that appears only in the table is ignored.
The output is a file with a name. A run is judged on what it leaves in {{results_dir}}.
There is at least one check. Without one, a run can only tell you that it finished.

The frontmatter is a default you can override per run, so don't spend time on it yet. The full anatomy is on the Runbooks page.

3. Decide what done means

Start with two kinds of check and keep both short. A code check is a command Jetty runs after the agent has finished, so the agent can neither skip it nor misreport it: the file exists, the JSON parses, the totals add up. A checklist is the list of conditions the agent confirms about its own work, fixing and re-checking before it reports.

Write each line so that a stranger could answer it yes or no. Three lines you trust is better than a dozen open to interpretation.

4. Run it, then change one thing

Run it once and open the run. Read the verdict, then open the output file yourself: a first run often passes its checks and is still wrong in a way you would spot in a few seconds. That gap is your next edit. Fix the step that produced the miss, or add the check that would have caught it, and run again. One change per run, so you know which change worked.

When two or three runs in a row pass and you agree with them, the runbook is ready to be used. Our guide on improving agent runbooks picks up from there.

When one runbook isn't enough: runbooks that run runbooks

A deployed runbook is a named thing you can call: parameters go in, files and a verdict come out. That makes it a building block. Any step of another runbook can launch it, wait for it and read what came back, which means the agent in the parent run is orchestrating other agents, each in its own sandbox with its own checks.

A runbook that runs runbooks: one step of the parent launches the same child runbook once per configuration. Each child is a full run with its own checks; the parent’s checks judge the investigation, starting with whether every child reported. Illustrative agents and results.

Taylor Hodge at Primitive uses this to test one plugin across every coding agent. A child runbook takes one agent and model through Primitive's whole workflow and scores it. A parent runbook, the investigation, launches that child once per agent and model, gathers every verdict and writes a single report that diagnoses each failure. A sweep is 13 runs and takes about 20 minutes, and because the sweep is itself a runbook it can be rerun on demand or put on a schedule.

The launch is an ordinary step in the parent, calling the same API you would:

Markdown

## Step 2: Launch the child runbook

For each line of {{results_dir}}/configurations.txt, start one run of the
child and save the id the call returns:

```bash
curl -fsS -X POST "https://flows-api.jetty.io/api/v1/run/{{collection}}/{{child_task}}" \
  -H "Authorization: Bearer $JETTY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"init_params": {"agent": "<agent>", "model": "<model>", "vars": {"companies": "{{companies}}"}}}'
```

Do not retry a launch that returned an id: a retry starts a second run.

The parent needs a Jetty API token to make that call: store it on the collection and declare it under secrets: in the parent's frontmatter (see Secrets and environment variables). The run endpoint and the one that reads a run back are in the API reference.

Four habits keep a composition honest:

Author the child first. Get it passing on its own with the four moves above. A parent cannot fix a child that doesn't work alone.
Same runbook, different inputs. Change only the parameters or the runtime configuration between children, so a difference in the results points at the thing you varied.
Check at two levels. The child's checks judge the work. The parent's checks judge the investigation: every child reported, and every failure in the report is explained.
Ask for evidence. Have the parent quote the children's own output for every conclusion it draws, and relaunch once any child that ends without a report.

The same shape covers a benchmark across models, one job fanned out over a hundred inputs, or a nightly investigation of yesterday's runs. Creating a benchmark and Scheduling AI workloads take it further.