Runbooks
A runbook is a short markdown file that describes the job an agent will perform. There are three ways to get started:
- Ask Pelly. Let Pelly, the Jetty agent, guide you through the creation process in minutes.
- Bring your agent. Use the Jetty Skill and MCP to bring your agent context into runbook creation.
- Try a Project. Fork a public runbook from the directory.
What a runbook looks like
The markdown file has four parts: frontmatter, setup, steps, and evaluations. The frontmatter is a YAML block at the top that declares a default runtime, model and agent configuration. Note that these can be overridden when the AI workload runs with a specified runtime configuration. The setup section is a short description of the environment the work needs. The steps section describes the sequence of instructions the agent should follow to complete the job. Finally, the evaluations section describes the criteria for success and which files the agent should produce.
The full anatomy, including the frontmatter schema, the output manifest, parameters, and the two evaluation styles is detailed in here.
Why runbooks are different from prompts and skills
Runbooks are borne out of the lived experience of watching AI workloads fail in unpredictable ways. That structure is what separates a runbook from the other ways of telling an agent what to do:
| Agent prompt | Agent skill | Runbook | |
|---|---|---|---|
| What it captures | A one-off ask | How to do something, reusably | A whole job: setup, task, checks, outputs |
| Definition of done | Whatever the agent decides | Inherited from whoever invokes it | Explicit criteria in the file |
| Checking the work | You read the output | None built in | The agent scores its output and iterates until it passes |
| Output | A reply in a session | Depends on the session | Declared artifact files, persisted on every run |
| Repeatability | Copy, paste, hope | Consistent method, unmanaged results | Parameterized, deployable, schedulable, comparable run to run |
Each execution produces a run, which includes the inputs, outputs, logs, telemetry, and a copy of the runbook.
Working with runbooks
Runbooks serve two purposes inside Jetty: executing a long-running AI workload, and running investigations across runs to figure out how your AI workloads are performing. Get started on the Jetty web platform, then keep managing them by bringing your own agent to Jetty.
| Activity | Jetty Web | Jetty Agent CLI | |
|---|---|---|---|
| Author | Building a runbook | Yes | Yes |
| Runbook editing | Beta | Yes | |
| Run | Launch runs | Yes | Yes |
| Schedule workloads | Beta | Yes | |
| Investigate | Model benchmarking | Beta | Yes |
| Cost and quality analysis | Beta | Yes | |
| Improve | Add evaluations | Beta | Yes |
Runbook Context
The runbook markdown itself is the agent's instruction. Three mechanisms parameterize it:
- Template variables.
{{name}}placeholders substitute from the run'svars(declared with defaults in the runbook's## Parameterstable). Substitution is literal string replacement — declare every variable, and restate parameters imperatively in the body; agents ignore values that only appear in metadata. - Uploaded files. Files attached to a run are mounted in the sandbox and their listing is appended to the instruction automatically.
- Prior runs. In multi-step workflows, path expressions pull the outputs of earlier actions into the instruction.
One hard limit: the instruction after substitution must fit in ~120 KB (the instruction is passed to the agent CLI as a single argument). Large corpora belong in uploaded files, not inlined variables.
How files get into and out of the sandbox, and how the agent reaches the web, is covered under MCPs & Skills in Runtime Configuration.