version	1.0.0
evaluation	programmatic
evals_version	2
agent	claude-code
model	anthropic/claude-haiku-5.5
model_provider	openrouter
snapshot	python312-uv
primary_outputs	result_card.html

Who Chose This Trade-off? Transparent Rewards for Insulin Dosing (SMASH 2026 · Paper #15)

Objective

This runbook is an interactive companion to the SMASH 2026 paper "Towards Transparent Reward Design in Clinical Reinforcement Learning" (Caro Strickland, Baptiste Bonin, Audrey Durand). Read the paper: https://drive.google.com/file/d/1NuHr7dRpH-Qh5nYx7jZ9j1t1NRRW5bXk/view?usp=sharing

The paper's claim: an RL insulin-dosing policy trained on one scalar reward silently encodes a trade-off between competing clinical priorities, chosen by whoever wrote the reward. Splitting the reward into three objectives and reporting the Pareto front makes that trade-off visible. Two policies with the same time-in-range (TIR) can differ by tens of percentage points in time spent hypoglycaemic, which TIR alone never shows.

The participant brings their own preference weights over the three objectives (and their own hypo safety limit), picks a simglucose virtual patient, and the run: simulates a transparent family of 48 basal-only dosing rules on the FDA-accepted UVa/Padova simulator (simglucose); scores each rule on the paper's three-objective reward; finds the Pareto set; shows which rule their weights would choose; repeats the paper's "same TIR within 3 points, different hypo time" analysis; and maps every possible weight vector to the rule it picks. Output: a shareable result card.

Target wall-clock: under 5 minutes. The experiment takes about 25–60 s (96 one-day simulations, run in parallel).

The method as in the paper (§2), as implemented

• Reward vector per step: r = (r_gly, r_hypo, r_dose) with r_gly = −RI(g_t), r_hypo = −LBGI(g_t) (Kovatchev risk index, RI = LBGI + HBGI), and r_dose = −max(0, a_t − b_p) / (a_max − b_p), where b_p is the patient's basal need and a_max = 4 b_p.
• Each component is normalized by its empirical standard deviation (over every simulated step of every rule).
• Actions: seven basal levels spaced linearly across [0, 4 b_p]; basal-only dosing (no meal boluses).
• Matched pairs: drop policies whose SD across episodes exceeds 20 points on TIR or hypo time; among all pairs within 3 TIR points, report the median and maximum hypo-time gap.

What this runbook does differently (and says so on the card): the paper trains SAC baselines and a preference-conditioned Q-network (envelope MORL). Training deep RL is out of scope for a 5-minute CPU run, so this runbook replaces the learned policies with an enumerated family of simple rules: the 7 basal levels × a CGM hypo-suspend threshold (none / 80 / 110 mg/dL) × a high-glucose correction step (+0 / +1 / +2 levels while CGM > 180 mg/dL). "Your weights' pick" is the rule maximizing w · r̄ (normalized mean reward), i.e. an oracle scalarized optimum within the family, not a trained agent. Rules with TIR < 20% are excluded from the matched-pair analysis (clinically absurd, e.g. no insulin at all). Episodes are one simulated day (480 × 3-min CGM steps, random meals from simglucose's scenario generator) and are not cut short at simglucose's done flag.

Paper headline numbers (quote exactly)

• child#001: Baseline A (optimizing glycemic risk alone) hypoglycemic exposure 55.2%; baseline B (w = (0.5, 0.3, 0.2)) 0.0% hypo at the cost of 56.0% hyperglycemic exposure; both report a similar TIR.
• Table 1, hypo-exposure gap (pp) among policies within 3 TIR points: Adult 39/40 reliable, 98 pairs, median 0.0, max 42.8; Adolescent 39/40, 55 pairs, median 0.0, max 26.5; Child 28/40, 148 pairs, median 6.4, max 39.8 (text: child 90th percentile 21.1).
• Differential ("risk-decreasing") reward is exploitable: "a policy can collect higher reward by oscillating into and out of a hazardous state than by avoiding it altogether", so the paper uses absolute risk.

REQUIRED OUTPUT FILES (MANDATORY)

All in {{results_dir}}:

File	Description
result_card.html	Primary output. Self-contained, shareable one-page result card
results.json	All computed numbers (written by experiment.py)
card_data.json	Numbers/figures for the card (written by experiment.py)
narrative.json	Your interpretation of this run (Step 4)
fig_*.png	Figures (written by experiment.py)
summary.md	5–10 line plain-text summary
validation_report.json	Evals v2 report: steps, code checks, checklist (see Write Validation Report)

Parameters

Parameter	Template Variable	Default	Description
Patient	{{patient}}	child#001	simglucose virtual patient: adult#001, adolescent#001, child#001 (paper's three), or any adult#001–#010, adolescent#…, child#…
Weight: glycaemic risk	{{w_glycemic}}	0.5	Your preference weight on r_gly (paper's baseline B: 0.5)
Weight: hypo safety	{{w_hypo}}	0.3	Your preference weight on r_hypo (paper: 0.3)
Weight: insulin exposure	{{w_dose}}	0.2	Your preference weight on r_dose (paper: 0.2). Weights are renormalized to sum to 1
Hypo limit	{{max_hypo_pct}}	4	Your maximum acceptable % time < 70 mg/dL (international consensus target: < 4%)
TIR tolerance	{{tir_tolerance}}	3	Two rules count as "same TIR" if within this many percentage points (paper: 3)
Days per rule	{{n_days}}	2	Simulated days (meal scenarios) per rule; ≥ 2 enables the paper's reliability filter
Seed	{{seed}}	1	Seed for meal scenarios and CGM noise

Dependencies

numpy, matplotlib, pandas, scipy (preinstalled or pulled in by simglucose) plus simglucose (pure Python; pulls in gym/gymnasium/pathos, all small). simglucose imports pkg_resources, so it needs setuptools<70. No other downloads.

Step 1: Set up

Write /app/work/params.json (numbers as JSON numbers; if a value still looks like an unsubstituted {{…}} placeholder, use the table default):

Validate before running: patient must match adult#0NN, adolescent#0NN or child#0NN with NN in 01–10 (else use child#001); weights must be ≥ 0 and not all zero (else use 0.5, 0.3, 0.2); clamp max_hypo_pct to [0, 50], tir_tolerance to [0.5, 10], n_days to [1, 4]. Record any correction as a caveat. The script itself lowers n_days if the sandbox has too few cores for the time budget (it prints a line saying so; mention it as a caveat).

Step 2: Get the code (extract it, don't retype it)

The tested implementation of the method is embedded in the Appendix at the end of this runbook. Do not rewrite, "improve", or re-derive it. Always run the command below first, exactly as written. It copies the files out of this runbook's text (which the sandbox exposes to running processes) in about a second, and it is far faster and more reliable than retyping several hundred lines. Do not skip it on the assumption that it won't work: run it and read its output.

If it exits non-zero or prints MISSING, write each missing file from the Appendix exactly as given, one heredoc per file (quoted delimiter, e.g. cat > experiment.py <<'PYEOF'). If a later step fails, fix only environment problems (missing package, network retry), never the methodology.

Step 3: Run the experiment

Expected: 25–60 s depending on cores. Ends with a done: line (max same-TIR hypo gap, your pick, % of unsafe preferences).

The sandbox has 4 CPUs even though nproc may report more; the exports keep numpy/BLAS and process pools from oversubscribing them.

Step 4: Interpret: write narrative.json

Read {{results_dir}}/results.json and card_data.json, and look at every fig_*.png. Write {{results_dir}}/narrative.json with keys headline, idea (exactly 3), findings (2–4), try_next (2–3), caveats (2–4):

• headline (≤ 30 words): the most striking result of this run with 1–2 numbers copied from results.json. Usually the worst matched pair (worst_pair.hypo_gap at worst_pair.tir_diff TIR difference), or that the participant's own weights pick a rule above their hypo limit (your_pick.hypo vs max_hypo_pct) when a safer rule with similar TIR exists.
• idea: (1) a scalar reward hides a trade-off chosen by the reward designer; (2) split it into glycaemic risk, hypo risk and insulin exposure and report the Pareto front instead; (3) TIR alone cannot tell a "dangerously low" policy from a "dangerously high" one.
• findings: (1) matched pairs: n_matched_pairs, gap_median, gap_max vs the paper's Table 1 row for the same cohort; describe the two traces in fig_pair_traces.png in plain words; (2) your weights' pick (your_pick: label, TIR, hypo, hyper) vs the two fixed weightings (baseline_A_pick = w (1,0,0), baseline_B_pick = paper's w) and whether any Pareto rule (rules with pareto: true) has similar TIR and lower hypo; (3) the simplex map: simplex_unsafe_fraction of weight vectors pick a rule above the hypo limit, simplex_distinct_rules different rules are reachable; (4) optional: differential vs absolute reward (diff_vs_abs_spearman, best_by_differential), noting that an undiscounted differential return telescopes to RI(start) − RI(end) and so is blind to everything in between.
• Use plain words: "hypo" = blood glucose below 70 mg/dL; "TIR" = % of time 70–180 mg/dL.
• try_next: 2–3 contrasting runs with exact values, using only the keys patient, w_glycemic, w_hypo, w_dose, max_hypo_pct, tir_tolerance, n_days, seed. Good contrasts: adult#001 (the paper's 42.8-point adult gap; the easiest patient), adolescent#001, weights leaning hard on hypo safety (e.g. 0.1, 0.8, 0.1), or a stricter tir_tolerance of 1.
• caveats (2–4): simulated patients (UVa/Padova via simglucose), not people; an enumerated family of simple basal-only rules replaces the paper's trained SAC / preference-conditioned agents, so "picks" are oracle optima within that family; one-day episodes with random meals and no meal boluses; not clinical guidance.

Step 5: Render the card and summary

Write {{results_dir}}/summary.md (5–10 lines: patient, weights, matched-pair stats vs paper, your pick, unsafe fraction, headline).

Writing style for narrative.json

These rules apply to every field and override any longer limits stated earlier in this runbook. Let the paper and the numbers speak; the card is not the place for commentary.

• Length: headline one sentence; idea 1–3 bullets of at most 30 words; findings 1–2 paragraphs of at most 60 words each; caveats 1–2 bullets of at most 20 words. Cut anything the tiles, figures or tables already show.
• Plain statements of what was measured. Write "Sensitivity gap −0.284 [−0.354, −0.218]; the paper reports +0.088", not an interpretation of what it means for the field.
• Avoid: em-dash asides; "not X, but Y" framings; colon-then-reveal sentences; rhetorical questions; words such as crucially, notably, strikingly, remarkably, collapses, illustrating, underscores, highlights, delve, powerful, key insight, worth noting; exclamation marks; restating the method before the result.
• Use the paper's own terms for its concepts. Quote the paper's numbers exactly.

Code Checks

Jetty runs these checks itself in the sandbox after the agent finishes, so they cannot be skipped or misreported. validate.py (see Write Validation Report) runs the same checks first so you can fix problems before you stop.

outputs-exist — Result card, numbers, narrative and summary were written

results-valid — Experiment numbers are valid JSON and the experiment stayed within its time budget

narrative-complete — The narrative is complete and within the length limits

try-next-valid — Every try-next suggestion uses only this runbook's parameters

headline-grounded — Every number in the headline appears in the run's results

card-integrity — The card embeds its figures, shows the headline, and contains no NaN

Checklist

• The headline states this run's finding with its numbers, not a description of the method
• Paper numbers are quoted exactly and any disagreement with the paper is stated plainly
• Caveats say what is synthetic, reconstructed or an extension of the paper
• Each try-next suggestion would plausibly give a contrasting result

Write Validation Report

Judge the four Checklist items honestly against your outputs and write them to /app/work/checklist.json as a list, using the checklist ids Jetty gives you (one entry per item):

Then write the report and self-check in one command:

validate.py writes {{results_dir}}/validation_report.json in the v2 shape Jetty reads (version: 2, one step entry per runbook step, one code_check entry per check above, one checklist entry per item). Jetty replaces the code_check entries with its own independent run of the same checks. Do not write the report by hand.

Iteration (up to 2 rounds)

If validate.py prints any FAIL, fix the named problem (usually a headline number that does not match results.json, or an unknown try_next key), re-render the card (Step 5) and rerun the command above. Never rerun the experiment just to change the narrative.

Appendix: code

These files are extracted verbatim in Step 2.

/app/work/experiment.py

/app/work/render_card.py

/app/work/validate.py