SMASH 2026 · Paper #15 · Result card

Towards Transparent Reward Design in Clinical Reinforcement Learning

Caro Strickland, Baptiste Bonin, Audrey Durand · Université Laval · Read the paper

For child#001, two dosing rules within 3 points of time-in-range differ by 64.0 points in time hypoglycaemic. The paper's weights pick a rule at 16.4% hypo.

patient child#001weights (gly, hypo, dose) 0.50, 0.30, 0.20hypo limit 4%TIR tolerance ±3 ptssimulated days / rule 2dosing rules 48seed 1

Same TIR, different hypo

64.0 pts

max hypo-time gap among 154 rule pairs within 3 TIR points (median 8.9)

Your weights pick

48% TIR

basal 2/6 · +1 if >180; hypo 16.4%, hyper 35.3%

Two fixed weightings

50% vs 48%

TIR of the w = (1,0,0) pick vs the paper's w = (0.5,0.3,0.2) pick; hypo 21.5% vs 16.4%, insulin 16 vs 15 U/day

Preferences that break your limit

54%

of 231 weight vectors pick a rule with > 4% hypo time

From the abstract

Using automated insulin dosing as a testbed, we show that conventionally trained policies occupy one such undisclosed point on a tradeoff surface, producing clinically consequential failure modes invisible to standard metrics. We construct an approximate Pareto frontier through preference-conditioned RL and show that it recovers policies with similar aggregate performance but different risk profiles across three simulated patients.

Evidence

Which risk does each dosing rule trade for its time in range?

Who chose the trade-off? Every preference vector, mapped

Paper vs. your run

Metric	Paper	Your run
Matched pairs within 3 TIR pts (child)	148	154	41/48 rules kept (SD ≤ 20 pts, TIR ≥ 20%); paper: 40 preference vectors
Median hypo gap among matched pairs (pp)	6.4	8.9
Max hypo gap among matched pairs (pp)	39.8	64.0
child#001 fixed-weight failure modes	Baseline A hypo 55.2%; baseline B hypo 0.0%, hyper 56.0%	A-pick hypo 21.5%; B-pick hypo 16.4%, hyper 35.3%	paper baselines are SAC agents; yours are the best rule in the family
Differential (risk-decreasing) reward	a policy can collect higher reward by oscillating into and out of a hazardous state	rank corr. with absolute reward 0.50; its favourite rule has 15.0% hypo	undiscounted sum telescopes to RI(start) − RI(end)

Try next

Adult#001 — chase the paper’s 42.8-point gap — The paper reports a maximum 42.8-point matched-pair hypo gap for adult#001, making it the cohort where the TIR blind-spot is most dramatic. Running adult#001 lets you compare directly with the paper’s headline figure.
```
{"patient": "adult#001", "w_glycemic": 0.5, "w_hypo": 0.3, "w_dose": 0.2, "max_hypo_pct": 4, "tir_tolerance": 3, "n_days": 2, "seed": 1}
```
Hypo-safety-first weights — does your pick change? — Shifting most weight onto r_hypo should push the scalarized optimum toward the suspend-rule corner of the Pareto front and bring hypo time inside the 4% limit, demonstrating how the trade-off attribution shifts with the designer’s choice.
```
{"patient": "child#001", "w_glycemic": 0.1, "w_hypo": 0.8, "w_dose": 0.1, "max_hypo_pct": 4, "tir_tolerance": 3, "n_days": 2, "seed": 1}
```
Tighter TIR tolerance — how many pairs survive at ±1 point? — Reducing tir_tolerance to 1 requires rules to be within 1 TIR point to count as ‘same’, giving a stricter test of the paper’s claim. If large hypo gaps persist even at ±1 point, the finding is more alarming.
```
{"patient": "child#001", "w_glycemic": 0.5, "w_hypo": 0.3, "w_dose": 0.2, "max_hypo_pct": 4, "tir_tolerance": 1, "n_days": 2, "seed": 1}
```

Read this before sharing

48 simple dosing rules replace trained RL policies.
Simulated patients; not clinical guidance.