{
  "headline": "Equal weights hand gamified a narrow win (55% of resamples), but the verdict flips if learning's share of the E/L weight exceeds 51%.",
  "idea": [
    "Engagement, learning, intention and persistence are all legitimate educational outcomes, but they need not move together — an AI that picks one proxy optimizes a normative choice it may never have stated.",
    "Keeping E, L, B and P as separate outcome classes makes the weighting explicit (outcome accountability): the 'best' intervention changes as soon as those weights change.",
    "Immediate measurements like engagement may not predict durable change; the paper's T2 follow-up is still ongoing, which is why persistence here is an assumption rather than data (temporal accountability)."
  ],
  "findings": [
    "**Case 1 (STEAM crossover, n = 49):** Gamified beats conventional on all three engagement outcomes (E d_z = 0.58) but loses on the single learning outcome (L d_z = −0.55). Under your equal E/L weights (renormalized, since Case 1 never measured B or P), gamified just wins — composite utility +0.017 — and that holds in 55% of 4,000 bootstrap resamples. The margin is tiny: the flip point is a learning share of 51%, only 1 percentage point above your current 50% position. Fifty percent of your total weight (the B and P shares) is on outcomes Case 1 never measured; the card reports what that unmeasured 50% means for how much you can trust this verdict.",
    "**Case 2 proxy — can engagement find who learned?** With the paper's r ≈ .24 between engagement and knowledge gain, an AI that picks the top third of learners by engagement captures 43% of the actual top-third knowledge gainers — above the 33% chance level but far from reliable. About 23% of synthetic learners pile up at the 7-point scale ceiling (engagement = 7), introducing saturation. Removing the ceiling entirely (`proxy_capture latent` = 43%) barely changes the capture rate, so the ceiling is not the main reason the proxy is weak — the correlation itself is just modest. The proxy capture curve shows that capturing two-thirds of top gainers would require r ≈ 0.55, roughly twice the paper's observed value.",
    "**Case 2 — four 'success' numbers for one intervention:** Engagement sits 3.68 SDs above the scale midpoint, making the card game look like an overwhelming success on that metric. Learning (mean of knowledge d_z = 1.53 and competence d_z = 1.99) averages 1.76 — large but more modest. Behavioural intention barely moves: d_z = 0.03. Persistence at 0.88 is purely your 0.5 retention assumption applied to the learning effect — T2 is ongoing. An AI that watched only engagement would pick this intervention and see enormous effect sizes; an AI watching intention would see almost nothing."
  ],
  "try_next": [
    {
      "label": "Tip past the flip: weight learning more than engagement",
      "why": "With the flip at L-share 51% and your current position at 50%, shifting just a little weight from E to L reverses the winner. This run makes the reversal explicit.",
      "params": {"w_engagement": 0.20, "w_learning": 0.30, "w_intention": 0.25, "w_persistence": 0.25, "seed": 3}
    },
    {
      "label": "Scale up to n = 200 — does more data firm up the verdict?",
      "why": "At n = 49 the gamified win is only 55% across resamples. A larger synthetic cohort shows whether the uncertainty shrinks or whether the close d_z values mean both conditions are genuinely plausible.",
      "params": {"w_engagement": 0.25, "w_learning": 0.25, "w_intention": 0.25, "w_persistence": 0.25, "n_case1": 200, "seed": 3}
    },
    {
      "label": "Raise the engagement ceiling to 10 — does the proxy improve?",
      "why": "23% of Case 2 learners hit the 7-point ceiling. Using a 10-point scale removes most of that saturation; check whether the proxy capture rate rises meaningfully or stays near 43%.",
      "params": {"w_engagement": 0.25, "w_learning": 0.25, "w_intention": 0.25, "w_persistence": 0.25, "engagement_ceiling": 10, "seed": 3}
    }
  ],
  "caveats": [
    "All learners in this run are synthetic: they were generated by a random-number model calibrated to the paper's published summary statistics (means, SDs, d_z and confidence intervals), not sampled from the authors' original data, which are not public.",
    "Correlations between outcomes that the paper did not report — such as the relationship between an individual's engagement benefit and their learning benefit in Case 1 — are model assumptions (here: zero); the results would shift under different assumptions.",
    "Persistence (P) is your own assumption: the value 0.88 is the chosen retention fraction (0.5) applied to the learning effect, not a measurement. The paper's T2 follow-up is still ongoing.",
    "All outcome measures are self-reports, as the paper itself notes; perceived knowledge gain and engagement are not the same as objective performance or long-term retention."
  ]
}