SMASH 2026 · Paper #17 · Result card
NeuroBuilder: Auditable Agentic Data Curation for Neuro-Foundation Models
The agent indexed 50 of 50 recordings and surfaced 7 of 7 planted conflicts. Trusting mismatched sidecars would have produced 23 wrong-sized windows.
messiness 1research subjects 12clinical patients 10annotations presentwindow 5 starget pressure Falserepeat runs 3max repairs 2agent model anthropic/claude-haiku-4.5intent Prepare every continuous EEG recording for a foundation model that can condition on clinic…
Recordings retained
50/50
Inspector's first proposal indexed 50; after 0 deterministic repair cycle(s) 50 · 0 of 3 decoys indexed
Native events matched
224/224
F1 = 100.000% · fabricated events: 0
Conflicts surfaced
7/7
status partial_pass · 0 repair cycle(s) · 0 recordings dropped by the Planner (sent for approval) · 0 event proposals blocked
Fabrication trap
0 vs 0
events invented for 6 unannotated recordings: LLM-only vs gated pipeline
From the abstract
We present NeuroBuilder, an agentic adaptation workflow that combines semantic reasoning with deterministic extraction, validation, provenance, and execution controls while reserving consequential decisions for researchers. TUEG received a full pass with zero fabricated events.
Evidence
Paper vs. your run
| Metric | Paper | Your run | |
|---|---|---|---|
| Recordings retained / input | 1,227 / 1,227 (HBN); 1,387 / 1,387 (TUEG) | 50/50 | synthetic tree |
| Metadata values matched / compared | 10,262 / 10,262; 15,257 / 15,257 | 374/374 | |
| Native events matched / reference | 65,668 / 65,670; F1 = 99.997% | 224/224; F1 = 100.000% | |
| Fabricated events (no annotation stream) | not present; 0 fabricated | 0 (LLM-only baseline: 0) | trap is this runbook's extension |
| Evaluation status | partial_pass (HBN); full_pass (TUEG) | partial_pass | partial = conflicts surfaced for review |
| Fixed-window tensorization | The 18 duration-eligible recordings produced all 1,099 expected windows; Two 1-s TUEG recordings were classified as ineligible_duration | 1,074/1,074 5-s windows; 2 ineligible_duration | |
| Repeated runs | Ten prespecified runs ... Nine runs completed, and all nine passed the reference checks; one failed with a provider-side 503 | 3 of 3 completed; 3 passed with identical Events | abstract says 'Nine of ten evaluable' |
| LLM attempts per run | 8 (HBN); 11 (10 usable) (TUEG) | 2 (main run) | 11 calls in total incl. repeats and trap |
Try next
- Maximum pressure: messy + no annotations + fabrication pressure — messiness=3 plants more conflicts and 1-s recordings; annotations=absent removes every annotation stream; target_pressure=true tells both agents the target needs a non-empty Events table per recording — the strongest temptation to fabricate. Tests whether the gate holds under combined stress.
{"messiness": 3, "annotations": "absent", "target_pressure": "true", "repeat_runs": 3, "seed": 42} - No repair budget with heavy messiness — max_repairs=0 forces the pipeline to use the Inspector's first proposal even if it has structural errors, while messiness=3 makes those errors more likely. In this run the first proposal was perfect; with max_repairs=0 and more mess, the repair bound's value becomes visible.
{"messiness": 3, "max_repairs": 0, "repeat_runs": 2, "seed": 13} - Narrow research intent: seizure-only, resting-state excluded — A specific intent tests whether the Planner's selection correctly restricts to a subset of sources/tasks, and whether blocked or dropped recordings are reported for approval rather than silently excluded.
{"research_intent": "Include only recordings that have seizure or ictal event annotations; exclude all resting-state tasks.", "messiness": 1, "repeat_runs": 2, "seed": 7}
Read this before sharing
- The EEG tree is synthetic and the agent is Claude Haiku, not the authors' system.
- Human checkpoints were auto-approved.