SMASH 2026 · Paper #8 · Result card
Implementing Ambient Artificial Intelligence (AI) Scribes in Hospital-based Specialty Care: A Mixed-Methods Evaluation on Administrative Burden and Clinical Workflows
Generic scribe drafts needed 78% of words edited. At the paper's median usage of 0.94% of encounters, a promised 4,550 clinic hours a month becomes 6.6.
specialty mixedencounters 6 syntheticbaseline note 6 minedit speed 1.5 s/wordencounters/month 390patients declining 10%trainee-led 20%promise 5 min/note
Edit burden of scribe drafts
78% → 27%
words changed to sign the draft: generic → specialty-customized
Net minutes per note
+1.5 / +3.3
vs writing it yourself in 6 min (generic / customized)
Clinic hours saved / month
6.6 h
at the paper's adoption, vs 4,550 h promised
Drafts needing substantial edits
33%
judge's call on generic drafts (paper survey: 88%)
From the abstract
Among 17 mid-pilot survey respondents, while 52% reported the AI scribe was easy to use, 88% reported that AI generated notes required substantial editing … These interim findings suggest that the implementation of an AI scribe is feasible in hospital-based ambulatory care but may not independently reduce documentation burden …
Evidence
Paper vs. your run
| Metric | Paper | Your run | |
|---|---|---|---|
| AI notes needing editing | 88% reported that generated notes required frequent editing (n=17 survey) | 33% generic / 17% customized drafts judged to need substantial edits | different construct: clinician perception vs LLM judge |
| Mismatch with specialty documentation | mismatch between output and specialty-specific documentation requirements (n=4) | format errors per draft 0.8 generic vs 0.3 customized | |
| Inaccurate content | inaccurate content information (n=4) | inaccuracy + hallucination errors per draft 0.0; planted-fact recall 100% | synthetic transcripts |
| Time savings | 41% felt they experienced overall time savings; 29% reported reduced documentation time | 83% of generic / 100% of customized notes faster than 6 min from scratch | modelled minutes, your assumptions |
| Adoption | 43 enrolled; 21 (49%) used the AI scribe | used as model input | calibration |
| Utilization among users | median 0.94% of encounters (IQR 3.04%), range 0.05-18.4% | lognormal σ = 1.87; mean 3.1% per active user | fitted, capped at 18.4% |
| Documentation time (EHR) | longer clinical notes, greater editing time, slightly more time spent documenting per encounter | drafts 205 words; finalizing a draft 4.5 min vs 6 min assumed baseline |
Try next
- Fast note-writer — A clinician who writes notes in 3 minutes and types slowly will likely see the scribe add net time, consistent with the paper's EHR finding of more time documenting. Edit overhead exceeds the baseline.
{"baseline_doc_minutes": 3, "edit_sec_per_word": 2.5} - Psychiatry only — The paper identifies psychiatry as where specialty-mismatch and personal-style barriers were most prominent. Isolating these two encounters should show the largest generic-vs-custom gap and highest format error counts.
{"specialty": "psychiatry"} - Academic clinic with heavy trainee load — Increasing the trainee share to 50% (academic teaching clinic) reduces eligible encounters substantially, shrinking hours at every rung of the ladder and making the barrier-removal lever especially visible.
{"trainee_share": 0.5}
Read this before sharing
- Transcripts, scribe and reviewing specialist are all simulated with LLMs.
- Time figures rest on this runbook's stated assumptions, not measured use.