Deliverable 03 · Measurement
Did the support system actually move the needle?
Four KPIs, four data sources. Each one ties a measurable outcome back to a part of the workflow or learning blend.
Time-to-proficiency
6 wks
−50% vs baseline
Preceptor sign-off + scenario scores
Mis-triage rate
4.6%
−54% vs baseline
EHR retrospective audit
Workflow consistency
86%
+18 pts vs baseline
Inter-rater ESI agreement study
Learner confidence
8.1 / 10
+2.9 vs baseline
Post-shift pulse survey
Time-to-proficiency
Weeks to independent triage sign-off. Current cohort uses TriageOS; prior cohort did not.
Source · preceptor sign-off log + monthly scenario scores
Error reduction
Mis-triage and documentation error rates, before vs after PSS rollout.
Source · EHR retrospective audit, n=1,420 charts
Workflow consistency
ESI assignment agreement vs gold-standard cases, by nurse and acuity level. ESI 3 is consistently the hardest call.
| ESI 1 | ESI 2 | ESI 3 | ESI 4 | ESI 5 | |
|---|---|---|---|---|---|
| N. Alvarez | 98 | 92 | 78 | 88 | 96 |
| T. Bryant | 97 | 88 | 72 | 84 | 94 |
| S. Chen | 96 | 90 | 80 | 86 | 95 |
| R. Davies | 95 | 85 | 68 | 80 | 92 |
| M. Eze | 99 | 94 | 82 | 90 | 97 |
| L. Fox | 94 | 86 | 70 | 82 | 93 |
| K. Gomez | 97 | 91 | 76 | 87 | 95 |
| J. Huang | 96 | 89 | 74 | 85 | 94 |
% agreement with gold-standard ESI assignment
Source · quarterly inter-rater reliability study, 40 cases per nurse
Learner confidence
Self-rated confidence across six triage competencies, pre vs post.
Source · pulse survey, n=84 nurses, 10-point scale
What this tells us
- Faster autonomy: nurses sign off ~6 weeks earlier, freeing preceptor capacity.
- Safer judgment: under-triage — the most dangerous error — dropped by more than half.
- Targeted gap: ESI 3 consistency lags. Next iteration adds an ESI 3-specific decision tree.
- Confidence ≠ competence: we track both, and the gap between them is the early-warning signal.