Evidence & tools
Make the pilot
worth citing.
An open measurement protocol, a vendor scorecard and a publication checklist for independent practices. Practical tools to produce evidence you can inspect.
Published October 18, 2026 · MD Transform
Measure the whole workflow.
A fast AI step can move work somewhere else. Include preparation, review, corrections, handoffs and exceptions before describing a result as recovered capacity.
Pilot worksheet
Define the task, comparison, sample, outcome measures and stop criteria before the pilot starts.
Download the worksheet ↓Vendor evaluation scorecard
Score evidence strength separately from workflow fit. Record unanswered questions instead of treating a demo as proof.
Download the scorecard ↓Case study checklist
Document denominators, limitations and consent before turning a pilot into a public claim.
Download the checklist ↓These are planning tools developed for this library, not reported study results, validated instruments or independent-practice benchmarks. No client outcomes or sample sizes are claimed here.
A comparison someone else can understand.
Published October 18, 2026. Adapt the design to the risk and volume of the workflow before collecting data.
Define one unit of work
Specify what counts as an eligible call, note, authorization request or message thread. Write exclusions and the required completed outcome before you see results.
Record a comparable baseline
Capture active staff minutes and elapsed time separately. Record staffing, volume, case mix, payer mix where relevant, and other changes that could affect the comparison.
Count every eligible attempt
Record baseline and pilot sample sizes independently. Include failures, transfers and manual fallbacks. Record missing observations and reasons for exclusion.
Account for review and rework
Capture preparation, AI interaction, review, correction, exception handling and handoff time. Record clinician and staff effort separately when the work shifts between roles.
Report quality beside time
Define a material error for the task before auditing. Record the number audited, the audit method, reviewer role, disagreements and unresolved exceptions.
Publish the boundary of the result
Report dates, units, denominators, distributions, exclusions and limitations. A small uncontrolled before/after pilot describes an observation; it cannot isolate causality or establish a clinical benefit.
Use definitions that travel with the numbers.
| Measure | Definition | Report alongside it |
|---|---|---|
| Active effort per task | Total observed preparation, handling, review, correction and exception minutes divided by eligible tasks. | Sample size, role, median, range and missing observations. |
| Observed effort change | Baseline minutes per task minus pilot minutes per task. Relative change divides that difference by baseline minutes; omit the percentage when baseline is zero. | Both raw values, case mix and staffing changes. Negative values mean more work. |
| Correct completion rate | Tasks meeting the prespecified completion and quality rules divided by all eligible attempts. | Failures, transfers, abandonments and a separate audit sample. |
| Material correction rate | Audited outputs requiring a predefined material correction divided by all audited outputs. | Error categories, consequence, reviewer role and selection method. |
| Cost per completed task | Allocated license, integration, support, usage and labor cost divided by correctly completed tasks. | One-time costs, allocation assumptions, period and volume. |
Make the evidence inspectable.
A future case study should identify the workflow, method, comparison dates, sample sizes, outcome definitions, limitations, funding or vendor relationships, author, reviewer and substantive update history.
Before releasing a case study
- Obtain permission to publish and remove patient and identifying client information.
- Include unfavorable outcomes, failed attempts and missing data.
- Distinguish observed time savings from capacity actually redeployed.
- Label simulated or illustrative values explicitly; never mix them with observations.
- Separate measured practice results from vendor-reported claims.