Most healthcare AI evaluations over-weight the demo and the model, and under-weight the things that actually decide whether a tool survives contact with a hospital.
After building one of these products and using plenty of others at the bedside, here's the checklist I'd run a vendor through before signing anything. It's organized around the questions that predict success — not the ones that are easy to ask.
Clinical value
- Can the vendor name the specific decision or task this improves, in one sentence, without buzzwords?
- Does the value land with the clinician doing the work, or only with the executive buying it?
- What's the evidence — real deployments and outcomes, or a polished pilot and a wall of logos?
- Would a busy clinician choose to use this if it weren't mandated?
Workflow fit
- Where exactly does this live in the clinical day — inside the EMR, or in yet another tab?
- Does it remove steps or add them? Tools that create a second documentation path tend to die.
- How many clicks from "open" to "useful"?
- Who has to change their behavior for this to work, and has anyone actually asked them?
Trust and safety
- What happens when the model is wrong, and how does the clinician know?
- Is there a review pattern, or is the output trusted blindly?
- Who owns the risk when an error reaches a patient?
- How does the vendor handle fabrication specifically? "The model is very accurate" is not an answer.
HIPAA, security, and governance
- Will they sign a BAA, and what does it actually cover?
- Where does PHI live, who can access it, and how is that access logged?
- What's the data-retention policy, and can you delete on request?
- Is there an incident-response plan you can read — not just a reassurance?
- Are they training models on your data, and can you opt out?
Adoption and support
- What does implementation actually require from your team, and over what timeline?
- What's their real-world adoption rate at comparable sites — not their pilot rate?
- Who supports clinicians at 2 a.m. when it breaks?
- What's the off-ramp if it doesn't work: contract length, data export, switching cost?
The one question that cuts through
If you only have time for one, ask the vendor to walk you through a case where their tool failed and what they changed afterward. A vendor who can't tell that story either hasn't deployed at real scale or isn't being straight with you. The ones who can are usually the ones worth buying.
Prabhat Garg, M.D. is a practicing hospitalist and clinical AI strategy advisor. He helps health systems, health-tech companies, and investors decide what clinical AI to buy, build, avoid, and deploy.