74% of Healthcare AI Never Leaves the Pilot. Here's What's Actually Killing It.

I once watched a Stripe dashboard hit a million dollars in a day.

We were running a DTC health brand at the time, and someone had the dashboard up on a projector in a meeting room. Someone else started clapping. I remember thinking: this is the part where something goes wrong.

Not because I'm pessimistic. Because I'd learned by then that scale hides problems. The faster something grows, the longer it takes to notice the crack in what you built. Revenue is a great anaesthetic. You stop feeling the early warning signals right when they matter most.

I think about that often when I look at the state of healthcare AI. Billions went into it in 2025. Hundreds of companies promising transformation. And yet, according to data cited in Corti's February 2026 launch of their agentic framework, only 11% of AI agents actually reach production in healthcare. Innovaccer puts the number even lower: 74% of healthcare AI is stuck in pilots.

Not because the AI is bad. Because the floor beneath it keeps giving way.

Why do most healthcare AI projects never reach production?

The short answer: healthcare AI fails in production for the same reason a lot of systems fail at scale. The thing that made it seem to work in the pilot was the human attention surrounding it. Remove that attention, and you see what the system actually does.

In healthcare, the stakes of this are different from other industries. When an AI agent makes a coding error in a revenue cycle workflow, that error doesn't stay local. It propagates through downstream systems, corrupts billing records, affects reimbursement claims, and often looks correct to every system it passes through until a human catches it months later.

Corti calls this a "safety spiral." An error cascade that compounds faster than humans can intervene. It's not a dramatic failure. It's a quiet one. The system keeps running. The outputs keep generating. The damage accumulates.

What is a "safety spiral" and why does it kill healthcare AI deployments?

A safety spiral is what happens when an AI agent makes a mistake in a connected workflow and that mistake gets treated as ground truth by the next step in the chain. Clinical documentation feeds prior authorisation. Prior auth feeds claims. Claims feed financial reporting. If the AI hallucinates a diagnosis code at step one, by the time anyone notices, the error has been ratified by four other systems.

This is why most healthcare AI stays in pilots. Not because the model was wrong. Because nobody built the governance layer that would catch the error at step one.

The problem isn't intelligence. It's autonomy without accountability.

I see this pattern outside healthcare too. Any time I hear someone describe an AI deployment as "fully automated," I ask one question: who designed the failure mode? If they look confused, that's the answer.

At HANA, we designed for this from day one. Over one million patient interactions. Zero critical adverse events. That's not because the AI never makes a mistake. It's because every call has a structured escalation path, every flagged concern routes to a human, and every interaction is logged in an auditable format. You can read about how we approach safety in real deployments at hana.health/research.

Does the model matter more than the infrastructure?

Less than you'd think. Actually, a lot less.

This is going to sound counterintuitive, but the AI model is one of the least important decisions you'll make when deploying clinical AI. What matters far more is the architecture it runs inside: how decisions get logged, what triggers human review, what the system does when it encounters an ambiguous response, and how it integrates with the clinical workflows that already exist.

Corti frames it well: "Healthcare doesn't need smarter agents. It needs safer autonomy." That line hit me when I read it because it's the exact tension we've been living inside while building HANA.

Voice AI for patient follow-up is, on the surface, a pretty contained use case. The agent calls a patient. It asks a set of structured questions. It flags or it doesn't flag. But the downstream implications of getting it wrong, of missing a patient who's deteriorating, of failing to escalate a medication concern, are serious enough that the infrastructure around the call matters as much as the call itself.

HANA is fully open-source and self-hosted. No OpenAI dependency. No black box you can't inspect. The system you deploy is the system you can audit. You can explore the technical architecture at docs.hana.health.

What does "governed execution" actually mean for a clinical AI system?

It means the system has hard rules about what it's allowed to do, baked in at the execution layer, not applied after the fact as a content filter.

Corti's Agentic Framework uses a deterministic orchestrator that validates agent actions against clinical protocols before execution. Sub-millisecond validation. Complete audit trails. Every action traceable to a triggering input.

For voice AI in patient engagement, governed execution looks like this: the AI doesn't decide whether a patient is in crisis. It detects the linguistic signals that suggest concern, applies a threshold developed with clinical teams, and routes to a human who decides. The AI never substitutes for clinical judgment. It extends the reach of clinical judgment.

That's the design philosophy behind HANA's clinical use cases. The agent handles the volume. The human handles the judgment calls. Every time, documented, auditable, reviewable.

Why does open-source matter in healthcare AI?

It matters because trust in healthcare is institutional, not personal. A vendor saying "trust us, the AI is safe" is not the same as a health system being able to inspect the code, run it on their own infrastructure, and verify the behaviour themselves.

Most AI companies would rather you didn't look inside.

Open-source changes the accountability relationship. If you can read the code, you understand what the system does. If you can run it on your own servers, you own the data. If you can modify it, you can adapt it to your specific clinical workflow without waiting for a vendor release cycle.

HANA is fully open-source and self-hosted, with no dependency on commercial AI APIs. Five countries. Three languages. In production, not in pilots. The total cost of that infrastructure is orders of magnitude lower than proprietary alternatives, which is part of how clinics see a 31:1 ROI rather than a break-even on a subscription fee.

What should a health system actually demand before deploying AI at scale?

Three things, minimum.

Full audit trail. Every decision the AI makes should be traceable: what input triggered it, what threshold was crossed, what output was produced, what human action followed. If the vendor can't show you this, they haven't built it.

A defined failure mode. What happens when the AI encounters something it doesn't know how to handle? The answer shouldn't be "it tries its best." It should be a specific, designed escalation pathway that routes to a human.

Data ownership. Your patient data should never leave your infrastructure to train a vendor's model. Full stop. Ask the question explicitly. A lot of vendors will deflect. The right answer is: your data is yours, period.

If you want to walk through how HANA handles these questions for your specific environment, hana.health/about has the full background on how we've built this, and book a discovery call if you want to go deeper.

Key Takeaways

The gap between healthcare AI that works in a demo and healthcare AI that works in a real clinical environment is mostly an infrastructure story, not a model story. The research, the deployment data, and the pattern of where things go wrong all point in the same direction: governed execution, clear audit trails, and a hard limit on what AI is allowed to decide autonomously are what separate the 11% that reach production from the 89% that don't.

HANA was built inside that constraint. Voice AI for patient follow-up. Open-source. Self-hosted. Fully auditable. Not because we thought those things would be good marketing (honestly, "open-source voice AI" has never closed a deal faster), but because we were building for clinical environments and we knew what accountability looks like there.

The organisations that will successfully deploy AI at scale in healthcare are the ones asking the infrastructure questions now, before the safety spiral starts.

FAQ

Why do so few healthcare AI projects reach production?

Industry data suggests only 11% of healthcare AI agents make it to production. The primary reason is what Corti calls "safety spirals": error cascades where AI mistakes in connected workflows compound faster than humans can intervene. The solution is governed execution infrastructure, clear audit trails, and human-in-the-loop design at the system level, not just at the model level.

Is open-source healthcare AI safe for clinical deployment?

Yes, and arguably more verifiable than proprietary alternatives when deployed correctly. Open-source means the system's behaviour can be inspected and confirmed by the health system directly. HANA's open-source, self-hosted platform has processed over one million patient interactions across five countries with zero critical adverse events. Transparent systems are auditable systems. Visit hana.health/about for the full background.

How do you evaluate whether a healthcare AI vendor has the right infrastructure?

Ask three questions: Can you show me the full audit trail for a specific AI decision? What is the defined failure mode when the AI encounters something ambiguous? Will patient data ever be used to train your models? A vendor that answers all three clearly has built for production. If any answer is vague, the infrastructure isn't there yet.