- Epic and other major vendors are shipping autonomous healthcare AI agents that can take actions inside the EHR, but public validation strategies for these agents are largely absent.
- Healthcare AI is shifting from “suggestion mode” (AI recommends, human acts) to “agent mode” (AI acts, human may review), which fundamentally changes the risk profile and what validation must prove.
- Existing clinical validation frameworks and FDA pathways are built for recommendation/diagnostic tools and don’t adequately cover autonomous operational agents that write notes, submit claims, or message patients.
- Agent failures have a much larger error surface and higher-stakes failure modes (e.g., incorrect chart entries, billing fraud exposure, contradictory patient instructions), so they shouldn’t share the same validation approach as suggestion tools.
- Healthcare is adopting agentic AI early despite being a high-consequence environment, creating a regulatory and safety gap for autonomous behavior in clinical and administrative workflows.
Epic Systems just stood on stage at HIMSS 2026 and introduced three AI agents to the largest gathering of healthcare IT professionals in the world. "Art" writes clinical notes. "Penny" handles hospital billing. "Emmie" manages patient communication and scheduling. Not suggestion tools that highlight a recommendation and wait for a human to click approve. Autonomous agents that take actions inside the electronic health record, the single most consequential software system in a patient's care journey.
Big applause. No validation strategy in sight.
Epic isn't alone. Google, Microsoft, and Oracle are all pushing healthcare AI agents through their cloud platforms, racing to embed autonomous decision-making into clinical and administrative workflows. The competitive pressure is obvious, the EHR market is massive, sticky, and desperate for efficiency gains. But nobody on any of these stages talked about the gap between deploying an agent and proving that agent is safe to deploy.
The shift nobody is treating like a shift
For the past three years, healthcare AI has mostly operated in suggestion mode. The model reads a chest X-ray and highlights a potential finding. A physician reviews it, agrees or disagrees, moves on. AI assists, human decides. Existing clinical validation frameworks, built around sensitivity, specificity, and FDA clearance pathways, were designed for exactly this setup. They work reasonably well for it.
Agents are a different animal entirely. An agent doesn't suggest. It acts. Art doesn't recommend a clinical note for the physician to review and edit. It generates the note and places it in the record. Penny doesn't flag a billing code for a coder to verify. It processes the bill. The human may still be in the loop somewhere, but the default has flipped. Instead of a human acting on an AI's suggestion, a human is now reviewing an AI's completed action. If they review it at all.
That inversion changes everything about validation.
| Capability | Suggestion Mode | Agent Mode |
|---|---|---|
| Clinical notes | AI drafts, physician writes | AI writes, physician may review |
| Billing | AI flags codes, coder confirms | AI submits claims, exceptions reviewed |
| Patient comms | AI suggests message, staff sends | AI sends message, staff monitors |
| Error surface | Limited to flagged items | Every action the agent takes |
| Failure mode | Missed suggestion (low risk) | Wrong action taken (high risk) |
| Validation need | Accuracy of recommendations | Accuracy, safety, and behavioral bounds of autonomous actions |
When a suggestion tool gets something wrong, the worst case is a clinician ignoring a useful flag. When an agent gets something wrong, the worst case is a billing fraud claim, an incorrect medication note in the chart, or a patient receiving a message that contradicts their care plan. Not the same risk category. Shouldn't share a validation framework.
The 6% problem
Across all industries, only about 6% of enterprises have fully implemented agentic AI. Healthcare is one of the most aggressive adopters. Sounds like good news until you think about what "most aggressive" actually means here. The sector with the highest consequences for error is also among the first to hand autonomous capabilities to software that has no established validation standard for autonomous behavior.
The FDA has a clearance pathway for AI as a medical device. It covers diagnostic algorithms, imaging analysis tools, clinical decision support. It does not cover an agent that autonomously generates clinical documentation, processes insurance claims, or sends patient communications. Those functions sit in a regulatory gap, too operational for medical device oversight, too consequential for no oversight at all.
Epic, to their credit, built these agents on top of their existing EHR platform, which means they inherit some access controls and audit logging that healthcare IT already requires. But access controls aren't validation. Logging that an agent took an action isn't the same as proving the action was correct. And the sheer volume of actions an agent can take (thousands per hour across a health system) makes human review of every action mathematically impossible.
What a real validation framework would require
The gap isn't that validation is impossible. It's that nobody has built the framework yet, and the deployments aren't waiting. From what I've seen working with organizations deploying AI in regulated environments, a credible healthcare agent validation framework would need at least five components that don't exist today.
Behavioral boundaries, not just accuracy metrics. Traditional AI validation asks: how often is the model correct? Agent validation needs a different question: what is the model allowed to do, and does it stay within those bounds? An agent that generates billing codes with 98% accuracy but occasionally submits claims for procedures that never happened isn't a 98% accurate system. It's a liability.
Continuous monitoring, not point-in-time testing. FDA clearance for a diagnostic AI is a snapshot. You test the model, demonstrate performance, get clearance. Agents operate continuously and their behavior can drift as underlying models update, as the data environment changes, as edge cases accumulate. Validation has to be continuous too.
Adversarial testing for healthcare-specific failure modes. What happens when an agent encounters a patient with an unusual name that confuses its parsing? When billing codes change mid-quarter? When a patient responds to an automated message with a medical emergency? These aren't theoretical scenarios. They're Tuesday.
Explainable action chains. When a physician writes a note, you can ask them why they wrote what they wrote. When Art writes a note, you need an equivalent. Not just "the model generated this text," but a traceable chain from input data to output action that a compliance officer or malpractice attorney can actually follow.
Cross-agent interaction testing. Epic now has three agents operating in the same EHR environment. What happens when Penny's billing decision depends on a note that Art generated incorrectly? When Emmie schedules a follow-up based on a billing status that Penny changed? Agent-to-agent failure cascades are a known problem in software engineering. In healthcare, those cascades hit patients.
| Validation Component | Current Status | Risk of Absence |
|---|---|---|
| Behavioral boundaries | Not standardized | Agent takes actions outside intended scope |
| Continuous monitoring | Rare in practice | Performance degradation goes undetected |
| Adversarial testing | Ad hoc at best | Edge cases cause harm in production |
| Explainable action chains | Not required | Liability and compliance exposure |
| Cross-agent interaction testing | Largely unexplored | Cascading failures across workflows |
The deployment pressure is the problem
I get why Epic, Google, Microsoft, and Oracle are moving fast. The administrative burden in healthcare is genuinely crushing. Physicians spend two hours on documentation for every one hour of patient care. Billing errors cost the U.S. healthcare system billions annually. Patient communication is fragmented, inconsistent. Real problems. AI agents are a plausible solution.
But "plausible solution" and "validated solution" are different things, and healthcare has learned this lesson before. Electronic health records themselves were deployed under similar pressure, promising efficiency and quality improvements that took a decade to partially materialize. The unintended consequences (physician burnout, alert fatigue, interoperability failures) nobody validated for in advance.
The pattern is familiar. Technology with genuine potential arrives. Pressure to deploy is enormous. Validation frameworks lag behind. Early adopters discover failure modes in production, on real patients, instead of in controlled evaluation.
The right response isn't to stop deploying healthcare AI agents. The administrative waste is real, the potential efficiency gains matter. The right response is to build the validation frameworks in parallel with the deployments, not after them. Health systems should be demanding validation standards from their vendors, not just feature demos. The FDA or an equivalent body needs to address the regulatory gap for autonomous operational AI in healthcare. And the 94% of enterprises that haven't yet fully implemented agentic AI should treat that number as a chance to learn from the 6% who went first. Including learning from their mistakes.
Epic put three agents on stage at HIMSS. The question that should have followed every demo isn't "when can we have this," but "how do you know this is safe." Until that question gets the same stage time as the product announcement, healthcare AI adoption is running ahead of the infrastructure meant to keep it honest.
If you’re evaluating or deploying these agents, treat them like autonomous operators, not smarter assistants: ask vendors to show how they bound behavior, measure safety beyond accuracy, and handle edge cases and escalation. Insist on clear human-review points (or explicit justification for when there is none), robust audit trails, and monitoring that can detect harmful actions—not just model errors. Watch for how your organization will manage accountability when an agent’s “completed action” becomes the default in documentation, billing, or patient communications.