Early-Detection Medical AI as a Design Pattern for High-Stakes Alerts

Mayo Clinic researchers published results on REDMOD, a model that detects pancreatic cancer from routine abdominal CT scans up to three years before clinical diagnosis. Pancreatic cancer is one of the deadliest diseases on the planet, partly because by the time symptoms appear, the cancer is already advanced. A model that can flag it from a scan ordered for an unrelated reason is not a faster diagnosis. It is a different category of medicine.

What I want to talk about is not the medical breakthrough. I want to talk about the system design, because REDMOD solves a problem that any team building high-stakes alerts faces: how do you build a model that catches rare and consequential events without drowning your reviewers in false positives?

The pattern is not new. The execution discipline is what separates systems that work from systems that get switched off.

The two-stage architecture

REDMOD is a screening model. It does not diagnose. It flags scans that warrant a closer look by a specialist. The clinical workflow has two stages.

[Routine CT scan] -> [Screening model] -> [Risk score]
                                              |
                              risk > threshold? -> Yes -> [Specialist review]
                                              |
                                              No  -> [Standard report]

The screening model has one job: keep the false negative rate as low as possible while keeping the workload on specialists manageable. The specialist has the second job: make the actual call. The two are not interchangeable. The screening model is allowed to be wrong in one specific direction (false positives), and forbidden from being wrong in the other (false negatives).

This split lets you use a fast, less expensive model for the first pass, and reserve the expensive review (a human radiologist, a slower foundation model, an ensemble) for the small fraction of cases where the stakes justify it.

Calibration is the whole job

The hard part is the threshold. Set it too low and you flag every scan, the specialists ignore the alerts, and the system collapses into "another dashboard nobody looks at." Set it too high and you miss the cases the system was built to catch.

Calibration means tuning the model's output so that "70% risk" actually corresponds to a 70% rate of true positives in your population. Most ML systems are not calibrated. Their probabilities are scores, not probabilities. You can fix this with a small post-hoc step.

from sklearn.calibration import CalibratedClassifierCV

# Wrap your trained model
calibrated = CalibratedClassifierCV(
    base_estimator=trained_model,
    method='isotonic',  # or 'sigmoid'
    cv='prefit',
)
calibrated.fit(X_calibration, y_calibration)

# Now predict_proba gives actual probabilities
risk_scores = calibrated.predict_proba(X_test)[:, 1]

You need a held-out calibration set, separate from training and test. The calibration set should mirror the population the model will see in production. If you train on hospital A and deploy at hospital B, recalibrate.

False-positive budgeting

The right framing is not "minimize false positives." It is "what false positive rate can your reviewers absorb?" That number is your budget. You set the threshold to hit that budget, then you measure the false negative rate that falls out.

For REDMOD-style systems, the math looks like this.

Reviewers available: 10 radiologists, 1 hour per case
Total reviewer capacity: 50 cases per day
Daily scan volume: 5,000 scans
Maximum alert rate: 1% (50 cases)

Set threshold so that ~1% of scans are flagged.
Measure recall on validation set at that threshold.
If recall is too low, you need a better model
or more reviewer capacity. Not a different threshold.

This is the conversation that doesn't happen in most teams. The model team optimizes for a metric. The operational team finds out at deployment that the alert rate is unsustainable. The threshold gets cranked up to reduce volume. The model now misses what it was supposed to catch. The system is technically running and operationally useless.

Clinician-facing explanations

REDMOD does not just emit a score. It highlights regions on the scan that drove the prediction. The radiologist looking at the alert sees what the model saw.

This matters for three reasons. It builds trust, since reviewers can validate the model is looking at the right thing. It catches model errors, since a model focused on a scanner artifact instead of pancreatic tissue is obviously wrong to a human. And it satisfies regulators, since black-box decisions in medicine are increasingly not allowed.

The pattern: every alert ships with structured evidence. For images, attention maps or saliency. For text, highlighted spans. For tabular data, the top features and their values. The reviewer should never have to ask "why did the model flag this?" The answer ships with the alert.

Where else this pattern lives

Strip the medical context off and you have a general design for any high-stakes alerting system.

Fraud detection in payments. The screening model flags transactions, the analyst makes the call, calibration prevents the team from drowning in alerts, and the explanation shows which features drove the score (unusual location, atypical merchant category, velocity).

Predictive maintenance in industrial equipment. The screening model flags components likely to fail, the field engineer inspects, calibration tunes the rate to the available crew, and the explanation shows which sensor readings drove the prediction.

Security incident response. The screening model triages SIEM alerts, the analyst investigates, calibration is set to the team's bandwidth, and the explanation surfaces the indicators that matched.

Same pattern, three different industries. The discipline is identical: cheap fast first pass, expensive slow review, calibrated thresholds, evidence with every alert.

The trap to avoid

The trap is treating the screening model as the whole product. It is not. It is one component of a system that includes the reviewers, the calibration, the threshold tuning, the alert routing, the audit trail, and the feedback loop that improves the model over time.

REDMOD works because Mayo built the system, not just the model. They have the radiologists, the workflow integration, the IRB approval, the validation cohorts. The model is the easy part. The system is the hard part.

If you are building a high-stakes alert system, copy the system design before you copy the model. The model will get better. The system is what determines whether the better model actually saves lives, or money, or anything at all.