How exposed is your healthcare AI deployment?
Fourteen yes/no questions. Five risk categories. Live score. No email gate.
Calibrated against the Apr 2026 receipts: the Rao et al. PrIME-LLM JAMA Open paper (Harvard / MGB, 21 frontier LLMs evaluated, conclusion: off-the-shelf LLMs unsafe for unsupervised clinical use), and the Utah Medical Board's first-of-kind suspension of Doctronic autonomous AI prescribing on Apr 24, 2026.
Rao et al. 2026, JAMA Open · April 13. 21 frontier LLMs benchmarked on PrIME-LLM. Conclusion: "the responsible role is targeted, clinician-supervised use in low-uncertainty tasks" — off-the-shelf LLMs unsafe for unsupervised clinical use.
Harvard / Mass General Brigham · published Apr 13 2026
Utah Medical Board → Doctronic · April 24. First state regulator suspension of autonomous AI prescribing. Crystallizes the physician-attestation thesis at the regulatory layer.
STAT News + Health Affairs op-ed · Apr 24 2026
25 states, 40+ bills in 2026. Delaware bars AI from being licensed to practice medicine; Texas (TRAIGA) mandates human review of AI outputs in the EHR; California requires disclosure of clinical genAI. Oversight is becoming statutory, and physician liability is non-delegable.
Holland & Knight + AMA state trackers · 2026
Does every clinical-grade output (notes, plans, orders, claims) require a licensed physician to review and attest before downstream action?
What "yes" looks like: Hard intercept on every clinical write. Physician signature + timestamp + provenance.
Does the physician have at least 30 seconds of viewport-active time, with scroll-depth ≥ 80% on the AI output, before they can attest?
What "yes" looks like: Anti-rubber-stamp metrics enforced server-side AND in the database.
If the AI surfaces a high-uncertainty case, is there a configurable threshold that escalates to a second physician or blocks output entirely?
What "yes" looks like: Confidence-scored routing. "I don't know" is a first-class output.
Is there a signed BAA with every model provider that touches PHI (OpenAI, Anthropic, Google, etc.)?
What "yes" looks like: Active BAA on file, scoped to specific data flows.
Is PHI kept entirely out of URLs, browser logs, third-party analytics, and any LLM context window you don't control?
What "yes" looks like: Zero PHI in URL paths, query strings, browser storage, or non-BAA-covered telemetry.
Is there an automated PHI-redaction layer between the user input and the LLM prompt for every clinical surface?
What "yes" looks like: Named-entity redaction (names, MRNs, DOB, addresses) before any non-local model call.
For every AI output, can you produce — within 60 seconds, 7 years from now — the prompt, model, version, parameters, physician attestation, and downstream action?
What "yes" looks like: Append-only, hash-chained provenance ledger with external WORM anchor.
Are AI outputs anchored to a tamper-evident store (hash chain, blockchain, WORM bucket) that an external auditor can verify without your cooperation?
What "yes" looks like: Daily SHA-256 anchor of the decision ledger to a 7-year-COMPLIANCE-locked R2 bucket (or equivalent).
When a model version changes, does every downstream artifact know which version generated it — including artifacts already attested and billed?
What "yes" looks like: Model version pinned per encounter; immutable on attestation.
For every state where AI output reaches a patient, is the attesting physician licensed in that state?
What "yes" looks like: State-of-patient → physician-license matching enforced at write-time.
Does every AI output stay within the attesting physician's own scope of practice and training (specialty-matched, with explicit deny on mismatch)?
What "yes" looks like: Specialty-of-AI → specialty-of-physician matching with explicit deny on mismatch.
If a state board issued a cease-and-desist tomorrow, can you flip a feature flag and stop AI-driven clinical output in that state within one hour, without a deploy?
What "yes" looks like: Per-state kill switch, hot-reloadable, with audit log of every toggle.
When the AI applies a clinical decision rule (Perry/ACEP for SAH, Wells for PE, HEART, NEXUS, etc.), does it explicitly verify the prerequisite conditions are present in the prompt before quoting the rule's statistical claims?
What "yes" looks like: Mandatory parameter checking. AI halts and asks for missing variables ("What was the exact time of symptom onset?") before applying any rule. Boundary conditions and exclusion criteria surfaced alongside every retrieval.
Does the AI integrate pre-test probability — including high-risk modifiers like cocaine use, uncontrolled HTN, family history of aneurysm, anticoagulation — into its recommendation, even when a general-population rule would otherwise rule out the diagnosis?
What "yes" looks like: Conservative defaulting on the diagnostic gradient. For "can't-miss" diagnoses (SAH, aortic dissection, MI, PE), AI defaults to the safest workup unless prerequisites for a rule-out rule are incontrovertibly met. Pre-test probability surfaced explicitly in every output.
Want a real audit?
HarnessHealth provides physician-attested AI governance infrastructure. If your self-check surfaced more than three "no" answers, schedule a 30-minute review with our team. No sales call — just a working session against your actual deployment.
HarnessHealth is the AI governance infrastructure layer of the SolvingHealth stack — physician-supervised AI deployment, hard-intercept attestation, audit-ready provenance.