How exposed is your healthcare AI deployment?
Fourteen yes/no questions. Five risk categories. Live score. No email gate.
Calibrated against the Apr 2026 receipts: the Rao et al. PrIME-LLM JAMA Open paper (Harvard / MGB, 21 frontier LLMs evaluated, conclusion: current LLMs “cannot yet be relied on for unsupervised patient-facing clinical decision-making”), and the Utah Medical Licensing Board's April 20, 2026 letter recommending that Doctronic's AI prescription-renewal pilot be suspended.
Rao et al. 2026, JAMA Open · April 13. 21 frontier LLMs benchmarked on PrIME-LLM. The authors: “their most responsible role today is targeted, clinician-supervised use in low-uncertainty tasks”; current LLMs “cannot yet be relied on for unsupervised patient-facing clinical decision-making.”
Harvard / Mass General Brigham · published Apr 13 2026
Utah Medical Licensing Board → Doctronic · April 20. The board asked that Doctronic's AI prescription-renewal pilot be suspended. The state Office of AI Policy, which runs the sandbox, declined the next day, and the pilot continues with a licensed practitioner authorizing every renewal. Licensing boards and AI sandboxes already disagree about who answers for the output.
Utah Office of AI Policy, Doctronic pilot page (both letters) · STAT News, Apr 24 2026
27 states, 46 bills in 2026, seven enacted in six states. Delaware (HB 191) bars AI from being licensed to practice medicine; Texas requires practitioners to review AI-created medical records (SB 1188) and to disclose AI use to patients (TRAIGA); California requires disclosure of clinical genAI messages unless a licensed provider reviews them first. Oversight is becoming statutory, and physician liability is non-delegable.
Manatt Health AI Policy Tracker · bill texts: DE HB 191, TX SB 1188 and HB 149, CA AB 3030
Does every clinical-grade output (notes, plans, orders, claims) require a licensed physician to review and attest before downstream action?
What "yes" looks like: Hard intercept on every clinical write. Physician signature + timestamp + provenance.
Does the physician have at least 30 seconds of viewport-active time, with scroll-depth ≥ 80% on the AI output, before they can attest?
What "yes" looks like: Anti-rubber-stamp metrics enforced server-side AND in the database.
If the AI surfaces a high-uncertainty case, is there a configurable threshold that escalates to a second physician or blocks output entirely?
What "yes" looks like: Confidence-scored routing. "I don't know" is a first-class output.
Is there a signed BAA with every model provider that touches PHI (OpenAI, Anthropic, Google, etc.)?
What "yes" looks like: Active BAA on file, scoped to specific data flows.
Is PHI kept entirely out of URLs, browser logs, third-party analytics, and any LLM context window you don't control?
What "yes" looks like: Zero PHI in URL paths, query strings, browser storage, or non-BAA-covered telemetry.
Is there an automated PHI-redaction layer between the user input and the LLM prompt for every clinical surface?
What "yes" looks like: Named-entity redaction (names, MRNs, DOB, addresses) before any non-local model call.
For every AI output, can you produce — within 60 seconds, 7 years from now — the prompt, model, version, parameters, physician attestation, and downstream action?
What "yes" looks like: Append-only, hash-chained provenance ledger with external WORM anchor.
Are AI outputs anchored to a tamper-evident store (hash chain, blockchain, WORM bucket) that an external auditor can verify without your cooperation?
What "yes" looks like: Every signed decision hash-anchored in a public registry whose root is countersigned into Bitcoin daily (OpenTimestamps, or equivalent) — verifiable by anyone without trusting the operator.
When a model version changes, does every downstream artifact know which version generated it — including artifacts already attested and billed?
What "yes" looks like: Model version pinned per encounter; immutable on attestation.
For every state where AI output reaches a patient, is the attesting physician licensed in that state?
What "yes" looks like: State-of-patient → physician-license matching enforced at write-time.
Does every AI output stay within the attesting physician's own scope of practice and training (specialty-matched, with explicit deny on mismatch)?
What "yes" looks like: Specialty-of-AI → specialty-of-physician matching with explicit deny on mismatch.
If a state board issued a cease-and-desist tomorrow, can you flip a feature flag and stop AI-driven clinical output in that state within one hour, without a deploy?
What "yes" looks like: Per-state kill switch, hot-reloadable, with audit log of every toggle.
When the AI applies a clinical decision rule (Perry/ACEP for SAH, Wells for PE, HEART, NEXUS, etc.), does it explicitly verify the prerequisite conditions are present in the prompt before quoting the rule's statistical claims?
What "yes" looks like: Mandatory parameter checking. AI halts and asks for missing variables ("What was the exact time of symptom onset?") before applying any rule. Boundary conditions and exclusion criteria surfaced alongside every retrieval.
Does the AI integrate pre-test probability — including high-risk modifiers like cocaine use, uncontrolled HTN, family history of aneurysm, anticoagulation — into its recommendation, even when a general-population rule would otherwise rule out the diagnosis?
What "yes" looks like: Conservative defaulting on the diagnostic gradient. For "can't-miss" diagnoses (SAH, aortic dissection, MI, PE), AI defaults to the safest workup unless prerequisites for a rule-out rule are incontrovertibly met. Pre-test probability surfaced explicitly in every output.
Want a real audit?
HarnessHealth provides physician-attested AI governance infrastructure. If your self-check surfaced more than three "no" answers, bring the scorecard to a pilot request. The founder reads every one; there is no sales call, only a working session against your actual deployment.
HarnessHealth is the AI governance infrastructure layer of the SolvingHealth stack — physician-supervised AI deployment, hard-intercept attestation, audit-ready provenance.