PAN Lab example
IDx-DR Autonomous Screening
The decision with no reader: an authorized autonomous diagnostic
A medical assistant who has never taken a retinal photograph sits a patient with diabetes at a camera, takes two pictures of each eye, and about a minute later the software says one of exactly two things: refer to an eye care professional, or negative, come back in twelve months. No clinician reads the image. No clinician reads the result before it is the result. Modeled on the first device the FDA authorized, in April 2018, to return a screening decision without a clinician also interpreting the image or the result — the first such authorization in any field of medicine. Most boards in this atlas show a human check being worn away by workload, deference or bad alerts. This one shows the check being removed on purpose, in public, by a regulator, with the compensating controls named out loud: a fenced patient population, one paired camera, a two-message output, an internal gate that refuses to classify an image it judges insufficient, a twelve-month rescreen cadence, postmarket surveillance, and a manufacturer that says it insures the diagnosis. The record that followed is comparatively good, and this network does not pretend otherwise. Screening completion went from 22% to 100% in a randomised trial arm; adherence rose 7.4 points across 30-plus primary care sites while comparison sites stood still, with the largest gains among Black patients; direct queries of the federal device databases in August 2026 returned no adverse-event reports and no recalls. Two things stay honest anyway. The gate that makes autonomy defensible refused a quarter of patients in independent real-world use — the older, cataract-prone ones a screening programme most needs — against 4% in the trial. And a negative message books a year of scheduled quiet, which is the one failure the postmarket watch was never shaped to see. Before you pick a target level: this board cannot be won under Service and Safety Targets or All Governance Targets, and money is not what blocks it. Inside the budget the failure regime comes down to calm and the benefit reads clear of both margins — one strong escalation lever wins Explore and Service Targets Only on its own. The pathway gate is the only gate that fails, and it fails at any budget: the whole nine-lever list at once, costing more than twice what you have, still leaves nine pathways open. They are the camera feed and the labeling the software reads, the eligibility screen and capture the operator performs, the result issuing to the clinician, the decision writing itself into the chart, two ordinary pieces of record-keeping, the handoff between the two staff, and the one locked classifier running in every clinic. Closing that set means not screening anyone. That is a measurement of what this authorization actually built, not a puzzle waiting to be cracked. Explore and Service Targets Only can be won.
Open this example in PAN Lab v0.1 to apply pressures and levers and watch what the system does.
What this models
This example runs on the IDx-DR-class autonomous retinal screening network: 10 components and 22 pathways between them. Every context in the Lab is a stylized model, never a reconstruction of any actual deployment, and each assumption behind it carries a provenance label.
Evidence base: 4 assumed · 5 published baseline. In the Lab, the shaded evidence band behind each headline readout draws its width from the least-established class below.
- assumed
D48-derived new org (Phase 6, clinical-decision-support). TOPOLOGY. Ten nodes, all documented, none decorative. The structural claim is an ABSENCE that the evidence makes affirmative: there is no interpretive human node between the model and the clinical decision, because the FDA's April 2018 De Novo grant authorized a screening decision issued without a clinician also interpreting the image or the result — the first such authorization in any field of medicine. Every human class here therefore sits around the loop rather than in it: an operator who screens eligibility and captures images, a provider who receives a two-message result, an eye-care service reached only by the refer output, and a postmarket function reading a device docket. Four further structural choices are equally derived. A GUARDRAIL, because the image-quality gate is the deployment's only refusal channel and is exactly a bounded automated screen — it carries modelToOperatorCheck, partial by construction, screening image quality and never the classification that passes it. An INPUTSOURCE, because the authorization pairs the software to one nonmydriatic camera model and every measured driver of real-world non-analyzability sits on that feed. NO enforcement node, because nothing downstream is actioned automatically from the record. NO worklist, because no source documents a queue, backlog or response clock — follow-through is measured as a completion share. NO externalBoundary, because the analysis is point of care and no egress, unsanctioned tool or ungoverned host appears anywhere in the record.
- baseline
THE CLASSIFIER READS TWO STORES AND NO CHART, and that is a derivation rather than a simplification. The paired camera's images and the locked configuration are its whole input; the eligibility facts in the patient's chart reach the decision through the OPERATOR who administers the labeled exclusion list, which is why that pathway is drawn operator-to-model and not as a second record feed into the model. This is precisely what separates the shape from the domain's alerting siblings, whose models read the electronic health record continuously and score every patient in it. The same derivation explains an absence on the other side: the algorithm is locked, so nothing writes back into the configuration store except a federal version review, and the record-contamination loop that every learning deployment in this catalogue carries does not exist here by construction.
- baseline
DEMAND 3 / CAPACITY 1. Demand 3: the eligible population is every adult with diagnosed diabetes every year in general primary care (roughly 17,600 diabetes patients a year across one system's 30-plus sites; roughly 151,000 across a five-system programme), and the documented state of that workload is a standing shortfall — adherence to diabetic eye examination sat at 46.1% before deployment, so more than half the eligible stream went unserved annually — with independent real-world measurement adding re-work on top (26.1% of patients not analyzable, 10.5% yielding no image at all, each a re-capture and often a dilation). Capacity 1: uniquely in this catalogue the manual counterfactual was measured by a randomised trial rather than inferred. Under scripted specialist referral, 22% of patients completed an eye examination within six months against 100% in the autonomous arm, and follow-through after an abnormal result was 22% against 64%; at system scale, comparison sites moved -0.3 points over two years while deploying sites gained 7.4. The unassisted channel delivers roughly a fifth of the eligible stream: 1, not 0 and not the 2 default. Both benefit studies carry vendor-affiliated investigators, and the trial was investigational pediatric use below the cleared adult indication; both labels travel with the figures wherever they appear.
- baseline
BASELINES. The three pathways at 3 are the three the authorization actually created: the camera feed into the classifier (its whole input, on every screen), the result issued to the ordering provider with no interpretive intermediary, and the decision written into the care record with no human co-author. Their width is the authorized design, not a governance lapse, and reading them as a lapse would misstate the case. The configuration read is 2 — read on every decision but a fixed artifact that moved twice in eight years. The eligibility-and-capture pathway is 2 on the labeled exclusion list plus the measured examiner variability. The quality gate is 2: a real, exercised, labeled refusal channel, held below 3 because it screens image quality and never the classification. Everything on the referral leg is 1, because every published measurement of it is a minority — 24% of suitable screens positive, 64% follow-through in a supported trial arm, 22% under referral alone. The surveillance reads are 1 against a documented history of two version reviews and no filed reports. The docket linkage is 0, and the record is what puts it there: direct queries of the public device databases on 28 August 2026 returned no adverse-event report and no recall for this device or its firm.
- baseline
THE ONE LATENT PATHWAY IS AN ABSENCE OF LINKAGE, NOT AN ABSENCE OF SAFETY. Direct queries of the public federal device databases on 28 August 2026 returned no adverse-event reports for this device under either product name, no recalls for the firm or the product code, and exactly one report across the entire retinal-diagnostic-software class, for a competitor device; no litigation or enforcement action against the manufacturer or the device was located. This network states that as what it is — an empty reported-event docket after eight years — and never converts it into a measurement of clinical safety, because the deployment's characteristic failure is one the reporting pathway was not shaped to receive: a case missed at screening surfaces, if ever, a year later in an eye clinic, unlinked to the screen it passed through. That is why the care-record-to-docket linkage is the board's one latent pathway. It is the honest residual in an otherwise favourable record, and it is drawn rather than narrated.
- baseline
THE REFUSAL CHANNEL IS BOTH THE SAFETY FEATURE AND THE EQUITY CONSTRAINT, and the diagram carries both readings without choosing. The gate declines to classify rather than guessing, which is what makes an autonomous design defensible, and the labeling gives its refusal a clinical meaning (re-capture, dilate, and treat a persistent no-result as possible vision-threatening disease). The same channel is where trial performance and field performance part company: 96.1% imageability under trial conditions against 26.1% of patients not analyzable and 10.5% producing no image at all in the one fully vendor-independent evaluation, with a five-system US programme reporting 28% of screenings not suitable for diagnosis. The measured drivers are capture-side and demographic — smaller pupils (2.65mm against 4.28mm), older age (63.9 years against 46.4), cataract, examiner variability — so the patients the gate most often refuses are the older, cataract-prone patients a screening programme most needs to reach. That inversion is drawn as the gate's real strength beside thin operator-side pathways, and stated here rather than folded into a number.
- assumed
EVIDENCE TIERING IS ASYMMETRIC HERE AND THE ASYMMETRY IS ON THE FAVOURABLE SIDE. The pivotal trial was funded by the manufacturer with company-affiliated authors; the randomised pediatric trial and the system-scale equity evaluation carry investigator ties to the vendor; the payment-milestone sources are a vendor's and a competitor vendor's announcements; the five-system adoption programme is an education-journal, industry-supported evaluation used for texture and never for a headline. The single fully vendor-independent evaluation in the record is also the least flattering on operability. Every parameter on this board that rests on a sponsor-labeled figure carries that label in its anchor, and where the two classes of evidence disagree — imageability above all — the independent measurement is the one the topology is drawn from. Two further discipline notes: the FDA's own announcement states 87.4% and 89.5% for the same pivotal trial as the paper's 87.2% and 90.7% (an analysis-population difference), and the two sets are never mixed; and the earlier Dutch study evaluated the pre-authorization European version, so it is cited for what it shows about reference-standard dependence and never as this device's US performance.
- assumed
WHERE THE RECORD IS SILENT, THE CONSERVATIVE VALUE. No source publishes an override or symptomatic-referral rate for the ordering providers, a per-screen defect rate in the care record, a rate at which operators misapply the labeled exclusion list, or any audit of this deployment's negative results against a specialist dilated examination in the US programme. Those magnitudes are estimated within the qualitative rungs the documented figures support, matching the PAN org's own estimated-on-every-value discipline. The one place a check's per-item depth and its reach diverge sharply — the specialist examination, thorough per patient and reaching a documented minority — is drawn at the reach and explained in the copy, never averaged into a middling number.
- assumed
SERVED PATIENTS ARE NOT IN THE DYNAMICS. People screened in a primary care clinic, referred to an eye service, or booked for a rescreen in twelve months are a boundary population recorded in the case file. No node, edge, baseline or lever here computes a clinical outcome for any patient, and no vision loss, diagnosis or progression is derived from this network. The deployment's measured equity findings — adherence rising 12.2 points among Black patients at deploying sites and the Asian-to-Black adherence gap narrowing from 15.6 to 3.5 points, in a study with vendor-affiliated investigators — are recorded external observations about served populations, carried in the case file and never computed here. So is their counterweight: the refusal channel's demographic skew toward older, cataract-prone patients, measured in an independent evaluation and likewise never derived from the diagram.
What this example does not show
- Patients are not modeled. No vision loss, no diagnosis, no disease progression and no clinical outcome for any person is computed anywhere on this diagram; the network propagates record and workflow error through operators and stores. Whether a missed screen cost anyone their sight is a question this Lab never answers, and the deployment's own record does not answer it either — no audit of this programme's negative results against a specialist dilated examination has been published for the US deployment.
- The benefit evidence is real, replicated, and affiliated. The pivotal trial was funded by the manufacturer with company-affiliated authors; the randomised pediatric trial and the 30-site equity evaluation carry investigator ties to the vendor; the payment-milestone sources are vendor announcements; the five-system adoption programme is an industry-supported education-journal evaluation used for texture, not for headline figures. The one fully vendor-independent evaluation in the record is also the least flattering on operability, and where the two disagree this network is drawn from the independent measurement. The pediatric trial studied ages 8 to 21, below the device's cleared adult indication — investigational use, never described here as within the authorization.
- No litigation, no enforcement action and no recall appears in this record. Direct queries of the federal device databases on 28 August 2026 returned no adverse-event reports for this device under either product name and no recalls for the firm or product code, against exactly one report across the whole retinal-diagnostic-software class. That is an empty reported-event docket, not a measurement of clinical safety: a screening false negative waiting out a twelve-month rescreen would rarely generate a device report at all. The manufacturer's stated assumption of liability for the diagnostic output is documented in academic legal literature and company statements, is reported as contractually scoped to the output rather than downstream care, and has never been tested in court.
- Two sets of pivotal figures circulate for the same trial — the paper's 87.2% sensitivity and 90.7% specificity, and the regulator's announcement's 87.4% and 89.5%, an analysis-population difference. The paper's are used here and the two are never mixed. The earlier Dutch study cited on this board evaluated the pre-authorization European version of the software and is used for one thing only: it showed measured sensitivity moving from 68% to 91% on the same eyes depending on which human grading scheme served as the reference standard. It is never a statement about this device's US performance.
- The equity findings run in both directions and neither is derived from the diagram. Adherence rose 12.2 points among Black patients at deploying sites and the Asian-to-Black adherence gap narrowed from 15.6 to 3.5 points, in a study with vendor-affiliated investigators. In independent real-world use, the patients the image-quality gate most often refused were the older and more cataract-prone — the same people a screening programme most needs to reach. Both are recorded external observations about served populations, carried in the case file, and this network computes neither.
Sources and evidence
What this example rests on, claim by claim. Every entry resolves to the same ledger the Evidence Registry publishes.
On April 11, 2018 the US Food and Drug Administration granted De Novo request DEN180001, received January 12, 2018 under Breakthrough Device review, permitting IDx LLC of Coralville, Iowa to market IDx-DR — in the agency's own words the first device authorized for marketing that provides a screening decision without the need for a clinician to also interpret the image or results, and the first autonomous diagnostic authorized in any field of medicine. The software is locked and deterministic, paired by the authorization to one nonmydriatic fundus camera model, and returns exactly one of two messages: more than mild diabetic retinopathy detected, refer to an eye care professional; or negative for more than mild diabetic retinopathy, rescreen in 12 months. In the pivotal trial (Abramoff et al., npj Digital Medicine 2018 — funded by the manufacturer, with company-affiliated authors) 900 adults with diabetes were enrolled at 10 primary care sites and 819 were fully analyzable, yielding 87.2% sensitivity (95% CI 81.8-91.2), 90.7% specificity (95% CI 88.3-92.7) and 96.1% imageability against a reading centre's widefield stereo photography and macular imaging, exceeding pre-specified endpoints of 85% and 82.5%; the FDA's own announcement states 87.4% and 89.5% for the same trial, an analysis-population difference. Camera operators were existing clinic staff who attested they had never performed ocular imaging, after a single four-hour standardized training.
empirical- Academic Abramoff, M. D., Lavin, P. T., Birch, M., Shah, N., & Folk, J. C. (2018). Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. npj Digital Medicine, 1, 39 https://pmc.ncbi.nlm.nih.gov/articles/PMC6550188/
- Government U.S. Food and Drug Administration (2018, April 11). FDA permits marketing of artificial intelligence-based device to detect certain diabetes-related eye problems (press announcement) https://www.fda.gov/news-events/press-announcements/fda-permits-marketing-artificial-intelligence-based-device-detect-certain-diabetes-related-eye
- Government U.S. FDA openFDA device databases: De Novo DEN180001 record, PIB clearance list, MAUDE and recall queries (api.fda.gov, queried 2026-08-28) https://api.fda.gov/device/510k.json?search=k_number:DEN180001&limit=1
The autonomy is fenced by the labeling rather than by a reviewer. The device is indicated for adults 22 and over with diagnosed diabetes who have not previously been diagnosed with diabetic retinopathy, used with the paired camera; patients are not to be screened if pregnant — retinopathy can progress rapidly in pregnancy — or if they report persistent vision loss, blurred vision, or floaters, or have previously been diagnosed with macular edema, severe non-proliferative, proliferative, or radiation retinopathy or retinal vein occlusion, or have had retinal laser treatment, intraocular injections, or retinal surgery. It detects diabetic retinopathy and no other condition, exposes no severity gradations, and issues no confidence figure. That eligibility and exclusion screen is administered by clinic staff before any image is taken, which places the surviving human judgment in this workflow before the camera rather than after the result. The escalation runs the same way: when image quality is insufficient the operator re-images, with pharmacologic dilation as the labeled next step — 23.6% of pivotal-trial participants required it — and current labeling warns that a patient who still yields no result after dilation may have vision-threatening diabetic retinopathy.
empirical- Vendor Digital Diagnostics. LumineticsCore Indications for Use, US (current labeling summary) https://www.digitaldiagnostics.com/products/eye-disease/indications-for-use-us/
- Government U.S. Food and Drug Administration (2018, April 11). FDA permits marketing of artificial intelligence-based device to detect certain diabetes-related eye problems (press announcement) https://www.fda.gov/news-events/press-announcements/fda-permits-marketing-artificial-intelligence-based-device-detect-certain-diabetes-related-eye
- Academic Abramoff, M. D., Lavin, P. T., Birch, M., Shah, N., & Folk, J. C. (2018). Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. npj Digital Medicine, 1, 39 https://pmc.ncbi.nlm.nih.gov/articles/PMC6550188/
The measured benefit of this deployment is replicated, and every study of it carries a sponsorship or investigator-affiliation label. In the ACCESS randomised trial at two Johns Hopkins pediatric diabetes clinics (Wolf et al., Nature Communications 2024; investigator ties to the vendor disclosed; ages 8 to 21, investigational use below the device's cleared adult indication) 164 youth were randomised, and eye-exam completion within six months was 100% (81 of 81) in the autonomous-AI arm against 22% under scripted specialist referral, with follow-through after an abnormal result 64% against 22%, in a cohort that was 35% Black and 47% Medicaid-insured, and no adverse events. At system scale (Huang, Channa, Wolf et al., npj Digital Medicine 2024; same affiliation caveat; 30-plus primary care sites, roughly 17,600 diabetes patients a year) sites that deployed the system raised diabetic-eye-exam adherence from 46.1% to 54.5% between 2019 and 2021 while comparison sites moved -0.3 points, with Black patients gaining 12.2 points and the Asian-to-Black adherence gap narrowing from 15.6 to 3.5 points. A five-health-system programme covering roughly 151,000 diabetes patients (Journal of CME 2025, an industry-supported education-journal evaluation used for texture rather than headline figures) recorded 20,160 screenings, 72% suitable for diagnosis, 24% of suitable screens positive, ophthalmology referrals up 23%, and anti-VEGF treatment up 27%, with one system reporting a 118% rise in screening rate, against barriers of physician hesitancy and workflow fragmentation. What scaled the deployment was payment rather than the authorization: the AMA CPT Editorial Panel created code 92229 for point-of-care automated retinal analysis effective January 1, 2021, and CMS's CY2022 Physician Fee Schedule final rule of November 2, 2021 set the first national Medicare payment for it — approximately 45 to 47 dollars nationally at launch by crosswalk to CPT 92325, effective January 1, 2022, with later-year rates drifting down.
empirical- Academic Wolf, R. M., et al. (2024). Autonomous artificial intelligence increases screening and follow-up for diabetic retinopathy in youth: the ACCESS randomized control trial. Nature Communications, 15, 421 https://pmc.ncbi.nlm.nih.gov/articles/PMC10784572/
- Academic Huang, J., Channa, R., Wolf, R. M., et al. (2024). Autonomous artificial intelligence for diabetic eye disease increases access and health equity in underserved populations. npj Digital Medicine https://www.nature.com/articles/s41746-024-01197-3
- Academic Journal of CME (2025). Revolutionizing Diabetic Retinopathy Screening: Integrating AI-Based Retinal Imaging in Primary Care, 14(1), 2437294 https://pmc.ncbi.nlm.nih.gov/articles/PMC11703125/
- Vendor Digital Diagnostics (2021). Digital Diagnostics Celebrates CMS's Historic Decision to Finalize a National Rate for CPT 92229; and Eyenuk (2021, via GlobeNewswire/Yahoo Finance). Eyenuk Applauds CMS CY 2022 Medicare Physician Fee Schedule Final Rule https://www.digitaldiagnostics.com/resources/insights/digital-diagnostics-celebrates-cmss-historic-decision-to-finalize-a-national-rate-for-cpt-92229/
The one place trial performance and field performance part company is the image-quality gate rather than the classifier. In the only fully vendor-independent evaluation located (Hunfeld et al., Scientific Reports 2026; Karlsburg Diabetes Hospital, Germany; 875 patients, February 2020 to November 2021) 26.1% of patients' images could not be analyzed by the device and 10.5% of patients yielded no image at all, against 96.1% imageability in the sponsor-funded pivotal trial; the measured drivers were capture-side and demographic — mean pupil diameter 2.65mm against 4.28mm, mean age 63.9 years against 46.4, impaired retinal view with cataract documented in 61% of non-analyzable right eyes, and variability between examiners. Among images the device could analyze it held up, at 94.4% sensitivity and 90.5% specificity for severe disease, with exact grade agreement against an ophthalmologist at 54.2%. A five-health-system US programme found 28% of screenings not suitable for diagnosis, corroborating the direction. An earlier external evaluation of the pre-authorization European version of the software (van der Heijden et al., Acta Ophthalmologica 2018; Hoorn Diabetes Care System, Netherlands; 1,415 patients with type 2 diabetes, 898 of sufficient image quality; the founder a co-author) adds a separate caution about measurement itself: sensitivity for referable disease was 68% under one human grading scheme and 91% under another on the same eyes, with the human reference graders agreeing among themselves only 40% to 61% of the time.
empirical- Academic Hunfeld, M., et al. (2026). Real-world performance of the AI diagnostic system IDx-DR in the diagnosis of diabetic retinopathy and its main confounders. Scientific Reports, 16, 4349 https://www.nature.com/articles/s41598-026-36970-9
- Academic Abramoff, M. D., Lavin, P. T., Birch, M., Shah, N., & Folk, J. C. (2018). Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. npj Digital Medicine, 1, 39 https://pmc.ncbi.nlm.nih.gov/articles/PMC6550188/
- Academic van der Heijden, A. A., et al. (2018). Validation of automated screening for referable diabetic retinopathy with the IDx-DR device in the Hoorn Diabetes Care System. Acta Ophthalmologica, 96(1), 63-68 https://onlinelibrary.wiley.com/doi/10.1111/aos.13613
- Academic Journal of CME (2025). Revolutionizing Diabetic Retinopathy Screening: Integrating AI-Based Retinal Imaging in Primary Care, 14(1), 2437294 https://pmc.ncbi.nlm.nih.gov/articles/PMC11703125/
- Vendor Digital Diagnostics. LumineticsCore Indications for Use, US (current labeling summary) https://www.digitaldiagnostics.com/products/eye-disease/indications-for-use-us/
The compensating controls that replaced the physician overread are regulatory and contractual, and their record is thin in the way an empty docket is thin. Direct queries of the FDA's public device databases on August 28, 2026 return no adverse-event reports for this device under either product name, no recalls for the product code or for the firm, and exactly one adverse-event report across the entire retinal-diagnostic-software class, for a competitor device; no product-liability or other litigation over the system was located. That is an empty reported-event docket after eight years, not a measurement of clinical safety: a screening false negative that waits out its 12-month rescreen interval would rarely generate a device adverse-event report at all. The De Novo created a durable device class — retinal diagnostic software, 21 CFR 886.1100, product code PIB, with special controls — through which seven follow-on clearances across four firms have since passed, including this device's own version 2.3 in 2021 and 2022, the documented change-control channel for a locked algorithm. IDx renamed itself Digital Diagnostics Inc. on August 19, 2020 alongside an acquisition, and the product was later renamed LumineticsCore. Academic legal literature records that the manufacturer carries medical malpractice liability insurance for the system and assumes liability for injuries arising from it; trade reporting scopes that assumption contractually to the system's diagnostic output rather than downstream care management, and no case has tested it.
empirical- Government U.S. FDA openFDA device databases: De Novo DEN180001 record, PIB clearance list, MAUDE and recall queries (api.fda.gov, queried 2026-08-28) https://api.fda.gov/device/510k.json?search=k_number:DEN180001&limit=1
- Academic Research Handbook on Health, AI and the Law, chapter 9: Liability for use of artificial intelligence in medicine (NCBI Bookshelf NBK613216, 2024) https://www.ncbi.nlm.nih.gov/books/NBK613216/
- Vendor Digital Diagnostics via PR Newswire (2020, August 19). Digital Diagnostics, formerly IDx, Expands Global Impact of Healthcare Autonomous AI with Acquisition of 3Derm Systems, Inc. https://www.prnewswire.com/news-releases/digital-diagnostics-formerly-idx-expands-global-impact-of-healthcare-autonomous-ai-with-acquisition-of-3derm-systems-inc-301115054.html
Where this connects
Institutional pressures in this domain
- Workload surge — Demand outruns staffing; per-case attention shrinks and review becomes triage.
- Reviewer bottleneck — One fixed-capacity checking stage sits between AI output and consequence; everything queues behind it.
- Vendor opacity — The deploying institution cannot inspect the model, data, or update pipeline it is accountable for.
- Data & policy drift — The world, the intake process, and the rules change under a system trained on how things used to be — two mechanisms with different remedies: the statistical properties of what the system processes move (concept drift), or the mixture of inputs arriving in deployment differs from the mixture it was trained on (covariate shift).
- Deadline pressure — Statutory or managerial timeliness rules reward fast approval of machine output over slow disagreement.
All of them in context on the Clinical decision support & deterioration alerting domain page.
Levers available here and the patterns behind them
- Understand the system — Understand the system
- Gate vendor updates — Vendor quality gate
- Upgrade model — Improve the model
- Review on schedule — Oversight cadence & retrospectives
- Review the riskiest first — Risk-tiered oversight
- Escalate checks — State-feedback vigilance
Documented case histories
- IDx-DR Autonomous Screening
- TREWS sepsis early-warning system
- Advance Alert Monitor (AAM) deterioration model
- Sepsis Watch deep-learning detection system
- Proprietary EHR sepsis model (external validation)
- nH Predict Utilization Review
- Cost-Proxy Care Stratification
- CA-CDS Child Abuse Alerting
- Viz.ai LVO Stroke Triage
- IBM Watson for Oncology
- OPTN eGFR Waiting-Time Correction
- Practice Fusion Pain CDS
- UBH Level of Care Guidelines (Wit v. UBH)
- EviCore by Evernorth: the review threshold
- Cigna PxDx