What it is
No organization can check every automated answer with the same care. Risk tiering sorts cases by what is at stake for the person, such as a child's safety, a family's benefits, or a patient's care. The highest-stakes cases go to the most careful review. In the documented cases cited below, human review cut error and disparity roughly in half. It worked because reviewers knew things about the case that the model did not.
What it pushes on in the Lab
In the Lab, this lever raises the checking and correcting that the people using the system can do. It does so by focusing their review on the cases with the most at stake.
You can also pull this lever at a strong tier, which costs more. At the strong tier, each effect below the Lab's strongest setting pushes harder.
In the modes that offer aiming, you can aim this lever at particular parts and pathways of a network. Otherwise it applies to the whole network.
This lever pushes less hard when you switch on lingering effects in the Lab's options, unless you also pull Understand the system.
Its pattern in the Practice Library
The Practice Library describes the pattern behind this lever:Risk-tiered oversight
The pressures it answers
These pressures list this lever among the levers that answer them:
A lever answers a pressure when it pushes the other way on something the pressure pushes on.
Where you can pull it
Networks in the Lab that offer this lever:73
- A commercial code assistant across three enterprises
- A heavy-industry predictive-maintenance deployment
- Accelerated Safety Analysis Protocol (ASAP Tool)
- Advance Alert Monitor (AAM) deterioration model
- Allegheny Family Screening Tool
- Allegheny Hello Baby
- Allegheny Housing Assessment
- Amazon Flex driver standing and deactivation
Every network that offers it
- Amazon fulfillment-centre algorithmic management
- Ambient scribe RCT + monitoring playbook
- An ambient AI scribe at a multi-specialty health system
- Arkansas ARChoices / ARIA
- Audi press-shop inspection
- Automated visual inspection of injectable drugs
- BAMF dialect recognition
- Character.AI crisis-safety stack
- Chicago Public Schools On-Track indicator
- Colorado Family Safety and Risk Assessments
- Community Notes on X, formerly Birdwatch on Twitter
- Cost-Proxy Care Stratification
- Credit Acceptance's net-collections Score, inside CAPS
- CrimSAFE criminal-record tenant screening
- Danske Bank fraud scoring
- Douglas County Decision Aide
- Eckerd Rapid Safety Feedback
- Epic Sepsis Model
- Equifax's Online Model Server
- EviCore by Evernorth prior-authorization screening
- Fraud false positives that froze real accounts
- Google ML code completion
- Google's child-safety detection and account enforcement
- Homebase Risk Assessment Questionnaire
- IDx-DR Autonomous Screening
- Illinois Rapid Safety Feedback
- Kaiser Permanente ambient AI scribe
- Kaiser Permanente Suicide-Risk Model
- LA County Homelessness Prevention Unit
- Los Angeles County Project AURA
- Massachusetts DTA call summaries
- Meta content enforcement
- Meta's cross-check secondary review programme
- ML anti-money-laundering as primary monitoring
- MyFriendBen benefits screener
- NarxCare
- New Zealand MSD Predictive Risk Modelling
- Oportun Financial Corporation's legal-collections pipeline
- Oregon Safety at Screening
- Predictive maintenance on a high-speed rail fleet
- REACH VET
- Rotterdam welfare-fraud risk model
- SafeRent Tenant Screening Score
- Santander Consumer USA's loss forecasting score
- Sepsis Watch deep-learning detection system
- Sistema Alerta Niñez (Chile)
- Stratification Tool for Opioid Risk Mitigation
- The NCMEC CyberTipline reporting and triage system
- The Sama Nairobi content-moderation workforce for Meta
- The same AI with a human checking: the supervised office
- TikTok's EU and UK content-moderation operation
- TREWS sepsis early-warning system
- Udbetaling Danmark data-driven control (Denmark)
- Vanderbilt VSAIL suicide-risk alert
- VI-SPDAT
- Viz.ai LVO Stroke Triage
- Wells Fargo refinance underwriting (CORE/ECS)
- What Works for Children's Social Care ML pilots
- Wikipedia's edit-scoring service (ORES, now Lift Wing)
- Wisconsin DEWS
- Woebot (a governed app wind-down)
- X Multilingual Hate-Speech Enforcement
- Xantura OneView (predictive homelessness flagging)
- YouTube Covid-19 enforcement
- YouTube's Content ID copyright matching system
The evidence behind its effects
The Lab cites these claims from the evidence registry for this lever's effects.
In the documented AFST evaluation, screener overrides of the tool — roughly a third of its recommendations — cut screen-in disparity from about 20% to 9% relative to the tool acting alone.[4]
Rittenhouse, Algorithms, Humans and Racial Disparities in Child Protective Services https://krittenh.github.io/katherine-rittenhouse.com/Rittenhouse_Algorithms.pdf
https://krittenh.github.io/katherine-rittenhouse.com/Rittenhouse_Algorithms.pdf
Grounds: model org: allegheny_afst
Goldhaber-Fiebert & Prince (Stanford), Impact evaluation summary: Allegheny Family Screening Tool (Allegheny County DHS, April 2019) https://analytics.alleghenycounty.us/wp-content/uploads/2019/05/Impact-Evaluation-Summary-from-16-ACDHS-26_PredictiveRisk_Package_050119_FINAL-5.pdf
Appears in: PAN framework development
Grounds: capability governance: at-node control; model org: allegheny_afst
Topics: child-welfare
Centre for Social Data Analytics (AUT), AFST evaluation summary https://csda.aut.ac.nz/news-and-events/2019/allegheny-family-screening-tool-evaluation-improved-decision-accuracy,-reduced-disparities
Grounds: model org: allegheny_afst
Stapleton, L., Lee, M. H., Qing, D., Wright, M., Chouldechova, A., Holstein, K., Wu, Z. S., & Zhu, H. (2022). Imagining new futures beyond predictive systems in child welfare: A qualitative study with impacted stakeholders. 2022 ACM Conference on Fairness Accountability and Transparency, 1162–1177. https://doi.org/10.1145/3531146.3533177
doi.org/10.1145/3531146.3533177
Appears in: PAN framework development; Paramerge authored research
Grounds: deployment audit: Allegheny AFST
Topics: algorithmic-fairness, child-welfare
In the documented MiDAS case, error among no-review auto-adjudications ran roughly 93%, and determinations erred at about 85% without human review versus 44% with it.[4]
Michigan AG, settlement of civil-rights class action (Bauserman, 2022) https://www.michigan.gov/ag/news/press-releases/2022/10/20/som-settlement-of-civil-rights-class-action-alleging-false-accusations-of-unemployment-fraud
Grounds: model org: michigan_midas
IEEE Spectrum, Michigan's MiDAS unemployment system: Algorithm alchemy that created lead, not gold https://spectrum.ieee.org/michigans-midas-unemployment-system-algorithm-alchemy-that-created-lead-not-gold
Appears in: PAN framework development
Grounds: deployment audit: Michigan MiDAS
AI Incident Database, Incident 373 (MiDAS false fraud claims) https://incidentdatabase.ai/cite/373/
https://incidentdatabase.ai/cite/373/
Grounds: model org: michigan_midas
Benefits Tech Advocacy Hub, Michigan UI False Fraud Determinations https://www.btah.org/case-study/michigan-unemployment-insurance-false-fraud-determinations.html
https://www.btah.org/case-study/michigan-unemployment-insurance-false-fraud-determinations.html
Grounds: model org: michigan_midas
In contextual inquiries with Allegheny AFST call screeners, workers calibrated reliance using contextual case knowledge unavailable to the model and reliably detected and overrode erroneous risk scores — complementary human information, not generic distrust, was the safeguard's mechanism.[2]
Kawakami, A., Sivaraman, V., Cheng, H.-F., Stapleton, L., Cheng, Y., Qing, D., Perer, A., Wu, Z. S., Zhu, H., & Holstein, K. (2022). Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support. In CHI Conference on Human Factors in Computing Systems (CHI '22). ACM. https://doi.org/10.1145/3491102.3517439
doi.org/10.1145/3491102.3517439
Appears in: PAN framework development
Topics: algorithmic-fairness, child-welfare, human-ai-interaction
De-Arteaga, M., Fogliato, R., & Chouldechova, A. (2020). A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores. In CHI Conference on Human Factors in Computing Systems (CHI 2020). ACM. https://doi.org/10.1145/3313831.3376638
doi.org/10.1145/3313831.3376638
Appears in: Evidence reverification (2026)
Topics: algorithmic-fairness, child-welfare, human-ai-interaction