Every example in the PAN Lab

The same AI running hands off: the agentic office

The AI here doesn't just draft casework — it acts on cases, and a stretched staff waves most of it through. Same model as the other two offices. In the sociotechnical simulation — a modeled office, not a real one —…

1 cited claim · 13 levers · Details · Open in the Lab

The same AI with a human checking: the supervised office

Same AI — but a human checks every output before it reaches the record. In the sociotechnical simulation — a modeled office, not a real one — that holds errors to about 20%, a quarter of the hands-off rate. The failure…

1 cited claim · 12 levers · Details · Open in the Lab

The same AI under full guardrails: the professional office

Same AI, full guardrails: verification time is budgeted, writes are gated, records carry provenance labels, and someone with authority reviews the deployment on a schedule. In the sociotechnical simulation — a modeled…

1 cited claim · 12 levers · Details · Open in the Lab

REACH VET

The flag that works and the outcome it misses: a suicide-risk model

A national model flags the highest-risk 0.1% of patients each month; a coordinator routes each flag to a clinician, who re-evaluates and reaches out. Modeled on REACH VET. The human loop here actually works - so watch…

2 cited claims · 12 levers · Details · Open in the Lab

Vanderbilt VSAIL suicide-risk alert

The alert that had to be dismissed: an EHR suicide-risk model

The same suicide-risk score, the same threshold, the same patients — and the only thing that changes is whether the alert interrupts the clinician or waits quietly in the chart. In the record that shape came from, that…

2 cited claims · 11 levers · Details · Open in the Lab

Kaiser Permanente Suicide-Risk Model

The added sensor: an EHR-embedded suicide-risk score

A machine-learning score flags a patient's suicide risk within about 30 minutes of a virtual mental-health intake, and either the score or the existing self-report screen routes that patient into the same…

2 cited claims · 9 levers · Details · Open in the Lab

Crisis Text Line & Loris.ai

The corpus and the spinoff: governing crisis-conversation data

A crisis line's in-house model reorders which texter a counselor sees first, and the human loop around it genuinely works. The exposure is somewhere else: the same conversations become a corpus, and that corpus was…

2 cited claims · 11 levers · Details · Open in the Lab

NarxCare

The score you cannot see: an opaque prescribing-risk model

A proprietary model reads a patient's prescription-monitoring record and returns a secret 000-999 risk score that shows up in the patient header next to vitals and allergies — and can decide whether they get pain…

2 cited claims · 13 levers · Details · Open in the Lab

Limbic Access (NHS Talking Therapies)

The front door that works: a self-referral triage chatbot

A self-referral chatbot handles the front door to talking therapies: it runs intake, stratifies risk, and hands a clinician a completed record to assess. For once the access story is real — more people referred…

2 cited claims · 9 levers · Details · Open in the Lab

Woebot (a governed app wind-down)

The responsible wind-down: retiring a peer-reviewed CBT chatbot

A rule-based CBT chatbot ran for years as a self-help app, used by roughly 1.5 million people over its lifetime, and its maker chose to retire it — deliberately, on a published schedule, with a window to download your…

2 cited claims · 10 levers · Details · Open in the Lab

LyssnCrisis counselor QA at ProtoCall Services (988)

The AI watches the counselor rather than the caller: a crisis-line QA scorer

For once, the AI is pointed the safe way round. It never answers a caller in crisis. It listens to the counselor's own call and scores their practice — did they check for suicide risk, did they stay present — then hands…

2 cited claims · 10 levers · Details · Open in the Lab

Gaggle Safety Management

The discontinuation that changed the vendor rather than the graph: a school monitoring scanner

A vendor scans everything students write on school accounts - email, homework, art, the school paper - and flags anything that might signal self-harm, 24/7. Modeled on Gaggle's school safety monitoring. A machine flags;…

2 cited claims · 11 levers · Details · Open in the Lab

Oxevision camera monitoring on NHS mental health wards

The evaluation was written by the seller: a bedroom monitor no one independent checked

A camera on the wall of a psychiatric bedroom measures your pulse and your breathing without touching you, and flags when you leave the room or stay too long in the bathroom. It is sold as safety. Modeled on Oxevision…

2 cited claims · 10 levers · Details · Open in the Lab

ODMAP overdose spike alerts

Aggregate alert from a contested store: a nationwide overdose spike-detection network

This one does not score a person. Agencies enter suspected, unconfirmed overdose events into one shared national store, and a plain rule - a rolling 24-hour count against two standard deviations above the county's own…

2 cited claims · 11 levers · Details · Open in the Lab

NYC Teenspace

The surface everyone could audit, and the one nobody did

A city bought population access to a commercial teletherapy platform for every teenager who lives there: 26 million dollars over three years, no insurance, no referral, parental consent and a licensed therapist. Inside…

4 cited claims · 8 levers · Details · Open in the Lab

Character.AI crisis-safety stack

The screen with nobody behind it

A companion-chat platform serving roughly twenty million people, about a tenth of them under eighteen, built its entire crisis-safety layer while under external pressure, in four dated steps. A phrase screen raising a…

4 cited claims · 13 levers · Details · Open in the Lab

Tessa chatbot replacing the NEDA eating-disorder helpline

The warrant stayed put while the system moved

A national nonprofit replaced a twenty-year helpline, which had fielded nearly 70,000 contacts in a year, with a chatbot. Modeled on a deployment whose tool had two layers with different evidence behind them: a closed,…

3 cited claims · 12 levers · Details · Open in the Lab

Stratification Tool for Opioid Risk Mitigation

The health system that randomized its own oversight lever

A national health system scores every patient with an opioid prescription for one-year risk of an overdose-related or suicide-related event, refreshes the score nightly, and requires an interdisciplinary team at all 140…

3 cited claims · 10 levers · Details · Open in the Lab

Nava assistive benefits chatbot

Done carefully: a verify-before-use copilot

For once, a system that starts safe. This copilot answers caseworker questions only from a vetted document set and hands back direct quotes to check — and it never writes to the record. Modeled on Nava's chatbot. The…

11 levers · Details · Open in the Lab

GOV.UK Chat

The gate that said not yet: a public assistant behind a staged pilot gate

This assistant answers the public's tax, benefits and visa questions directly, in an official voice, drawing only on curated GOV.UK guidance — and the thing worth studying is not the answer but the gate in front of it.…

2 cited claims · 10 levers · Details · Open in the Lab

Burokratt

The network of networks: a federated public-service chatbot

One chat window, many bots behind it. Each Estonian public institution runs its own assistant with its own knowledge base, a central classifier routes a citizen's question to the right one and oversees the handover, and…

1 cited claim · 11 levers · Details · Open in the Lab

Caddy adviser copilot at Citizens Advice

The gate is a job rather than a habit: a supervisor-checked adviser copilot

For once, a system built to stay safe. This copilot never talks to the public: every draft it writes goes to a separate supervisor to approve, edit, or reject before it reaches even the adviser, and only then does a…

2 cited claims · 10 levers · Details · Open in the Lab

Albert France Services

Killed without a number: a sovereign adviser assistant that no metric ever measured

This assistant answered a France Services adviser's benefits question with a sourced draft answer from a curated set of official documents, for the adviser to verify, modify and validate before relaying to the citizen.…

2 cited claims · 10 levers · Details · Open in the Lab

Benefits Data Trust wind-down

The node that could not be kept: winding down a benefits-navigation nonprofit

For twenty years this nonprofit sat between low-income people and the agencies that run public benefits: it used agency data-sharing to find likely-eligible people, ran a call center where navigators screened callers…

2 cited claims · 10 levers · Details · Open in the Lab

Propel in-app SNAP benefits assistant

Read-only by design: a benefits assistant grounded on the record it never writes

For once the memory loop is cut at the source. This assistant reads a state-verified deposit record to tell a recipient a payment is missing, then steers them to fix it on the state's own system — and it never writes to…

2 cited claims · 9 levers · Details · Open in the Lab

GetCalFresh

Most of a state's online intake with no authority at all: an assisted-application node and its handoff

For six years, most of a state's online food-benefit applications went through a web form built by a nonprofit - not a government system, not a scoring model, not a chatbot that decides anything. Modeled on GetCalFresh.…

2 cited claims · 10 levers · Details · Open in the Lab

Frida (NAV Norway)

Ask for a human: the handover boundary as a governed surface

This chatbot is the anonymous front door to a national welfare agency: it answers pensions, child support, unemployment and sick-leave questions 24 hours a day, sees no personal information, and decides no case. Modeled…

2 cited claims · 11 levers · Details · Open in the Lab

IRS collection chatbots

Expanded without a ruler: a federal collection chatbot with no performance measures

A federal collection agency built a chatbot and a live-chat line to steer people off its phone queue, then expanded them and made them permanent - without ever measuring whether they worked. Modeled on the IRS Automated…

2 cited claims · 10 levers · Details · Open in the Lab

Singapore's chatbot fleet refresh

Eighty engines into one: a whole-of-government chatbot fleet refresh

For a decade, eighty-odd government websites each ran their own scripted answer bot - independent, and when one went wrong a single agency could pull its own instance while every other agency ran on. Then one decision…

2 cited claims · 10 levers · Details · Open in the Lab

SSA 800-Number Conversational AI Assistant

The access gate: a national benefits phone line

A national benefits phone line puts an automated layer in front of every caller. It matches what you say against a fixed set of 74 answers, and it decides whether you reach a person, when, and through which door.…

4 cited claims · 13 levers · Details · Open in the Lab

EDD Virtual Assistant

Two levels up: a benefits assistant and its budget

A state benefits agency runs a conversational assistant in two tiers. One is open to anyone, around the clock, in eight languages, and answers general questions about unemployment, disability and paid family leave. The…

4 cited claims · 13 levers · Details · Open in the Lab

Mass.gov Virtual Assistant

The pages it reads, rewritten so it reads them better

A state built its own assistant and put it in front of the safety net. It answers questions about food assistance, unemployment, child support and family leave from a knowledge base rebuilt every night out of the…

3 cited claims · 13 levers · Details · Open in the Lab

MyFriendBen benefits screener

One engine under six screeners: a shared benefits substrate

Six state benefit screeners, run by six different nonprofits answering to six different boards, all compute their eligibility math in one open-source codebase owned by a seventh organization. That is the whole board.…

4 cited claims · 8 levers · Details · Open in the Lab

Magic Notes (Beam)

The drafted record: a case-notes copilot

A copilot drafts case notes; under time pressure, practitioners sign them with barely a read, and yesterday's notes feed today's drafts. Modeled on Beam's Magic Notes. Nothing here "decides" anything — that's the trap.…

11 levers · Details · Open in the Lab

Guided walkthrough: agentic low-oversight office

A walkthrough-only version of the agentic low-oversight office: an AI assistant that also acts as an autonomous agent, a stretched staff supervising many automated actions at once, and — added for the tour — the Outside…

1 cited claim · 13 levers · Details · Open in the Lab

Minute / Local Transcribe

The governed thing and the measured thing: a state-built meeting scribe

A government-built scribe transcribes council meetings and drafts standardised summaries, and every council in the cohort runs the same shared instance under one pooled assurance record. Modeled on the Minute / Local…

1 cited claim · 10 levers · Details · Open in the Lab

GDS Microsoft 365 Copilot cross-government experiment

Time saved then spent checking: a whole-office productivity copilot

This copilot drafts documents, summarises meetings, and writes email for the whole office at once, and the largest published trial of it saved a self-reported 26 minutes a day and left most users unwilling to go back.…

2 cited claims · 10 levers · Details · Open in the Lab

DWP Whitemail Insights and Vulnerability Scanner

The letter no one reads twice: an upstream vulnerability scanner

This scanner reads every one of roughly 25,000 paper letters a day sent to the welfare agency and decides which claimants surface on the potentially-vulnerable shortlist. Modeled on the Department for Work and Pensions…

2 cited claims · 10 levers · Details · Open in the Lab

UK Home Office asylum AI copilots

Compressed before judgment: the summary no one is told to check

Two copilots sit in asylum casework: one compresses the interview transcript — the claimant's own account — into a summary the decision-maker reads, and one summarises the country policy at decision time. Modeled on the…

2 cited claims · 10 levers · Details · Open in the Lab

Justice Transcribe

The note that scores you: a copilot at the head of a risk pipeline

A copilot transcribes probation supervision sessions and drafts the summary that becomes the case record - and it decides nothing. But the same record is read, at more than a thousand assessments a day, by a separate…

1 cited claim · 12 levers · Details · Open in the Lab

Learned Hand AI clerk pilot (LA and Riverside courts)

The only reviewer is the last authority: an AI clerk at the adjudication node

The largest trial court in the United States gave a handful of judges an AI clerk that reads the filings, researches the law, and drafts the order — in the judge's own writing style. Modeled on the Learned Hand LA…

2 cited claims · 10 levers · Details · Open in the Lab

SSA Insight

The verifier and the sign flip: a decision-checking copilot

This copilot runs the other way. A human writes the disability decision first; the tool reads the draft and raises quality flags before it issues, and every fully favorable draft has to be run through it. Modeled on the…

1 cited claim · 11 levers · Details · Open in the Lab

VA claims automation (automated survivor-benefit decisions)

The record no one read: automated survivor-benefit decisions

Rules read scanned death certificates and applications, then write the grant, the payment, and the letter into the record with no human when the rules match - and a thin summary sheet the record keeps as its only…

2 cited claims · 11 levers · Details · Open in the Lab

Trelleborg's Welfare Robot

The robot that decides: automation and the vanishing human loop

This copilot is not a copilot - it is the caseworker. For recurring monthly social-assistance reapplications, a rules-based robot logs into the case system as if it were a person, cross-checks the registers, and issues…

1 cited claim · 12 levers · Details · Open in the Lab

Amsterdam Smart Check

The governed exit: a fair welfare screener that shipped every safeguard but one

This one did almost everything right. A city built a deliberately fair welfare-fraud screener: an explainable model with sensitive attributes and postal codes left out, a bias audit, debiasing that approximately…

1 cited claim · 12 levers · Details · Open in the Lab

Massachusetts DTA call summaries

The record and its source: AI summaries of benefits calls

A state benefits agency pilots a vendor-built copilot that transcribes eligibility calls in real time and writes a structured summary the caseworker can edit before saving it into the eligibility system of record. The…

5 cited claims · 8 levers · Details · Open in the Lab

Illinois DCFS Augintel

The record that reads back: a case-note mining overlay

Caseworkers write the notes. An overlay reads every note ever written and hands search results, case overviews and safety alerts back to them. The same extractions travel upward as practice-fidelity measures, compliance…

3 cited claims · 12 levers · Details · Open in the Lab

CDTFA Axyom Assist

The renewal decision: a call-center answer assistant

A state tax department buys a retrieval assistant for its call center. It listens to the live call, finds candidate answers in 16,000-plus pages of the department's own published material, and drafts the summary…

4 cited claims · 14 levers · Details · Open in the Lab

NJ AI Assistant

The library the staff wrote: a state-built drafting assistant

A state built its own drafting assistant instead of buying one. The interface, the hosting and the logs belong to the government; a commercial model service sits behind them, and the state swapped that engine once while…

3 cited claims · 13 levers · Details · Open in the Lab

Allegheny Family Screening Tool

The score and the screener: a child-welfare risk tool

A risk score built from old records lands in front of call screeners, who decide — except at the top of the scale, where the design makes the default call: above 17 with a child 16 or younger, screen-in is mandatory…

2 cited claims · 10 levers · Details · Open in the Lab

Illinois Rapid Safety Feedback

The alarm that cried wolf: a child-safety scorer

A vendor model scores maltreatment reports and points investigators at the highest-risk children. Modeled on Illinois's Rapid Safety Feedback. It failed both ways at once: thousands of kids flagged at extreme risk who…

2 cited claims · 13 levers · Details · Open in the Lab

Allegheny Hello Baby

The help that keeps a file: a birth-risk prevention model

Every newborn in the county is scored from old records and sorted into service tiers — the top tier gets a knock on the door offering help, not an investigation. Modeled on Allegheny's Hello Baby. Two things make it…

1 cited claim · 10 levers · Details · Open in the Lab

Douglas County Decision Aide

The score read only at the edges: a child-welfare screening aide

A 1-to-20 risk score, built from years of administrative records and deliberately blind to race, lands in front of a consensus screening team that keeps full discretion. Modeled on Douglas County's Decision Aide. An…

1 cited claim · 10 levers · Details · Open in the Lab

Eckerd Rapid Safety Feedback

Endorsed then evaluated: a risk tool that spread on a claim

A proprietary profile flags open cases that look like past tragedies, and quality-assurance reviewers coach frontline workers on the ones it surfaces. Modeled on Eckerd's Rapid Safety Feedback and its…

2 cited claims · 11 levers · Details · Open in the Lab

ProKid (Netherlands)

The colour and the record: a child risk-profiler

A rule-based instrument sorts children under 12 into four colour risk bands from up to twelve years of police records — including children logged only as victims or witnesses — and police controllers decide who to refer…

1 cited claim · 10 levers · Details · Open in the Lab

Insight Bristol / Think Family Database

The database nobody could audit: a shared child-risk profiling system

Several risk models score children from one shared cross-agency database that every agency reads and writes — and front-line staff, distrusting the scores, often did not act on them. Modeled on Bristol's Insight Bristol…

2 cited claims · 12 levers · Details · Open in the Lab

Sistema Alerta Niñez (Chile)

Scored before anyone knocks: a child-risk targeting tool

A model scores every child in the system for future risk and hands OLN teams a ranked list of whom to reach out to first — built from data families gave to receive benefits. Modeled on Chile's Sistema Alerta Niñez.…

1 cited claim · 11 levers · Details · Open in the Lab

Los Angeles County Project AURA

Caught at the gate: a child-abuse risk model that never shipped

A proprietary model scores every abuse-and-neglect referral 1 to 1,000 from cross-agency records — but it is tested against past cases before anyone wires it to a live investigation. Modeled on Los Angeles County's…

1 cited claim · 12 levers · Details · Open in the Lab

What Works for Children's Social Care ML pilots

The bar it never cleared: a child-welfare prediction pilot

A government-funded evidence centre built prediction models to forecast whether a child's case would escalate, set a public success bar before it started, and scored every model against it. Modeled on England's What…

1 cited claim · 11 levers · Details · Open in the Lab

New Zealand MSD Predictive Risk Modelling

Halted before it ran: a national child-risk model

A predictive risk model would have scored every child's likelihood of a substantiated maltreatment finding by age five, computed from linked benefit and child-protection records, and handed the number to frontline…

1 cited claim · 11 levers · Details · Open in the Lab

Gladsaxe model

The screen that never ran: whole-population child scoring

A municipality builds an in-house tool to score, for every young child rather than only families already getting help, the probability that the child is living in vulnerability. Modeled on Denmark's Gladsaxe model. The…

2 cited claims · 10 levers · Details · Open in the Lab

Hackney / Xantura Early Help Profiling

The pilot that quietly failed: a small council's family profiler

A small council buys a vendor tool that mines data across its own services to flag families for early help, and sends social workers a monthly list of those judged most at risk — while the families themselves are never…

3 cited claims · 11 levers · Details · Open in the Lab

Oregon Safety at Screening

The fix then the off switch: a fairness-corrected screening tool

A fairness-corrected risk score lands in front of state hotline screeners, who decide whether to investigate — and an internal review later chose to switch the whole tool off. Modeled on Oregon's Safety at Screening…

1 cited claim · 12 levers · Details · Open in the Lab

Accelerated Safety Analysis Protocol (ASAP Tool)

The score nobody sees: choosing who gets a second look in child protection

A city child-protection agency scores every open investigation at day 10 with a model it built itself, and ranks them. The top of that ranking fills a quality-assurance review list of about 3,000 cases a year, against…

4 cited claims · 12 levers · Details · Open in the Lab

Colorado Family Safety and Risk Assessments

Two instruments, one decision, and a review that reached the legislature

Every screened-in child welfare referral in one state runs through two instruments at once: a structured safety determination a worker builds from field observation, and an actuarial risk scale computed largely from the…

4 cited claims · 12 levers · Details · Open in the Lab

Family-Match (Adoption-Share)

The ledger that decided whether the ledger was working

Two populations, one score. Adults who want to adopt register themselves and fill in a compatibility survey. Caseworkers and foster parents enter each waiting child's case data. A proprietary engine scores every family…

5 cited claims · 13 levers · Details · Open in the Lab

US Birth Match

Birth Match: a two-registry join that is itself the referral

Five US states join two registries and call the result a referral. New birth registrations meet a roster of parents with prior terminations of parental rights, serious-harm findings, or certain convictions, and a hit is…

3 cited claims · 12 levers · Details · Open in the Lab

CORA, the DC CFSA policy assistant

The guardrail that named one direction

A child-welfare agency built a staff-facing policy assistant into its new case-management system and did the governance homework first: a sixteen-page alignment report published on the city technology register before…

6 cited claims · 13 levers · Details · Open in the Lab

Predict-Align-Prevent

The map, not the score: a place-based risk surface and the records it concentrates

A nonprofit cuts a city into a grid of small cells and models which cells will see child maltreatment next, from crime, blight, and the built environment. No family is scored. No caseworker sees anything. The output is…

3 cited claims · 13 levers · Details · Open in the Lab

TREWS sepsis early-warning system

The alert that works only when confirmed: a sepsis early-warning model

A machine-learning model scores every inpatient for sepsis and alerts a clinician to evaluate. Modeled on TREWS. Its measured mortality benefit was real - but it accrued only to patients whose alert a provider confirmed…

2 cited claims · 8 levers · Details · Open in the Lab

Advance Alert Monitor (AAM) deterioration model

The alert that never reaches the bedside: a screened deterioration model

A deterioration model scores inpatients hourly and alerts about twelve hours ahead. Modeled on AAM. But the alert never reaches the bedside directly: it goes to a dedicated regional tier of critical-care nurses who…

1 cited claim · 8 levers · Details · Open in the Lab

Sepsis Watch deep-learning detection system

The nurse who gets the alert can't give the order: an authority-split detector

A deep-learning detector scores every emergency department (ED) patient every five minutes. Modeled on Sepsis Watch. The alert goes to a nurse - who cannot order treatment. Only the physician can, so the alert becomes…

2 cited claims · 8 levers · Details · Open in the Lab

Epic Sepsis Model

Switched on before anyone checked: a proprietary sepsis model at scale

A proprietary model shipped inside the electronic health record (EHR) is switched on across hundreds of hospitals - before anyone independent checks whether it works. When they do, it catches a third of sepsis at about…

2 cited claims · 8 levers · Details · Open in the Lab

nH Predict Utilization Review

The order of operations inside a coverage determination

A national health insurer decides whether a patient leaving the hospital gets nursing-home or rehabilitation care covered, and for how long. A vendor coordinator completes a similar-patient stay estimator while the…

4 cited claims · 14 levers · Details · Open in the Lab

Cost-Proxy Care Stratification

The score that was right about the wrong thing: a cost-trained care model

A commercial model scores every patient in a health system's risk contracts from last year's claims, and the top slice of the ranking is offered a scarce care-management program. Modeled on the cost-proxy stratification…

4 cited claims · 9 levers · Details · Open in the Lab

CA-CDS Child Abuse Alerting

The mandated report: alerting that writes into another organisation

An emergency department screens every child under 13 with a five-item nurse questionnaire, and a rule-and-text trigger reads the chart as it fills: the screen result, a free-text scan of the chief complaint and nursing…

4 cited claims · 9 levers · Details · Open in the Lab

Cigna's PxDx post-service claim review

The signature over the match

A national health insurer decides whether it will pay for a test a patient has already had. A code screen compares the procedure billed against an in-house list of diagnoses deemed acceptable for it. Matches are paid.…

5 cited claims · 12 levers · Details · Open in the Lab

EviCore by Evernorth prior-authorization screening

The threshold that decides who gets looked at

One company decides whether care is medically necessary for about 100 million people, on behalf of more than 100 competing insurers. A request arrives with its clinical documentation and is scored with a probability of…

5 cited claims · 14 levers · Details · Open in the Lab

IBM Watson for Oncology

The authored corpus: an advisor sold as machine-read literature

A hospital buys a cancer treatment advisor sold as machine reading of the medical literature. A nurse keys thirteen to seventeen attributes from the chart into its form — the system reads no local record — and it…

4 cited claims · 10 levers · Details · Open in the Lab

IDx-DR Autonomous Screening

The decision with no reader: an authorized autonomous diagnostic

A medical assistant who has never taken a retinal photograph sits a patient with diabetes at a camera, takes two pictures of each eye, and about a minute later the software says one of exactly two things: refer to an…

5 cited claims · 9 levers · Details · Open in the Lab

OPTN eGFR Waiting-Time Correction

The clock that started late: a race coefficient, and the waiting time given back

For over a decade the standard equation for estimating kidney function multiplied the result upward for any patient recorded as Black. A candidate needs an estimate of 20 mL/min or lower to start accruing kidney waiting…

7 cited claims · 8 levers · Details · Open in the Lab

Practice Fusion Pain CDS

The purchased alert: sponsored guidance at the point of care

A free record platform carries a pain alert into the exam room. It asks for a pain score, suggests an inventory when scores recur, then prompts a follow-up plan and offers nine options with opioid therapy sitting level…

7 cited claims · 12 levers · Details · Open in the Lab

Viz.ai LVO Stroke Triage

The alert that gates nothing: notification beside the standard read

A stroke code starts, a CT angiogram is acquired, and two things happen at once. A detector scores the study for a large-vessel occlusion and, on a hit, pushes an alert with a compressed preview to the…

4 cited claims · 11 levers · Details · Open in the Lab

United Behavioral Health's Level of Care Guidelines

The rulebook, and who was in the room when it was written

Counterexample: nothing at the center of this network is automated. Clinicians apply a written rulebook case by case, and the same governance failures the automated networks show arise here without a machine. A…

5 cited claims · 12 levers · Details · Open in the Lab

Kaiser Permanente ambient AI scribe

The draft becomes the record: a well-governed ambient scribe

An ambient AI drafts the clinical note; the clinician edits and signs. Modeled on the largest documented scribe deployment. The benefit is real - thousands of hours of documentation returned. But the draft does not stay…

2 cited claims · 11 levers · Details · Open in the Lab

Ambient scribe RCT + monitoring playbook

The trial and the playbook: an evidenced and monitored ambient scribe

An ambient scribe drafts the note; the clinician edits and signs. Modeled on the family's strongest-evidenced deployment: a randomized trial found real benefit - exhaustion down, ~22 minutes a day returned - and its…

2 cited claims · 11 levers · Details · Open in the Lab

An ambient AI scribe at a multi-specialty health system

One number and two outcomes: an ambient scribe that split by clinician

One ambient scribe drafts for two clinician groups. Modeled on an evaluation that measured the distribution, not just the average: it helped 85.8% of primary-care physicians but only 36.4% of specialists, and the…

2 cited claims · 11 levers · Details · Open in the Lab

YouTube Covid-19 enforcement

The reviewers went home and the error rate showed

Automated classifiers remove content at a scale no human team could match, backed by human reviewers and an appeals queue. Modeled on a natural experiment: when the pandemic sent reviewers home, the platform chose…

2 cited claims · 8 levers · Details · Open in the Lab

Meta content enforcement

Ninety percent overturned on the cases chosen to be seen

Automated classifiers enforce content standards at the scale of millions of decisions, backed by an internal appeals layer and an external oversight board. Modeled on the domain's most built-out correction structure:…

2 cited claims · 8 levers · Details · Open in the Lab

Sports Illustrated AI bylines

The byline nobody was behind

A storied outlet published product reviews by writers who did not exist - invented names, AI-generated headshots, fabricated bios - produced by a contractor, disclosed to nobody. Modeled on a documented case surfaced by…

2 cited claims · 7 levers · Details · Open in the Lab

CNET AI-drafted articles

Half the articles corrected under a byline that promised a review

An AI tool drafts finance explainers published under a human-sounding staff byline. Modeled on a deployment that did not disclose the AI use; when the practice came to light, the outlet's own audit found it had to…

2 cited claims · 8 levers · Details · Open in the Lab

StopNCII & Take It Down

Two ways to run a hash bank: the intimate-image removal pair

Two nonprofits run the same radical design from opposite ends. A person who holds intimate material of themselves computes a hash of it on their own device — the material never leaves their hands — and participating…

5 cited claims · 12 levers · Details · Open in the Lab

X Multilingual Hate-Speech Enforcement

The error table the law forced into twenty languages

Automated classifiers enforce a hateful-conduct policy across the 24 official languages of the European Union, on a platform the law obliges to publish its accuracy indicators broken down by language — the only…

4 cited claims · 12 levers · Details · Open in the Lab

Community Notes on X, formerly Birdwatch on Twitter

The crowd writes the correction; the operator sets the bar

A post goes up. Somebody thinks it is misleading. In every other deployment in this domain the next thing that happens is that a classifier scores the post and something is removed, restricted or downranked. Here…

8 cited claims · 15 levers · Details · Open in the Lab

The GIFCT hash-sharing database and member matching system

One member writes the entry; every member reads it

Thirty-nine platforms share one index. A member company finds terrorist or violent extremist material on its own service, judges it against its own terms of service, judges it a second time against a separate rulebook…

8 cited claims · 14 levers · Details · Open in the Lab

Google's child-safety detection and account enforcement

The flag was right about the image and wrong about the person

A father photographs his toddler son's swollen groin because an advice nurse asks him to, ahead of an emergency telehealth consultation during pandemic-era remote care. The doctor uses the photographs to diagnose the…

9 cited claims · 13 levers · Details · Open in the Lab

Meta's cross-check secondary review programme

Two lanes, one pool of reviewers

Most moderation systems fail by acting on something they should have left alone. This one fails by doing nothing yet. Modelled on the documented record of Meta's cross-check programme, as its own external review body…

7 cited claims · 13 levers · Details · Open in the Lab

The NCMEC CyberTipline reporting and triage system

The statutory funnel: one queue and a borrowed field set

Every platform in one country that finds material recording the sexual abuse of a child must report it to a single place, and that place is forbidden by law to throw anything away. Modeled on the documented statutory…

9 cited claims · 13 levers · Details · Open in the Lab

The Sama Nairobi content-moderation workforce for Meta

Two targets on one person

Every other board in this domain asks whether a system took down the right thing. This one asks what it costs to be the person who decides. Modelled on the documented record of the content-review operation Samasource…

7 cited claims · 11 levers · Details · Open in the Lab

TikTok's EU and UK content-moderation operation

The mandated table and its four examiners

Every video, photo and text item uploaded to this platform passes an automated review before anyone but its creator can see it. Detection runs on vision, audio, text and language technologies plus keyword lists; where a…

14 cited claims · 14 levers · Details · Open in the Lab

Wikipedia's edit-scoring service (ORES, now Lift Wing)

The score is published, the threshold belongs to the governed

A scoring service computes a damage probability for essentially every edit as it is saved, publishes it to anyone who asks without a credential, and has no way at all to act on it. Modeled on Wikipedia's edit-quality…

14 cited claims · 13 levers · Details · Open in the Lab

YouTube's Content ID copyright matching system

The party that gains answers the objection

Almost every deployment in this atlas is a machine an organisation points at its own decisions. This one is a machine an organisation points at everybody else's uploads on behalf of a third party, and the third party…

8 cited claims · 10 levers · Details · Open in the Lab

A contact centre's generative-AI agent assist

Fifteen percent on average and thirty for the newcomers

One agent-assist copilot drafts responses for two agent groups. Modeled on a randomized rollout that measured the distribution, not just the average: about 15% more issues resolved per hour overall, but novices gained…

2 cited claims · 7 levers · Details · Open in the Lab

DPD customer-support chatbot

The guardrails that stopped holding after an update

A parcel firm's support chatbot, after a system update, swore at a customer and composed a poem calling its own operator the worst delivery firm in the world. Modeled on a documented incident: the firm attributed the…

2 cited claims · 7 levers · Details · Open in the Lab

Air Canada chatbot

A policy the chatbot invented and the company that answered for it

A customer-facing chatbot answers policy questions on the organization's website. Modeled on a deployment where the bot stated a bereavement-fare policy that did not exist; a customer relied on it and was refused by…

2 cited claims · 7 levers · Details · Open in the Lab

Klarna AI assistant

Two-thirds of chats handled and a year later a rethink

A customer-facing AI assistant handled about two-thirds of chats in its first month - some 2.3 million conversations, described as ~700 agents' worth of work, resolution time cut from ~11 minutes to under 2,…

2 cited claims · 8 levers · Details · Open in the Lab

Chicago Public Schools On-Track indicator

The rule a teacher can explain

A district flags ninth graders with a rule anyone can read: enough credits, no more than one core failure. Modeled on the best-documented early-warning success in this domain - about 85 percent published accuracy, wired…

2 cited claims · 7 levers · Details · Open in the Lab

Wisconsin DEWS

Wrong most of the time and unevenly by race

An ensemble model scores every grade 6-9 student's risk of not graduating and hands a 'high risk' label to school staff through a dashboard. Modeled on a statewide deployment an audit found wrong ~74% of the time on…

2 cited claims · 10 levers · Details · Open in the Lab

Cleveland State remote proctoring

A camera in the bedroom and a flag that lands by skin tone

A university requires students to pan their webcam around their home before an online exam; proctoring software then flags suspected cheating. Modeled on a deployment where a federal court held the room scan an…

2 cited claims · 8 levers · Details · Open in the Lab

Amazon recruiting engine

It learned who was hired rather than who succeeds

A resume scorer is trained on ten years of the company's own hiring. Modeled on an experimental tool that learned the history rather than the merit: it penalized the word 'women's' and downgraded women's-college…

2 cited claims · 8 levers · Details · Open in the Lab

Workday AI screening

One model across thousands of employers and testing no one can see

One vendor's screening model runs inside thousands of employers' hiring pipelines at once. Modeled on a platform under a live collective action. Two structures to watch, both independent of how the case is decided: one…

2 cited claims · 8 levers · Details · Open in the Lab

Unilever and HireVue graduate hiring

Good audits with real savings — and a cohort no one can see

A graduate-hiring pipeline chains a games screen with video-interview scoring. Modeled on a deployment with its audits on the record: ~90% faster hiring, ~£1M saved, +16% diversity - all company- or vendor-reported -…

2 cited claims · 7 levers · Details · Open in the Lab

Amazon fulfillment-centre productivity discipline

The bar is the people: discipline generated from a percentile

A worker scans a package. The scan is a timestamp, the timestamps are a rate, and the gaps between them accumulate as idle time the system can measure to the second and cannot explain. At the end of the week the rate is…

10 cited claims · 10 levers · Details · Open in the Lab

Amazon Flex driver standing and deactivation

The act is instant; the reason takes a month

A driver reserves a four-hour block, collects packages from a delivery station, and delivers them in their own car. The app is measuring the whole time: arrival at the station, the minutes to each stop, whether the…

5 cited claims · 12 levers · Details · Open in the Lab

Aon's three-instrument pre-hire assessment suite

What the manual reports and what the sales page says

One vendor sells three pre-hire instruments into other companies' hiring pipelines. A computer-adaptive forced-choice personality test scores fifteen constructs from item pairs matched for social desirability, out of a…

6 cited claims · 11 levers · Details · Open in the Lab

Checkr's automated background-check platform

One report, every platform: a shared screening file and its thirty-day correction

A courier applies to drive. A platform sends the identity to a screening vendor, and the vendor does five things in a row: it pulls criminal items out of a bulk store assembled ahead of the request, it decides which of…

7 cited claims · 12 levers · Details · Open in the Lab

CVS Health's Massachusetts applicant video-interview screen

Integrity video screening under a 1959 lie-detector statute: the employer's notice duty

A national retail employer routes applicants for its positions in one state into a recorded video interview run on a third-party platform. The questions are about integrity: what it means to the applicant, and an…

7 cited claims · 8 levers · Details · Open in the Lab

HireVue's video interview and assessment platform

The engine is the vendor's; the duty is the employer's

One company builds the video-assessment engine. Hundreds of separate employers buy it. A candidate records answers to a fixed question set, or plays a game-based assessment; models score verbal and paraverbal features…

6 cited claims · 13 levers · Details · Open in the Lab

Intuit's recorded video assessment for promotion

Assessing an incumbent: a recorded promotion gate and the captioning request

A large technology employer runs an internal promotion through a standardised recorded assessment. A seasonal employee of five years applies for the next role up. She has already been promoted once, she leads a team…

7 cited claims · 12 levers · Details · Open in the Lab

iTutorGroup Tutor Application Screen

A rule someone wrote down, found by one changed field

An application form asks for a birthdate, and software at the top of the hiring funnel decides the application from it. Modeled on a deployment where, per the regulator's complaint, the rule was written by hand rather…

5 cited claims · 9 levers · Details · Open in the Lab

McHire, McDonald's franchise hiring platform

Everything the applicant typed, and who could ask for it

A person applies for a job at a restaurant and talks to a chatbot. It takes their name, email address, phone number, home address and shift preferences in ordinary conversation, asks the screening questions, hands them…

6 cited claims · 12 levers · Details · Open in the Lab

Meta employment-ad targeting and delivery optimization

The guardrail and the leak: who a job ad reaches

A job ad is written to reach everyone, and then something decides who actually sees it. This network is the second half of that sentence. An employer composes a job ad inside a restricted advertising portal that will…

7 cited claims · 11 levers · Details · Open in the Lab

pymetrics Soft Skills Platform cooperative audit

The examination that priced its own limits: a paid source-code audit as the deployment

A hiring-assessment company paid a university team to read its source code, and then let them publish what they found. That is the deployment on this board. Not the screening product — the examination of it. In March…

9 cited claims · 14 levers · Details · Open in the Lab

Sirius XM Radio's iCIMS-based applicant screening

Bought, configured, answered for: an employer's licensed applicant screen

A person applies for a job. The application goes into an applicant-tracking system the employer licenses from somebody else, and that system does several things before anyone at the company sees anything: it parses the…

8 cited claims · 12 levers · Details · Open in the Lab

Allegheny Housing Assessment

The score and the scarce bed: a coordinated-entry housing tool

A 1-to-10 score built from linked county records ranks who gets scarce housing first, and for the cases with too little history a self-report fallback stands in. Modeled on Allegheny's Housing Assessment. Watch two…

1 cited claim · 9 levers · Details · Open in the Lab

VI-SPDAT

The standard nobody validated: a homelessness triage score

A yes/no interview becomes a 0-to-17 score, and the score decides who reaches the top of the housing line. Modeled on the Vulnerability Index-Service Prioritization Decision Assistance Tool (VI-SPDAT). Watch the thing…

2 cited claims · 11 levers · Details · Open in the Lab

LA County Homelessness Prevention Unit

The help you have to be found for: a homelessness-prevention model

A model ranks tens of thousands of residents by their risk of losing housing and hands outreach workers a short list to go find — an offer of cash and help, never a denial, and about nine in ten of the people actually…

2 cited claims · 10 levers · Details · Open in the Lab

Xantura OneView (predictive homelessness flagging)

The flag no one can reach: a predictive homelessness-prevention platform

A vendor platform integrates fifteen-plus council data feeds into one household view and flags who may become homeless months ahead; the flags outnumber the single officer who can act on them, so most are never…

2 cited claims · 11 levers · Details · Open in the Lab

CHAI (chronic-homelessness prediction)

The people the data can't see: a consent-based homelessness-risk model

A city built its own model to flag shelter clients heading toward chronic homelessness, and did much of it right: it explains its reasoning to the caseworker, it lets people opt out, and it never makes the call itself.…

2 cited claims · 14 levers · Details · Open in the Lab

Imagine LA Benefit Navigator copilot

Best where you can check it least: a benefits-navigation copilot

This copilot answers a caseworker's benefits question from a curated set of approved government documents and hands back the exact quotes to check, all while the client is on the line. Modeled on the Imagine LA Benefit…

1 cited claim · 10 levers · Details · Open in the Lab

London's Strategic Insights Tool

One shared memory and thirty-three readers: consolidating a city's rough-sleeping records

A city merges three separately governed record systems - street-outreach contacts, charity casework, and borough statutory applications - into one linked picture of who is sleeping rough, and lets all 33 local…

2 cited claims · 10 levers · Details · Open in the Lab

San Jose's camera car

The rule you declare and the flow you leave open: a camera car aimed at who is sleeping outside

A city mounts cameras on a sedan and drives one district, training AI to spot potholes and trash - and, in the same pilot, RVs, lived-in vehicles, and homeless encampments. Modeled on San Jose's Road Safety Conditions…

2 cited claims · 12 levers · Details · Open in the Lab

LA's coordinated-entry triage revision

Two scores in one queue: the transition that let a retired bias back in

The largest homeless-services system in the country retires a triage survey that barely beat a coin flip and systematically under-scored people of color, and replaces it with a fairer score built on tens of thousands of…

2 cited claims · 10 levers · Details · Open in the Lab

Santa Clara County Homelessness Prevention System

A measured lever on an unmeasured target: a homelessness-prevention screen

A points questionnaire scores a household's risk of losing its home, and a caseworker offers the highest-scoring ones a few thousand dollars - back rent, a deposit, a car repair - never a denial. Modeled on Santa Clara…

2 cited claims · 11 levers · Details · Open in the Lab

Calgary Drop-In Centre

The canvas rather than the answer: interpretable screening a shelter's own staff choose to check

A shelter and a university build a deliberately simple screening tool — explicit rules like "81 or more stays in ninety days" that flag chronic shelter use months earlier than the official definitions — and then,…

2 cited claims · 10 levers · Details · Open in the Lab

SafeRent Tenant Screening Score

SafeRent Score: the number the leasing desk could not change

A screening company turns a rental applicant's file into a number between 200 and 800 and an accept, decline or conditional recommendation, measured against a cutoff the property company picked. Modelled on the SafeRent…

3 cited claims · 12 levers · Details · Open in the Lab

RealPage revenue management

One engine, many rivals: a shared pricing model and the record it writes back

One vendor-hosted engine recommends a daily rent for each unit. Its users are competitors. It is calibrated on their pooled lease transactions, and it sends the answer back to all of them at once. Modelled on the…

3 cited claims · 11 levers · Details · Open in the Lab

Homebase Risk Assessment Questionnaire

The prevention screener that has to describe itself in public every year

A caseworker asks a household fifteen questions and adds up the points. Somewhere between zero and twenty-five, the total decides whether the family gets full prevention help or a brief visit. Modelled on New York…

3 cited claims · 10 levers · Details · Open in the Lab

CrimSAFE criminal-record tenant screening

Criminal-record tenant screening: the flag the leasing desk was configured not to see

A rental application is screened against a criminal-records store aggregated from more than 800 jurisdictions. Modeled on the CrimSAFE tenant-screening litigation: its shape, not the real system. Nothing here computes a…

3 cited claims · 11 levers · Details · Open in the Lab

BAMF dialect recognition

One clue among many or the thing that decides

Dialect-recognition AI estimates an asylum applicant's origin from a speech sample, as one clue in a credibility assessment. Modeled on a deployment whose own caseworkers call it a 'rough compass' - imprecise by nature…

2 cited claims · 11 levers · Details · Open in the Lab

Home Office IPIC

A human decides and the form only asks why not

IPIC (Identify and Prioritise Immigration Cases) is an enforcement-triage algorithm that recommends people for immigration actions — returns, bail, casework — from sensitive detention, health, and location data, with a…

2 cited claims · 8 levers · Details · Open in the Lab

Predictive maintenance on a high-speed rail fleet

The contract that prices every miss

A high-speed rail fleet is maintained by the manufacturer as a service, against a promise that refunds the full fare if a journey runs more than fifteen minutes late. Modeled on a documented deployment: 300 sensors per…

2 cited claims · 7 levers · Details · Open in the Lab

Audi press-shop inspection

The inspection the model inherited

An automaker's in-house deep-learning system inspects pressed sheet-metal parts for hairline cracks. Modeled on a documented multi-year build: trained on terabytes of images pooled from seven presses and several plants,…

2 cited claims · 9 levers · Details · Open in the Lab

BMW AIQX inspection

The flag is not the catch until someone acts on it

Camera and acoustic AI flags defects on the assembly line in real time; a line worker responds and can stop the line. Modeled on an at-scale deployment (a thousand-plus vehicles a day, sub-minute takt) now a company…

2 cited claims · 8 levers · Details · Open in the Lab

A heavy-industry predictive-maintenance deployment

Ninety percent fewer false alarms while the crews still label

Sensor analytics forecast equipment failures; crews investigate the alerts and label which were real, and the model retrains on the labels. Modeled on a peer-reviewed case study that cut false alarms ~90% through…

2 cited claims · 8 levers · Details · Open in the Lab

Automated visual inspection of injectable drugs

Erring toward the scrap heap while guarding the one that gets through

ML visual inspection examines filled injectables for defects, deliberately tuned to over-reject: a missed defect reaching a patient is a safety failure, a scrapped good vial is only a cost. Modeled on a regulated…

2 cited claims · 9 levers · Details · Open in the Lab

Upstart lending model

Regulator-verified access — and a search left at an impasse

A machine-learning model approves, declines, and prices credit with no per-application human review. Modeled on a deployment with both regimes on the record: a regulator published its access gains - about 27% more…

2 cited claims · 5 levers · Details · Open in the Lab

Apple Card underwriting

Cleared on the numbers but unable to say why

An automated model underwrites a widely used consumer card. Modeled on a deployment a regulator investigated after a viral bias allegation: it analyzed about 400,000 applicants and cleared the model - no unlawful…

2 cited claims · 7 levers · Details · Open in the Lab

Earnest AI underwriting

A neutral-looking feature and the testing no one ran

An automated model underwrites student loans. Modeled on a deployment a state attorney general settled with for $2.5 million. The model priced a school's cohort default rate into an individual's terms - a feature that…

2 cited claims · 7 levers · Details · Open in the Lab

M-Shwari & Kenya's Digital Credit Market

Scored from a phone's records, listed where the market reads

A bank scores people who have no credit file at all, from six months of the records their phone company holds: airtime top-ups, low-balance days, money in and out. Modeled on Kenya's mobile-money credit market and its…

4 cited claims · 8 levers · Details · Open in the Lab

Citi Retail Services Judgmental Review

The reason on the form and the reason in the room

Counterexample: nothing at the center of this network is automated. Staff apply a screening rule that was taught, never written down, and the same governance failures the automated networks show arise here without a…

5 cited claims · 8 levers · Details · Open in the Lab

Credit Acceptance's net-collections Score, inside CAPS

The forecast counted what it would collect after the default

A person with damaged credit needs a car. A dealership enrolled in a lender's program pulls up that lender's software, enters the application, and starts assembling a deal: this vehicle, this price, this term, this down…

8 cited claims · 10 levers · Details · Open in the Lab

Dave ExtraCash (CashAI)

The number in the advertisement and the number the engine returns

A person opens an app, taps to connect their checking account, and inside five minutes a number appears. The advertisement above it says up to $500. Modeled on a consumer cash-advance product whose eligibility and…

6 cited claims · 13 levers · Details · Open in the Lab

Enova's CashNetUSA and NetCredit loan servicing

Eleven ways the money left

An online lender runs its own loan-servicing software, and that software takes money out of customers' bank accounts. This board is not about a credit decision. It is about the arithmetic and the plumbing behind a loan…

7 cited claims · 13 levers · Details · Open in the Lab

Equifax's Online Model Server

Three weeks of a frozen calendar

A lender pulls a credit score. Behind the request sits one platform that takes the consumer's credit file, derives credit attributes from it, runs third-party scoring models over those attributes, and returns a number —…

7 cited claims · 13 levers · Details · Open in the Lab

The Digit automated-savings tool, or Oportun Set & Save

The promise was the product, and the forecast moved the money

A member links their checking account, and from then on an algorithm reads it and moves their money. It forecasts the near-term balance, decides what the member can spare, and initiates a transfer out of the account…

5 cited claims · 14 levers · Details · Open in the Lab

Navy Federal mortgage underwriting

Three readings of the same lending

A credit union underwrites residential mortgages through what the pleadings and the appeals court both call an at-least semi-automated process, built on a proprietary algorithm whose variables and weights it does not…

5 cited claims · 8 levers · Details · Open in the Lab

Oportun Financial Corporation's legal-collections pipeline

Thirty filings a day, until a reporter asked

A lender exists to give people with no credit history a first loan. That is not a slogan here: it is a Treasury certification, held since 2009, and a business built on scoring bank transaction data and public records so…

9 cited claims · 11 levers · Details · Open in the Lab

Santander Consumer USA's loss forecasting score

The score set the price, and a different desk set the loan

A borrower with damaged credit walks into a dealership. The dealership picks the vehicle and its price, the term, the down payment and the products financed into the amount owed, and types in what the borrower says they…

8 cited claims · 12 levers · Details · Open in the Lab

TransUnion OFAC Name Screen

Two fields, and the file that held the rest

A credit bureau sold an add-on. When a bank or a landlord or a car dealership pulled somebody's credit report, the bureau took the first and last name typed on the inquiry, sent that name and nothing else to a third…

5 cited claims · 11 levers · Details · Open in the Lab

Wells Fargo refinance underwriting (CORE/ECS)

Fourteen thousand rules and nobody to read them

A bank decides refinance applications with three separate systems and one person. A front-end tool holds the file and shows it; on the bank's own sworn account, accepted as undisputed in a public court order, that tool…

4 cited claims · 10 levers · Details · Open in the Lab

UPS delivery route optimization

A hundred million miles saved and the discretion it cost

A route-optimization system computes and dictates each driver's delivery route and monitors adherence through telematics. Modeled on a documented OR success - ~100M miles and ~10M gallons of fuel saved a year - that is,…

2 cited claims · 9 levers · Details · Open in the Lab

Amazon fulfillment-centre algorithmic management

Units per hour up and a cost measured in bodies

Algorithmic management assigns tasks and sets the pace of warehouse work, with robots raising throughput. Modeled on a deployment whose human-robot picking benefit is peer-reviewed AND whose pace a safety regulator and…

2 cited claims · 10 levers · Details · Open in the Lab

Michigan MiDAS

Automation without review: a benefits-fraud system

A fraud system writes determinations straight into people's benefit records with no human in between — then enforcement copies them automatically. Modeled on Michigan's MiDAS. The damage flows through the records, not…

3 cited claims · 12 levers · Details · Open in the Lab

Michigan MiDAS

After the settlements: review returns to a benefits-fraud system

The same automated fraud-determination deployment as the MiDAS System network, at its second documented instant. Litigation changed the structure: the Zynda settlement required human review of fraud determinations to…

3 cited claims · 4 levers · Details · Open in the Lab

Rotterdam welfare-fraud risk model

The suspicion machine: a welfare-fraud risk model

A vendor model ranks welfare recipients by fraud risk, and investigators work the list top-down. Modeled on Rotterdam's system. Watch the retraining loop: the model learns from the outcomes its own rankings produced, so…

2 cited claims · 13 levers · Details · Open in the Lab

Arkansas ARChoices / ARIA

When the tool sets the hours: a home-care hours allocator

An algorithm sets each person's Medicaid home-care hours from a scored assessment, and nurse assessors are bound to its number with almost no room to override. Modeled on Arkansas's ARChoices/ARIA. The risk here isn't a…

2 cited claims · 11 levers · Details · Open in the Lab

Netherlands childcare-benefits scandal (Toeslagenaffaire)

The institutional amplifier: a childcare-benefits fraud-hunt

A modest fraud-risk model flags an application; a handler treats the flag as fraud rather than a question to check; the label lands on a 270,000-person blacklist that no one ever corrects; and an all-or-nothing statute…

2 cited claims · 13 levers · Details · Open in the Lab

SyRI (Netherlands)

Struck down before the harm was counted: a secret welfare-fraud dragnet

A secret model links records across many government databases and flags people for fraud investigation, and the flags land in a two-year register that other agencies re-read and re-link — but the people flagged are…

2 cited claims · 11 levers · Details · Open in the Lab

CNAF benefit-fraud risk score (France)

The score that suspects the vulnerable: a benefit-fraud risk model

A national model scores every benefit-receiving household for fraud suspicion each month, and controllers work the highest scores first — up to the most invasive on-site checks. Modeled on France's CNAF/CAF datamining…

2 cited claims · 11 levers · Details · Open in the Lab

Forsakringskassan VAB fraud-selection profile (Sweden)

The audit the agency refused: a secret fraud-selection profile no one outside could see

A secret machine-learning profile scores every applicant for a child-care benefit and routes the highest-scored to fraud investigators, and the outcomes of those investigations become the labels that train the next…

2 cited claims · 10 levers · Details · Open in the Lab

Udbetaling Danmark data-driven control (Denmark)

Standing surveillance by data-linking: a welfare-fraud control suite

A Joint Data Unit links around ten national registers over millions of benefit recipients and runs up to sixty models that score them into a wonderlist of high-risk people; a human control team then works the…

2 cited claims · 12 levers · Details · Open in the Lab

BOSCO (Spain)

The secret code: an eligibility engine that gives no reasons

A rules engine decides who qualifies for an electricity-bill discount and gives no reasons, so a wrong denial is nearly impossible to spot or contest — and one uniform bug denies thousands of eligible people at once.…

2 cited claims · 9 levers · Details · Open in the Lab

Serbia Social Card (Socijalna karta)

Cut off by a data match: a social-assistance registry

A rules-based registry cross-links roughly 130 to 135 data fields from other state registers to check who still qualifies for social assistance and to flag suspected income, and a matched flag flows into a benefit cut…

2 cited claims · 10 levers · Details · Open in the Lab

UK DWP Universal Credit Advances fraud model

The self-audited skew: a benefits fraud-scoring model

A Department for Work and Pensions (DWP)-built model scores every Universal Credit advance request for fraud risk and refers the highest-risk ones to a caseworker who is deliberately not shown the score. Modeled on the…

2 cited claims · 11 levers · Details · Open in the Lab

ID.me identity verification

The gate nobody counts: an identity check in front of benefits

A facial-recognition identity check sits in front of an unemployment claim: match a live selfie to a government-ID photo, pass and the claim proceeds, fail and you drop into a video-interview queue that in some states…

2 cited claims · 10 levers · Details · Open in the Lab

Medicaid unwinding ex-parte renewals

The unit of determination: an automated renewal system at population scale

A deterministic engine renews coverage automatically by matching records before asking anyone for a form — until one wrong setting, the unit of determination, evaluates every renewal by household instead of by person,…

1 cited claim · 10 levers · Details · Open in the Lab

INSS automated benefit analysis

Automation as queue management: when the metric makes denial the fastest way out

A benefit backlog in the millions, and a productivity metric that counts processes analyzed rather than the quality of each decision. Under that metric denial is the fastest way to clear a case, so automation used to…

1 cited claim · 11 levers · Details · Open in the Lab

Samagra Vedika

The match that cancels you: entity resolution as eligibility

A matcher links each person against thirty-plus government databases and decides eligibility from the merged profile, and a similarly-named stranger's car or land can be pulled onto you — silently flipping the flag that…

3 cited claims · 11 levers · Details · Open in the Lab

Workforce Australia Targeted Compliance Framework

The lesson not learned: automated compliance sanctioning after a scandal

A demerit-and-zone engine suspends and cancels welfare payments automatically for mutual-obligation failures — until a cancellation runs without the discretionary reasonable-excuse step the law required, and the…

1 cited claim · 11 levers · Details · Open in the Lab

NYC MyCity business chatbot

Exposure is not correction: a public-facing government advice chatbot

This chatbot answered business owners' questions about city rules in a confident, official-sounding voice — and it was repeatably wrong on the law, telling businesses they could take a cut of workers' tips or that…

3 cited claims · 9 levers · Details · Open in the Lab

Nevada DETR generative-AI unemployment appeals

The referee who signs: an AI that drafts the ruling

An AI reads an unemployment-appeal hearing and its evidence, then drafts the ruling itself — approve, deny, or modify the claim — along with the written legal decision, and hands it to a referee to sign. Modeled on…

2 cited claims · 12 levers · Details · Open in the Lab

Tennessee TennCare TEDS

The notice that never came: an automated Medicaid eligibility system

An automated system decides Medicaid eligibility and runs renewals on its own, then generates the termination notices that are supposed to let people appeal. Modeled on Tennessee's TEDS. When the notice is misleading,…

1 cited claim · 12 levers · Details · Open in the Lab

Robodebt (Australia)

The debt you must disprove: an income-averaging engine

An income-averaging engine raises a debt by spreading a year of tax-office income evenly across fortnights, then places the burden on the recipient to disprove it while recovery acts automatically. Modeled on…

2 cited claims · 10 levers · Details · Open in the Lab

Robodebt (Australia)

After the Commission: repaying the debts an engine raised

The same income-averaging deployment as the Robodebt Scheme network, at its second documented instant. The courts and a Royal Commission — the external actors the case file records as the scheme's actual end — have…

2 cited claims · 5 levers · Details · Open in the Lab

Indiana / IBM eligibility modernization

Denied for 'failure to cooperate': a privatized eligibility pipeline

A statewide benefits system is handed to a private consortium — call centers and document imaging replace local caseworkers, and a timeliness clock closes any case whose paperwork is not matched in time, for 'failure to…

2 cited claims · 11 levers · Details · Open in the Lab

ML anti-money-laundering as primary monitoring

Fewer alerts and more confirmed — but confirmed by whom?

A bank replaces rules-based monitoring with a machine-learning anti-money-laundering (AML) system: 2-4x more confirmed suspicious activity, 60% fewer alerts. Modeled on a vendor-reported deployment. Both numbers are…

2 cited claims · 9 levers · Details · Open in the Lab

Fraud false positives that froze real accounts

The wrong flag that took ninety days to reverse

A fraud model freezes accounts flagged as suspicious - and triggered by pandemic benefit deposits, it froze legitimate customers at scale. Modeled on a deployment a regulator later penalized. The visible failure is the…

2 cited claims · 10 levers · Details · Open in the Lab

Danske Bank fraud scoring

Better detection but worse reimbursement — and a rule that moved it

A bank replaces a legacy fraud system - 40% detection, 99.5% false positives - with a real-time deep-learning engine claiming far better detection. Modeled on a documented rollout. But the same bank ranked worst at…

2 cited claims · 8 levers · Details · Open in the Lab

A commercial code assistant across three enterprises

Big lift for novices but slower for experts: a coding assistant

One coding assistant suggests code to an org's whole engineering team. Modeled on a pre-registered randomized rollout: a big lift for less-experienced developers, and - measured independently - about 19% slower for…

2 cited claims · 11 levers · Details · Open in the Lab

Google ML code completion

Owning every node: the strength and the missing check

One organization builds the model, owns the repository, runs the review gates, and defines the telemetry. Modeled on a large in-house code-completion deployment. That unification is the strength - every lever is inside…

2 cited claims · 11 levers · Details · Open in the Lab

Gated coding-assistant rollout at a regulated bank

The gate that recorded what it couldn't resolve: a bank's rollout

A regulated bank piloted a coding assistant with ~100 engineers, evaluated, then scaled to ~1,000. Modeled on its own published rollout. It reported gains - and recorded the security impact as explicitly inconclusive: a…

2 cited claims · 7 levers · Details · Open in the Lab

GitHub Copilot at ZoomInfo

Measured carefully but measuring the wrong thing: an ordinary rollout

A well-run four-phase rollout to 400+ developers, carefully documented: acceptance rates, satisfaction, per-language deltas, stated limitations. Modeled on an ordinary competent adoption. It does most things right - but…

2 cited claims · 8 levers · Details · Open in the Lab