PAN Lab example

Eckerd Rapid Safety Feedback

Endorsed then evaluated: a risk tool that spread on a claim

A proprietary profile flags open cases that look like past tragedies, and quality-assurance reviewers coach frontline workers on the ones it surfaces. Modeled on Eckerd's Rapid Safety Feedback and its Hillsborough-to-multistate spread. The founding success was the vendor's own before-and-after numbers; a federal commission called the approach innovative; the tool spread state to state. Years later, an independent evaluation found it changed nothing. The trap here isn't a wrong number — it's a number everyone trusted because someone respectable repeated it.

Stylized model of a documented deploymentChild welfare & family services

Open this example in PAN Lab v0.1 to apply pressures and levers and watch what the system does.

What this models

This example runs on the RSF-origin-class spread-ahead-of-evidence risk tool network: 4 components and 11 pathways between them. Every context in the Lab is a stylized model, never a reconstruction of any actual deployment, and each assumption behind it carries a provenance label.

Evidence base: 3 assumed · 3 published baseline. In the Lab, the shaded evidence band behind each headline readout draws its width from the least-established class below.

assumed
This models the spread-ahead-of-evidence pattern documented in the Eckerd Rapid Safety Feedback origin case file — not a reconstruction of the actual tool.
baseline
The model-to-model coupling encodes the documented multi-jurisdiction spread as correlated error: one proprietary profile was copied and tailored to each new state, so a shared blind spot travels with it rather than being re-tested on arrival.
baseline
The independent-validation check is drawn on the map but runs dry at baseline: no independent evaluation tested the profile's predictive claims before it spread, and the peer-reviewed evaluation that eventually did found no effect on repeat high-severity maltreatment — the defining absence this shape turns on, and the gap the cross-model-check lever opens.
baseline
The quality-assurance coaches carry a real but thin second-read pathway of their own — the vendor's supervisory 'second set of eyes' — but the layer coached workers toward the flagged cases more than it tested whether the flags were right; whether that coaching improved outcomes is recorded in the case file, not computed here.
assumed
The feedback loop from a jurisdiction's own past serious-injury and fatality cases into the profile is present at baseline, reflecting the documented account of a tool matched against local historical outcomes; a documented landscape concern is that such tools can perpetuate past agency patterns.
assumed
Differential harm to children and families is not modeled here; the documented concern that historical-data tools can replicate past biases is a landscape warning, not a measured disparity for this tool, and any such harm is recorded in the case file and measured outside a diagram like this one.

What this example does not show

Differential harm to children and families is not modeled here; the Lab models institutional propagation only, and the concern that historical-data tools can replicate past biases is documented as a landscape warning in the case file, not a measured disparity for this tool.

Sources and evidence

What this example rests on, claim by claim. Every entry resolves to the same ledger the Evidence Registry publishes.

Eckerd's Rapid Safety Feedback spread from Hillsborough County, Florida to child-welfare agencies in several states — promoted on the vendor's own reported gains and highlighted as “innovative” in a 2016 federal commission report — years before an independent 2022 peer-reviewed evaluation found the process did not lower repeat high-severity maltreatment among children identified as high risk (a joint odds ratio of about 1.05).
empirical
- Vendor Eckerd Connects, Eckerd Rapid Safety Feedback Highlighted in National Report of the Commission to Eliminate Child Abuse and Neglect Fatalities (2016) https://eckerd.org/eckerd-rapid-safety-feedback-highlighted-national-report-commission-eliminate-child-abuse-neglect-fatalities/
- Trade press Route Fifty (Government Executive), Saving Children, One Algorithm at a Time (2016) https://www.route-fifty.com/digital-government/2016/07/saving-children-one-algorithm-at-a-time/299671/
- Academic Parker, Williams, Pecora and Despard, Examining the Effects of the Eckerd Rapid Safety Feedback Process on Repeat Maltreatment (Child Abuse and Neglect, 2022) https://pubmed.ncbi.nlm.nih.gov/36044790/
- Advocacy Florida's Children First, Eckerd Youth Alternatives Gets Child Protection Contract in Hillsborough County (2012) https://www.floridaschildrenfirst.org/eckerd-youth-alternatives-gets-child-protection-contract-in-hillsborough-county/
A single automated rule set applied uniformly and without human review produced tens of thousands of correlated wrongful fraud determinations in the documented Michigan MiDAS case — one flaw repeating at caseload scale rather than averaging out.
empirical
- Government Michigan AG, settlement of civil-rights class action (Bauserman, 2022) https://www.michigan.gov/ag/news/press-releases/2022/10/20/som-settlement-of-civil-rights-class-action-alleging-false-accusations-of-unemployment-fraud
- Investigative IEEE Spectrum, Michigan's MiDAS unemployment system: Algorithm alchemy that created lead, not gold https://spectrum.ieee.org/michigans-midas-unemployment-system-algorithm-alchemy-that-created-lead-not-gold

Where this connects

Institutional pressures in this domain

Workload surge — Demand outruns staffing; per-case attention shrinks and review becomes triage.
Deadline pressure — Statutory or managerial timeliness rules reward fast approval of machine output over slow disagreement.
Staff turnover — Experienced skepticism leaves; new staff calibrate their trust on the tool itself.
Data & policy drift — The world, the intake process, and the rules change under a system trained on how things used to be — two mechanisms with different remedies: the statistical properties of what the system processes move (concept drift), or the mixture of inputs arriving in deployment differs from the mixture it was trained on (covariate shift).
Compliance over substance — Paper controls (sign-offs, checklists) satisfy audits while the behavior they describe erodes.

All of them in context on the Child welfare & family services domain page.

Levers available here and the patterns behind them

Escalate checks — State-feedback vigilance
Gate record entries — Human-in-the-loop write gating
Gate vendor updates — Vendor quality gate
Require sign-off — Conformity assessment gate
Understand the system — Understand the system
Review on schedule — Oversight cadence & retrospectives
Check with a second model — Cross-model verification
Keep skills sharp — Deskilling-arrest mandate
Mark AI-written records — Provenance labeling
Upgrade model — Improve the model
Review the riskiest first — Risk-tiered oversight