Skip to content
ParamergeParamerge
Stephen Lieberman
Through Paramerge
Contact

PAN Lab example

Oregon Safety at Screening

The fix then the off switch: a fairness-corrected screening tool

A fairness-corrected risk score lands in front of state hotline screeners, who decide whether to investigate — and an internal review later chose to switch the whole tool off. Modeled on Oregon's Safety at Screening tool. Watch two things the other child-welfare rounds don't show as sharply: a bias-feedback loop the fairness correction was built to counteract but not sever, and a discontinuation authority that can simply stop the deployment.

Stylized model of a documented deploymentChild welfare & family services

Open this example in PAN Lab v0.1 to apply pressures and levers and watch what the system does.

What this models

This example runs on the Oregon-class fairness-corrected screening tool network: 5 components and 10 pathways between them. Every context in the Lab is a stylized model, never a reconstruction of any actual deployment, and each assumption behind it carries a provenance label.

Evidence base: 6 assumed · 1 published baseline. In the Lab, the shaded evidence band behind each headline readout draws its width from the least-established class below.

  • assumed

    This models the fairness-corrected, then discontinued screening-tool pattern documented in the Oregon Safety at Screening case file — not a reconstruction of the actual tool.

  • assumed

    The error-rate-balance correction is drawn as a mediating artifact on the model → screeners pathway — the documented post-processing that re-binned raw probabilities into the four-tier score using group-specific thresholds. It carries no flow of its own and does not affect the dynamics; whether it reduced documented disparity is recorded in the case file, not computed here.

  • baseline

    The equity-review node carries a light review pathway of its own and represents the internal actor that retired the tool in 2022. The documented discontinuation was a one-time governance event recorded in the case file, not a continuous oversight loop measured here.

  • assumed

    The feedback loop from decisions into future scores is present at baseline, reflecting the agency report's own acknowledgment that the administrative data embeds years of prior human decisions; the fairness correction was meant to counteract that loop, not to sever it.

  • assumed

    Oregon deliberately limited the model to internal child-welfare data (no call text or voice) and layered in automation-bias mitigations — a coarse four-tier score, scores shown only after the screener finished data entry, and framing as a historical indicator — so the score-anchoring pathway is authored as moderate rather than dominant, and the within-agency data-reuse pathway is not marked privacy-sensitive.

  • assumed

    Screener discretion is real and retained: the score was advisory, required only an acknowledgement that it had been reviewed, and left workers with full screen-in/screen-out authority.

  • assumed

    Documented racial-disparity concerns in child-welfare screening, and the fairness correction's measured effect on error-rate balance, concern served children and families. This Lab models institutional propagation, not demographics, and estimates no differential harm to served people; those are documented in the case file and measured outside any diagram like this one.

What this example does not show

  • Bias propagates here the way failures do — through records, retrieval, and the score's own feedback loop, as institutional workflow propagation. The Lab models no demographics and estimates no differential harm to served people; the documented racial-disparity concerns and the fairness correction's measured effect are recorded in the case file and measured outside any diagram like this one.

Sources and evidence

What this example rests on, claim by claim. Every entry resolves to the same ledger the Evidence Registry publishes.

  • Oregon's child-welfare agency dropped its AFST-derived Safety at Screening tool in 2022, citing equity concerns amid national scrutiny of racial disparity in child-welfare algorithms.

    empirical
    • Investigative NPR/AP, Oregon is dropping an AI tool used in child welfare system (2022) https://www.npr.org/2022/06/02/1102661376/oregon-drops-artificial-intelligence-child-abuse-cases
    • Investigative Willamette Week, Oregon DHS to End Its Use of Child Abuse Risk Algorithm (2022) https://www.wweek.com/news/state/2022/06/04/oregon-department-of-human-services-ends-its-use-of-child-abuse-risk-algorithm/

Where this connects

Institutional pressures in this domain

  • Workload surge — Demand outruns staffing; per-case attention shrinks and review becomes triage.
  • Deadline pressure — Statutory or managerial timeliness rules reward fast approval of machine output over slow disagreement.
  • Staff turnover — Experienced skepticism leaves; new staff calibrate their trust on the tool itself.
  • Data & policy drift — The world, the intake process, and the rules change under a system trained on how things used to be — two mechanisms with different remedies: the statistical properties of what the system processes move (concept drift), or the mixture of inputs arriving in deployment differs from the mixture it was trained on (covariate shift).
  • Compliance over substance — Paper controls (sign-offs, checklists) satisfy audits while the behavior they describe erodes.

All of them in context on the Child welfare & family services domain page.

Levers available here and the patterns behind them

Documented case histories