Skip to content
ParamergeParamerge
Stephen Lieberman
Through Paramerge
Contact

PAN Lab example

Insight Bristol / Think Family Database

The database nobody could audit: a shared child-risk profiling system

Several risk models score children from one shared cross-agency database that every agency reads and writes — and front-line staff, distrusting the scores, often did not act on them. Modeled on Bristol's Insight Bristol and its Think Family Database. The trap is not over-reliance: it is a distrusted tool that kept collecting, kept scoring, and kept shaping a shared record no one could later audit — the source code and the reason two models were switched off both lost.

Stylized model of a documented deploymentChild welfare & family services

Open this example in PAN Lab v0.1 to apply pressures and levers and watch what the system does.

What this models

This example runs on the Think-Family-class shared-database risk-profiling system network: 5 components and 12 pathways between them. Every context in the Lab is a stylized model, never a reconstruction of any actual deployment, and each assumption behind it carries a provenance label.

Evidence base: 3 assumed · 3 published baseline. In the Lab, the shaded evidence band behind each headline readout draws its width from the least-established class below.

  • assumed

    This models the shared-database predictive-profiling pattern documented in the Insight Bristol / Think Family Database case file — not a reconstruction of the actual system.

  • baseline

    The model-to-staff adoption pathway starts low, not high: the documented failure was distrust and non-use — an independent evaluation judged the risk-scoring models the weakest element of the database, staff concluded the exploitation models were not fit for operational use, and by one account victims of exploitation scored below people involved in burglary. This is the inverse of the deference concern that runs through most of the library.

  • assumed

    The shared cross-agency database is the center of gravity: the source agencies' record systems are drawn as a second store replicating into it, and several models scored from that one fused substrate — so the model-to-model coupling here encodes correlated blind spots inherited from the shared store rather than idiosyncratic model error.

  • baseline

    The two check pathways are drawn but dormant at baseline. No independent validation reconciled the models against ground truth, and the independent ethics and data-protection review had gone quiet — an ethics committee reportedly stopped revisiting the predictive analytics after 2017, and freedom-of-information (FOI) responses indicate no record was kept of why the exploitation models were later switched off. Both absences are this case's signature, and levers can open them.

  • baseline

    Independent scrutiny arrived late and the system's own provenance was lost: auditors could not locate the exploitation models' source code or variable lists. The Lab represents this as dormant reconciliation and review checks, not as any measured verifiability rate.

  • assumed

    Documented concerns about indirect discrimination and poverty-proxy inputs are recorded externally in the case file. This Lab models institutional propagation, not demographics, and estimates no differential harm to served people.

What this example does not show

  • The documented concerns include indirect-discrimination risk and inputs acting as proxies for poverty. The Lab models institutional propagation, not demographics, and estimates no differential harm to served people; that risk is documented in the case file and measured outside any diagram like this one.
  • The population-scale privacy exposure is represented as a starting gauge and pathway sensitivity, not a measured collection or egress rate. The scenario models the withdrawn exploitation models' shape; the NEET model and the database itself remain active and advisory.

Sources and evidence

What this example rests on, claim by claim. Every entry resolves to the same ledger the Evidence Registry publishes.

  • Bristol's Think Family Database drew on roughly 30 to 35 fused council, police, and other datasets covering about 55,000 families (some 170,000 residents in 2021 reporting), and its child sexual and criminal exploitation risk models were quietly withdrawn in 2023 as 'not fit for operational use' after an independent evaluation judged the risk-scoring models the weakest element and staff reported victims of exploitation scoring below people involved in burglary; FOI responses indicate no record was kept of why the models were switched off, and auditors could not locate their source code or variable lists.

    empirical
    • Investigative Sean Morrison, The Bristol Cable with Liberty Investigates, Lighthouse Reports and WIRED, Bristol data tools risked wrongly flagging victims and suspects, Children's Commissioner deeply concerned (2026) https://thebristolcable.org/2026/06/bristol-data-tools-risked-wrongly-flagging-victims-and-suspects-childrens-commissioner-deeply-concerned/
    • Investigative Mark Wilding and Matt Burgess, Liberty Investigates and WIRED, Police built a sprawling crime-prediction machine. Some results couldn't be trusted (2026) https://libertyinvestigates.org.uk/articles/predictive-policing-avon-somerset-bristol-police-ai-minority-report/
    • Investigative Sean Morrison, The Bristol Cable, Surveillance isn't safeguarding: Think Family and the fight for transparency (2026) https://thebristolcable.org/2026/01/think-family-education-data-gathering-fight-for-transparency/
    • Government Bristol City Council, Insight Bristol and the Think Family Database (2025) https://www.bristol.gov.uk/residents/social-care-and-health/children-and-families/insight-bristol
    • Investigative Jake Hurfurt (Big Brother Watch), The Bristol Cable, How a police and council database is predicting if your child is at risk of harm (2021) https://thebristolcable.org/2021/07/how-a-police-and-council-database-is-predicting-if-your-child-is-at-risk-of-harm/

  • Reporting and FOI responses on Bristol's Think Family Database indicate the exploitation models' source code and variable lists could not be located when auditors sought them, and that an ethics committee advising the police analytics reportedly did not revisit the analytics after 2017; a 2021 review warned that data gathered through 'legal gateways' meant 'legality is not the same as legitimacy.'

    empirical
    • Investigative Mark Wilding and Matt Burgess, Liberty Investigates and WIRED, Police built a sprawling crime-prediction machine. Some results couldn't be trusted (2026) https://libertyinvestigates.org.uk/articles/predictive-policing-avon-somerset-bristol-police-ai-minority-report/
    • Investigative Sean Morrison, The Bristol Cable with Liberty Investigates, Lighthouse Reports and WIRED, Bristol data tools risked wrongly flagging victims and suspects, Children's Commissioner deeply concerned (2026) https://thebristolcable.org/2026/06/bristol-data-tools-risked-wrongly-flagging-victims-and-suspects-childrens-commissioner-deeply-concerned/
    • Investigative Sean Morrison, The Bristol Cable, Surveillance isn't safeguarding: Think Family and the fight for transparency (2026) https://thebristolcable.org/2026/01/think-family-education-data-gathering-fight-for-transparency/

Where this connects

Institutional pressures in this domain

  • Workload surge — Demand outruns staffing; per-case attention shrinks and review becomes triage.
  • Deadline pressure — Statutory or managerial timeliness rules reward fast approval of machine output over slow disagreement.
  • Staff turnover — Experienced skepticism leaves; new staff calibrate their trust on the tool itself.
  • Data & policy drift — The world, the intake process, and the rules change under a system trained on how things used to be — two mechanisms with different remedies: the statistical properties of what the system processes move (concept drift), or the mixture of inputs arriving in deployment differs from the mixture it was trained on (covariate shift).
  • Compliance over substance — Paper controls (sign-offs, checklists) satisfy audits while the behavior they describe erodes.

All of them in context on the Child welfare & family services domain page.

Levers available here and the patterns behind them

Documented case histories