Skip to content
ParamergeParamerge
Stephen Lieberman
Through Paramerge
Contact

PAN Lab example

Hackney / Xantura Early Help Profiling

The pilot that quietly failed: a small council's family profiler

A small council buys a vendor tool that mines data across its own services to flag families for early help, and sends social workers a monthly list of those judged most at risk — while the families themselves are never told. Modeled on Hackney's Early Help Profiling System. The twist: this pilot was scrapped not after a scandal but on a cost-benefit judgment — the data was too poor to surface enough genuinely new cases, and everything that mattered sat upstream of the model: procurement, consent, and the quality of the records it ran on.

Stylized model of a documented deploymentChild welfare & family services

Open this example in PAN Lab v0.1 to apply pressures and levers and watch what the system does.

What this models

This example runs on the Hackney-class small-authority family-profiling pilot network: 5 components and 12 pathways between them. Every context in the Lab is a stylized model, never a reconstruction of any actual deployment, and each assumption behind it carries a provenance label.

Evidence base: 3 assumed · 4 published baseline. In the Lab, the shaded evidence band behind each headline readout draws its width from the least-established class below.

  • assumed

    This models the small-authority, vendor-built family-profiling pattern documented in the Hackney Early Help Profiling System case file — not a reconstruction of the actual tool.

  • baseline

    The model-to-social-worker adoption pathway is drawn modest and discretion is drawn high: the tool sent narrative summaries framed as a lead to consider, not a decision, and the documented failure was limited benefit — it surfaced only a handful of genuinely new cases — not over-reliance.

  • baseline

    The defining exposure is the no-consent reach: multi-agency data on families profiled without their knowledge and with no opt-out recorded in the impact assessment. That inflow is drawn strong and privacy-sensitive, while the pseudonymization-until-a-high-risk-alert design is drawn as a low, protective model-to-store write.

  • assumed

    No formal write-back of a persistent per-family score that then retrains the model is documented for Hackney: the model-to-store edge is drawn low, and the concern that predictive tools reinforce historical-record bias is a general expert critique in the sources, not a Hackney-specific measured loop.

  • baseline

    The independent-evaluation check and the family-transparency check are drawn on the map but run dry at baseline: no independent evaluation of the tool's accuracy or harm was published, the method was withheld as commercially sensitive, and data-sharing agreements were refused under freedom-of-information. Those absences are this case's signature, and levers can open them.

  • baseline

    One vendor engine would score every family in the borough, so a blind spot in it would be systematic rather than case-by-case; the model-to-model self-loop encodes that correlated reach.

  • assumed

    Demographics and differential harm to families are not modeled here. The documented concern centred on transparency, consent, and limited benefit; some evidence hints at a socioeconomic skew (a share of alerts involved families receiving benefits) but no independent demographic evaluation was published, and no differential harm to served people is estimated.

What this example does not show

  • The documented harm here is about transparency and consent, not a measured demographic disparity: families were profiled without being told. Some evidence hints at a socioeconomic skew (a share of alerts involved families receiving benefits), but no independent demographic evaluation was published. The Lab models institutional propagation, not demographics, and estimates no differential harm to served people; that harm, where it exists, is documented in the case file and measured outside any diagram like this one.
  • The system computed a risk score internally against a high-risk threshold, but presented social workers with narrative summaries rather than a number; nothing here implies workers were shown a raw numeric score.
  • The cost-savings figures associated with tools like this (a per-council annual saving, a per-child saving) were vendor or promotional projections, never outputs of an independent evaluation; the only published outcome was the council's own limited-benefit finding.

Sources and evidence

What this example rests on, claim by claim. Every entry resolves to the same ledger the Evidence Registry publishes.

  • Hackney paid the analytics firm Xantura £361,400 over four years to run an Early Help Profiling System that flagged families for preventive intervention from council data, but scrapped the pilot in 2019 after finding that, despite flagging about 350 families, it surfaced only 7 children previously unknown to the council and the available data was too limited and variable to justify continuing.

    empirical
    • Investigative Hackney Council pays 360k pounds to data firm whose software profiles troubled families, Hackney Citizen (18 October 2018) https://www.hackneycitizen.co.uk/2018/10/18/council-360k-xantura-software-profiles-troubled-families/
    • Investigative Town Hall drops pilot programme profiling families without their knowledge, Hackney Citizen (30 October 2019) https://www.hackneycitizen.co.uk/2019/10/30/town-hall-drops-pilot-programme-profiling-families-without-their-knowledge/

  • Families whose data Hackney's Early Help Profiling System processed were not informed directly: reporting describes families profiled without their knowledge, given notice only through a general online privacy notice, with no option to opt out recorded in the system's impact assessment and the method withheld as commercially sensitive; the council argued that disclosing the system could prejudice potential interventions.

    empirical
    • Investigative Town Hall drops pilot programme profiling families without their knowledge, Hackney Citizen (30 October 2019) https://www.hackneycitizen.co.uk/2019/10/30/town-hall-drops-pilot-programme-profiling-families-without-their-knowledge/
    • Academic Redden J., Dencik L. and Warne H., Datafied child welfare services: unpacking politics, economics and power, Policy Studies 41(5), 507-526 (2020), DOI 10.1080/01442872.2020.1724928 https://www.tandfonline.com/doi/full/10.1080/01442872.2020.1724928
    • Investigative Hackney Council pays 360k pounds to data firm whose software profiles troubled families, Hackney Citizen (18 October 2018) https://www.hackneycitizen.co.uk/2018/10/18/council-360k-xantura-software-profiles-troubled-families/

  • A single automated rule set applied uniformly and without human review produced tens of thousands of correlated wrongful fraud determinations in the documented Michigan MiDAS case — one flaw repeating at caseload scale rather than averaging out.

    empirical
    • Government Michigan AG, settlement of civil-rights class action (Bauserman, 2022) https://www.michigan.gov/ag/news/press-releases/2022/10/20/som-settlement-of-civil-rights-class-action-alleging-false-accusations-of-unemployment-fraud
    • Investigative IEEE Spectrum, Michigan's MiDAS unemployment system: Algorithm alchemy that created lead, not gold https://spectrum.ieee.org/michigans-midas-unemployment-system-algorithm-alchemy-that-created-lead-not-gold

Where this connects

Institutional pressures in this domain

  • Workload surge — Demand outruns staffing; per-case attention shrinks and review becomes triage.
  • Deadline pressure — Statutory or managerial timeliness rules reward fast approval of machine output over slow disagreement.
  • Staff turnover — Experienced skepticism leaves; new staff calibrate their trust on the tool itself.
  • Data & policy drift — The world, the intake process, and the rules change under a system trained on how things used to be — two mechanisms with different remedies: the statistical properties of what the system processes move (concept drift), or the mixture of inputs arriving in deployment differs from the mixture it was trained on (covariate shift).
  • Compliance over substance — Paper controls (sign-offs, checklists) satisfy audits while the behavior they describe erodes.

All of them in context on the Child welfare & family services domain page.

Levers available here and the patterns behind them

Documented case histories