PAN Lab: govern the network before failure spreads

The Lab speaks in pathways, pressures, levers, and gauges. This map connects that vocabulary to the formal failure-mode names used in the research grounding it — including a 2026 national survey of 1,179 U.S.-based social workers.

• Automation bias / overreliancecore

  The “Failures adopted by people or agents” pathway and the operator-deference-drift gauge. Staff turnover and autonomy expansion push it up; the deskilling-arrest lever caps it. Deference can also rise where accountability sits rather than where trust does: when following the tool is the defensible act, a worker who distrusts it may follow it anyway — and a training lever does not reach that driver.

  Evidence: In the 2025–2026 University of Texas at Austin / NASW national survey of U.S. social workers, 40.8% of respondents reported ethical concerns about relying on AI for decision-making, and overreliance on automated decision-making was among the most frequently cited concerns overall.

  Evidence: The same chapter reports that workers who distrust a screening tool may still follow it, because organizational and policy pressure makes following the tool the defensible act. Deference on this account is produced by where accountability sits, not only by how much the worker trusts the output.

  Evidence: The volume's child-welfare chapter reports that a widely deployed screening score predicts whether a child will be placed out of the home within two years, which is the system's own future response rather than the maltreatment the worker is deciding about. The chapter treats the gap between the modelled target and the decision's actual question as a design property of the deployment, not as a defect in the model's accuracy.

• Deskilling / professional judgment erosioncore

  Overreliance in slow motion: the deference gauge drifting upward while correction capacity thins. The deskilling-arrest lever is its deliberate counter-schedule.

• Sycophancy / agreement-seeking outputcore

  The pushback-heavy-usage pressure runs the “Operator framing biases the model” pathway hot and makes the agreeable answers easier to adopt; the framing-hygiene lever dampens the loop at its origin.

  Evidence: Research on AI sycophancy describes it as a fragmented construct — a family of distinct agreement-seeking behaviors that share a label but differ in form, mechanism, measurement, and required mitigation — and finds it intensifies under user pushback and across multi-turn interaction.

• Hallucination / incorrect-output propagationcore

  The Lab's core premise: every pathway in the diagram carries incorrect output away from its source, and every lever is a way of governing that propagation rather than assuming a perfect model.

• Unsafe data flow / privacy & confidentialitycore

  Modeled as pathways, gauged as exposure (Phase NP). Unsanctioned tool use opens a visible egress to the off-network sink; connector sprawl replicates an unverified cache between record systems; case-file-flagged pathways (MiDAS-class enforcement replication, records feeding vendor models) carry the same concern. While any such pathway runs, the Privacy gauge drains — and in Hard and Expert a full gauge is part of the win. “Vet connections” cuts the pathways structurally; “Store less data” shrinks what there is to expose.

  Evidence: In the 2025–2026 University of Texas at Austin / NASW national survey of U.S. social workers, concerns about data privacy and security were the most frequently reported challenge to using AI in practice (46.5% of respondents), and an increased focus on client privacy and confidentiality was the most requested improvement to AI tools for social work (50.4%).

• Bias propagation / institutional workflow biascore

  Biased framings and contaminated records travel the same workflow pathways failures do — an institutional propagation question, and the documented cases show the workflow (not the model alone) carrying the equity outcome in both directions. The Lab models no demographics: differential harm to served people is recorded outside the network, never computed from its dynamics.

  Evidence: Evaluation evidence on the Allegheny Family Screening Tool found that screener overrides of the tool's recommendations reduced racial disparity in screen-in rates relative to the tool alone.

  Evidence: Independent scrutiny of Rotterdam's welfare-fraud risk model — a 2021 municipal audit followed by a 2023 journalistic investigation that obtained the model itself — documented scores skewed against already-vulnerable groups, and the city suspended the system's use.

• Transparency / provenance failurecore

  The record-contamination gauge reads how much unlabeled machine content feeds back into decisions; “Mark AI-written records” discounts it and “Gate vendor updates” attacks opacity at procurement.

• Weak human oversight / safeguard failurecore

  The scenario axis itself: one model, three oversight cultures, three very different outcomes. The correction and authority gauges track it; “Review the riskiest first”, “Pause AI on alarms”, “Require sign-off”, and “Review on schedule” govern it. A control drawn on one of these diagrams is drawn as present, which is a claim about structure — not a claim that anyone exercises it.

  Evidence: In the 2025–2026 University of Texas at Austin / NASW national survey of U.S. social workers, 42.1% of respondents reported having no role in decision-making about AI adoption in their workplace; the report concludes most respondents have limited or no control over how AI technologies are selected or implemented within their organizations.

  Evidence: The volume's ethics chapter names ethics washing as addressing ethical concerns superficially, to gain public trust, while making no substantive change to practice. It identifies three forms this takes: ethical statements that are vague or go unenforced, ethics boards constituted with limited authority, and adoption of frameworks that carry no accountability mechanism. The chapter offers this as a taxonomy of forms, not as a measurement of how often each occurs.

• Low AI literacy / verification readinesscore

  The AI-literacy-gap pressure: verification skill (not time) drops and deference rises as trust calibrates on the tool itself. The correction-capacity gauge reads the result. The grounding literature proposes AI literacy as a core professional competency.

  Evidence: The 2025–2026 University of Texas at Austin / NASW national survey of U.S. social workers describes a gap between AI exposure and AI preparedness: 26.6% of respondents cited lack of training or understanding of AI technology as a challenge, 53.4% said training on AI tools and effective use would help, and clear guidelines on the ethical use of AI were the most-endorsed need (66.8%).

  Evidence: AI literacy — the knowledge and skills required to understand, use, and critically evaluate AI systems — has been proposed as a core competency for social work, relevant even to practitioners who never directly use AI tools.

• AI iatrogenics / governance backfireadvanced

  First-class here: purging records without reading them backfires exactly as the sociotechnical simulation found, and the efficiency readout will call a lever stack counterproductive to its face. The quieter trap — oversight whose gains are bought by rising deference — is why the deskilling-arrest lever exists.

  Evidence: In the sociotechnical simulation, deleting records without reading them raised the contaminated share by stripping out benign entries; only content-aware cleanup reliably reduced it. (PAN governance-lever audit)

• Monitoring failure / drift blindnessadvanced

  Two pressures carry it: the silent vendor update (drift arriving under controls tuned to old behavior) and monitoring going stale (dashboards nobody must act on — the authority gauge hollows while the regime worsens). “Review on schedule” and “Escalate checks” are the counters. Reported accuracy is where this mode hides: a held-out figure and a figure from another site are different quantities, and discrimination statistics are not the rates at which error spreads or gets caught.

  Evidence: How a model scores on data held back from its own training and how it scores at a different site are different quantities. The volume's disability chapter reports a named pair where the second is materially lower than the first. A figure quoted without saying which of the two it is does not tell a reader what the model will do in their setting.

  Evidence: The volume's substance-use chapter reports that nearly all models in the field it reviews are developed and evaluated on a single dataset. Where that holds, a reported ceiling describes that sample rather than a portable capability, and it should be read as the best case observed on one collection of records, not as what the tool will do elsewhere.

  Evidence: Discrimination statistics such as AUROC, precision, F1 and lift describe how a model separates cases on a labelled sample. They are not per-interaction rates at which an error is adopted, written into a record, or corrected. The volume's research chapter treats these as different quantities, and they must never be entered into a diagram as though one substitutes for the other.

• Service starvation / over-throttlingadvanced

  The sixth iatrogenic, symmetric to deference-load: governance so tight the work stops. The “Work getting done” gauge reads strained and the Net AI benefit track sits on the hurting side while the failure regime reads contained — a breaker or write-gate has zeroed an assistive arm, or checking layers have throttled it to a satisficing region where the tool is compliant but not constructive. It is a Lab-only service quantity, distinct from PAN's harm-reduction: you paid budget, deference, and compute for a tool your governance won't let help.

  Evidence: Safety-only alignment establishes a behavioral floor without a ceiling: systems can be 'not-unsafe' yet directionless — compliant without being constructive — and benefit must be assessed as scaffold versus crutch.

  Evidence: In a two-year child-welfare ethnography, an ill-fitting algorithmic tool imposed ongoing repair work on caseworkers — anticipatorily editing the inputs they supplied so the tool would return a usable result, and bending or working around procedure to reconcile its output with the case in front of them — labor spent making a poorly-suited tool usable rather than on the casework itself, distinct from any deliberate checking of the output.

• Repair work / tool-usability laboradvanced

  The drag the Work-quality component reads when a contaminated or ill-fitting tool imposes ongoing labor to make its output usable — anticipatory editing of inputs, bending procedure to reconcile the result — spent making the tool usable rather than on the casework itself, distinct from the deliberate checking priced as review latency.

  Evidence: In a two-year child-welfare ethnography, an ill-fitting algorithmic tool imposed ongoing repair work on caseworkers — anticipatorily editing the inputs they supplied so the tool would return a usable result, and bending or working around procedure to reconcile its output with the case in front of them — labor spent making a poorly-suited tool usable rather than on the casework itself, distinct from any deliberate checking of the output.

• Authority-capacity mismatch / oversight assigned without capacityadvanced

  Adjacent to two core modes and distinct from both: automation bias and low AI literacy are about the checker's skill, while this is about the distance between where authority sits and where capacity sits. A sign-off requirement names who answers for a decision; it does not create the class of people who would have to catch the error. On these diagrams that reads as the authority gauge registering a control the correction-capacity gauge cannot staff, with the deference gauge free to drift underneath it. “Require sign-off” and “Review the riskiest first” place the duty; the training levers the catalogue already ships are what would move the capacity. This entry adds no lever and no pressure — it is a reading rule over gauges the Lab already has.

  Evidence: Two chapters of the volume describe the same gap from opposite ends. The governance chapter names an ethical capacity gap: many social workers have not been trained in data science or AI oversight, which it argues leaves them ill-prepared to question or interpret the algorithmic outputs they are nonetheless answerable for. The literacy chapter's professional-development framework assigns audiences by tier, placing sanctioned-tool lists, ethics review boards and vendor bias-mitigation terms with agency leadership while the skill to audit a decision and advocate for a misclassified client is taught at the tier below; it also reports professional-body guidance placing the duty to train on the employer rather than on the individual practitioner. Both are arguments about where authority sits relative to capacity. Neither reports a measured rate of either.

• Purpose creep / function creepadvanced

  One family seen from two ends. A model leaves the scope it was published under, so an instrument built to study population-level risk gets read as a judgment about one person; and a store acquires a class of reader it was not collected for, so a school record kept for support is read for discipline or handed to an outside agency. The nearest surfaces today are honest about their reach: “Vet connections” closes pathways between systems structurally, “Store less data” shrinks what is there to reuse, and the scope-filter mediator ships as a starter default with no cited delta. None of them models a change in WHO may read a store that stays exactly where it is. Nothing here draws a perpetration or lethality predictor, or any risk threshold over students; the mode is crosswalked because a governance diagram blind to a scope change cannot govern one.

  Evidence: The volume's sexual and partner violence chapter draws a scope line around predictive risk modelling and restates it twice: such models are used in research contexts to study population-level risk factors, they are not intended for individual-level decision-making in clinical or legal settings, and they are not intended for practitioners to screen or label individuals directly in real-world settings. The chapter treats the distance between that declared scope and a case-level use as a central governance danger rather than as a modelling defect, and reports no measurement of how often the line is crossed.

  Evidence: The volume's school social work chapter names function creep, a term it takes from Koops (2021), as the long-term risk that data collected for a beneficial purpose such as identifying mental health needs is later repurposed for an entirely different function; it names student discipline and sharing with external law-enforcement agencies as its examples. The chapter's stated concern is the absence of specific, renewed consent for the new use and the erosion of trust that follows, not the volume of data held. It offers this as a risk argument and reports no incidence rate.

  Evidence: Under the GDPR (Regulation (EU) 2016/679, Article 5), personal data must be collected for specified purposes and not further processed in a way incompatible with them (purpose limitation), and kept adequate, relevant, and limited to what is necessary (data minimisation).

• Environmental burden (external context)external context

  Deliberately outside the network: nothing in this Lab computes environmental cost, and no gauge claims to. Practitioner concern about AI's environmental impact is documented in the survey evidence and belongs in deployment governance as external context.

  Evidence: In the open-ended comments of the 2025–2026 University of Texas at Austin / NASW national survey of U.S. social workers, ethical concerns — prominently including the environmental impact of AI infrastructure — were the most common theme, and the report's first recommendation includes environmental impact among the topics profession-wide ethical guidance should address.

• Coverage exclusion / exclusion error in targetingexternal context

  Deliberately outside the network, and the boundary is the point. When the output is a ranked worklist steering a scarce resource, the people that list does not name enter no pathway drawn here, no gauge reads them, and none should: these diagrams govern how a decision travels once a system has produced one, not who the system was able to consider in the first place. The targeting literature calls the two directions inclusion error and exclusion error, and only the first leaves a record inside the organization to correct against. Where a deployment has measured the second, the figure belongs to whoever measured it and is recorded in that case file — the Los Angeles homelessness-prevention unit's own equity audit is the shipped example. It is never computed from these dynamics, and no gauge here is a fairness metric.

  Evidence: The volume's poverty chapter uses the targeting literature's paired vocabulary for the two directions in which a system steering a scarce resource fails: an inclusion error reaches someone the program did not intend to reach, and an exclusion error leaves out someone it did intend to reach. The chapter reports reducing both as the stated aim of machine-learning-assisted eligibility and proxy-means targeting. It is a narrative review and measures neither rate itself; the accuracy results it summarises belong to its sources.

  Evidence: The Homelessness Prevention Unit's own November 2024 equity audit, on a test population of 47,582 individuals eligible to be scored, reported false-negative rates ranging from about 56% for Black individuals to roughly 63 to 65% for other groups: the model misses a majority of the people who later become homeless, while performing roughly consistently across race, ethnicity, and gender and identifying Black individuals slightly more strongly.