Skip to content
ParamergeParamerge
Stephen Lieberman
Through Paramerge
Contact

PAN Lab example

Robodebt (Australia)

The debt you must disprove: an income-averaging engine

An income-averaging engine raises a debt by spreading a year of tax-office income evenly across fortnights, then places the burden on the recipient to disprove it while recovery acts automatically. Modeled on Australia's Robodebt scheme. The correction the system needed sat outside it: no internal actor held a pre-authorized halt, so the only binding controls arrived years late, from the courts and a Royal Commission.

Stylized model of a documented deploymentPublic benefits & eligibility

Open this example in PAN Lab v0.1 to apply pressures and levers and watch what the system does.

What this models

This example runs on the Robodebt-class income-averaging debt engine network: 5 components and 12 pathways between them. Every context in the Lab is a stylized model, never a reconstruction of any actual deployment, and each assumption behind it carries a provenance label.

Evidence base: 4 assumed · 3 published baseline. In the Lab, the shaded evidence band behind each headline readout draws its width from the least-established class below.

  • assumed

    This models the income-averaging, reverse-onus automated-debt pattern documented in the Robodebt (Australia) case file — not a reconstruction of the actual system.

  • baseline

    The signature move is the reversed burden of proof: the engine's averaged figure was treated as an established debt, and the onus to disprove it was placed on recipients rather than the agency. The model-to-officer adoption edge carries that reading, and the internal halt / second-look check is drawn dormant because the contest was externalized onto the least-resourced actors in the system.

  • baseline

    The internal-review node represents the actor positioned to question or halt the scheme. Its halt pathway is drawn dormant at baseline: warnings reached senior levels but no pre-authorized circuit-breaker let anyone stop the engine, and the only binding controls — the courts and the Royal Commission — sat outside the institution and arrived years late. Those external events are recorded in the case file, not computed here.

  • baseline

    One averaging rule was applied to every recipient, so a single unlawful method produced correlated wrongful debts at national scale rather than scattered errors; the model-to-model self-loop encodes that correlated reach.

  • assumed

    The register-to-recovery replication is drawn strong and privacy-sensitive: a raised debt was actioned through garnishment, tax-refund interception, and external collectors before any independent review, and no step reconciled a recovery action against whether the underlying debt had ever been established.

  • assumed

    Prior raised debts re-entering later assessments is drawn low: the documented core failure was the averaging method and the reverse onus, not a strong retraining loop.

  • assumed

    Robodebt's harm fell on welfare recipients — people on low incomes, students, and people with disability — and its documented human toll is recorded in the case file and its sources, hedged as they hedge. This Lab models institutional propagation, not that harm; recipients are not represented as a node and no differential harm to served people is estimated.

What this example does not show

  • Robodebt's harm fell on welfare recipients — people on low incomes, students, and people with disability — and the documented human toll is recorded in the case file and its sources, hedged as they hedge. This Lab models institutional propagation, not that harm, and estimates no differential harm to served people; recipients are not represented in the diagram.

Sources and evidence

What this example rests on, claim by claim. Every entry resolves to the same ledger the Evidence Registry publishes.

  • The Royal Commission into the Robodebt Scheme documented hundreds of thousands of wrongful debts raised by an unlawful income-averaging method, with the onus placed on recipients to disprove automated assessments.

    empirical
    • Government Royal Commission into the Robodebt Scheme (2023) https://robodebt.royalcommission.gov.au/
    • Government Royal Commission into the Robodebt Scheme, Report (2023) https://robodebt.royalcommission.gov.au/publications/report
    • Government Prygodicz v Commonwealth of Australia (No 2) [2021] FCA 634 (Federal Court of Australia) https://robodebt.royalcommission.gov.au/publications/exhibit-2-2598-rbd999900010225-prygodicz-v-commonwealth-australia-no-2-2021-fca-634
    • Investigative Law Society Journal, Crude, cruel and unlawful: Robodebt findings https://lsj.com.au/articles/crude-cruel-and-unlawful-robodebt-royal-commission-findings/
    • Reference Royal Commission into the Robodebt Scheme (Wikipedia overview) https://en.wikipedia.org/wiki/Royal_Commission_into_the_Robodebt_Scheme

  • A single automated rule set applied uniformly and without human review produced tens of thousands of correlated wrongful fraud determinations in the documented Michigan MiDAS case — one flaw repeating at caseload scale rather than averaging out.

    empirical
    • Government Michigan AG, settlement of civil-rights class action (Bauserman, 2022) https://www.michigan.gov/ag/news/press-releases/2022/10/20/som-settlement-of-civil-rights-class-action-alleging-false-accusations-of-unemployment-fraud
    • Investigative IEEE Spectrum, Michigan's MiDAS unemployment system: Algorithm alchemy that created lead, not gold https://spectrum.ieee.org/michigans-midas-unemployment-system-algorithm-alchemy-that-created-lead-not-gold

Where this connects

Institutional pressures in this domain

  • Austerity & recovery incentives — Cost-cutting and overpayment-recovery targets tilt the system toward denial and enforcement errors.
  • Vendor opacity — The deploying institution cannot inspect the model, data, or update pipeline it is accountable for.
  • Compliance over substance — Paper controls (sign-offs, checklists) satisfy audits while the behavior they describe erodes.
  • Reviewer bottleneck — One fixed-capacity checking stage sits between AI output and consequence; everything queues behind it.

All of them in context on the Public benefits & eligibility domain page.

Levers available here and the patterns behind them

Documented case histories