The cooperative audit: a paid source-code examination, and what happened to its verdict

Most of this catalogue is a record of things going wrong. This one is not, and that is why it is here.

In March 2020, Northeastern University and pymetrics, inc. signed a sponsored research agreement. pymetrics built game-based candidate screening: a job applicant plays a set of short games, and a model trained on the company's high performers decides which tier the applicant falls into. The agreement covered an audit of the pipeline that builds those models. It fixed the scope, the money, the publication rights and the confidentiality boundary before anyone started work.

Over the summer of 2020, four auditors worked inside a virtual machine pymetrics provisioned for them. They read the template notebook and the custom Python modules that implement the company's bias metric. They read eight engagement notebooks: one blank template, five drawn uniformly at random from the previous six months of client work, one chosen deliberately because that client had asked for extensive changes to the adverse-impact testing, and one complete engagement with its full pseudonymous training and evaluation data. They sat through an onboarding day, asked for technical help without explaining why they wanted it, requested supplemental datasets that were produced for them, and closed with a live demonstration of a model being trained, tested and deployed in front of them.

What they were examining is a real control, and it is worth stating precisely because very little else in algorithmic hiring can be stated this precisely. For each client role, pymetrics fitted a model to an in-group of 50 to 100 high-performing incumbents against an out-group sampled from a player database of more than two million people. Before that model could deploy, it was scored against a held-out "bias group" — typically more than ten thousand players, engineered to equal proportions across the seven EEOC categories, drawn from a pool of over six hundred thousand who had answered a demographic survey. The impact ratio was computed at the 50th and the 70th score percentile. It had to reach at least four-fifths at both. A model that did not was not adjusted until it passed: the engagement was reworked, or nothing shipped. That is an abandon-if-noncompliant gate, and it is rarer in this catalogue than any audit.

The findings were favourable. The four-fifths testing was correctly implemented across the seven categories. Demographic data was not used as a training feature and no overt proxies such as zip codes were present. Most usefully, the auditors tried to break it: they attempted to construct manipulated incumbent data that would yield a biased model without being flagged, and they could not, because every control-flow path in the code reached the adverse-impact tests. They looked at the human layer too — a completion checklist of more than a hundred items per model, plus a mandatory second data scientist before production — and judged it a reasonable safeguard against carelessness and against one person acting badly. They named what it leaves open: there is no programmatic back-end re-check of adverse impact after the data-scientist stage, so two insiders acting together is the path that remains. They checked whether median imputation was quietly doing damage, found statistically significant demographic differences in missing data, and found that it did not substantively move the adverse-impact results.

Their sentence was: pymetrics passed the audit, subject to the qualifications and limitations we state.

The qualifications are the other half of the case, and they were not discovered at the end. They were agreed at the start. The contracted scope excluded the choice of fairness objective and metric, the EEOC category set itself, intersectional groups, post-deployment back-testing, customised client processes, security, privacy compliance, the newer reasoning games — and construct validity, which is the question of whether the games measure anything a job actually needs. The auditors say so plainly: that question was beyond their capabilities as computer scientists. Intersectionality was excluded because those groups are not EEOC-recognised and, the paper records, all parties agreed the legal risk of optimising to a non-regulatory standard barred it; pymetrics itself had expressed interest in it. So the deepest examination this pipeline has ever had verified that the arithmetic was done correctly, and could not ask what the arithmetic was for.

The auditors were unusually careful about the shape of the deal, and unusually public about it. Payment ran as a grant to the university rather than to individuals, and it was delivered in full before findings, so it could not be conditioned on the result: $104,465, of which $64,813 was salaries for the team, figures independent journalism took from the budget the auditors themselves posted. pymetrics supplied the compute on top. The auditors kept their test methods secret from pymetrics throughout. A thirty-day responsible-disclosure window applied before any issue could be discussed publicly. They held final editorial discretion and the contractual right to publish results reflecting negatively on the company. And they posted the instruments: the contract, the work plan, the non-compete, the budget spreadsheet, and the final report with one section redacted for proprietary information.

They also named the thing. Their term for the arrangement is a cooperative audit, and they place it explicitly outside the internal/external taxonomy the field's own cited definitions offer: they were not employees, and they had privileged access to source code, data and staff. By those definitions, this was not an external audit.

Then the verdict left the building, and it got shorter each time it moved.

pymetrics had already publicised the engagement in a company post on 13 May 2020, before any result existed, framing the decision to invite outside experts into the codebase as a trust asset. After publication, the product was marketed as independently audited. When Harver announced its acquisition of pymetrics on 11 August 2022, terms undisclosed, the release described a platform mitigating multiple forms of bias through an audited AI platform. At no point in that sequence does the qualifying clause travel, and at no point does the auditors' own classification of the arrangement travel either.

Because the instruments were public, someone could read them back. At FAccT 2022, Young, Katell and Krafft made this engagement the lead case of what they call publishing and certifying corporate apologia. Their argument, and it is an argument rather than a finding: four of the paper's eight authors were pymetrics employees including the chief executive as last author; the company funded the work and set the scope, precluding various scenarios from scrutiny; and it seems unlikely, they write, that the paper would have been submitted had the results negated the firm's validation claims. They documented the marketing loop, noted that a co-author sat on the FAccT Executive Committee when the paper was accepted, and noted that 2021 FAccT review did not mandate funding disclosure. Contemporaneous critics had already put the narrower version of it: the audit asked whether pymetrics was doing what it says it does, not whether that was the right thing to do, and a four-fifths rule can pass a tool that discriminates against subgroups it does not test.

None of that is adjudicated. No court, regulator, board or conference process has ruled on any of it. There is no lawsuit, no enforcement action and no documented harm involving this vendor anywhere in this record. What there is, is a peer-reviewed argument about a channel, made possible by the fact that the channel published its own terms.

And then the channel changed shape entirely. New York City's Local Law 144 took effect in July 2023, and the same product now sits under an annual mandated bias audit performed by a paid third-party audit firm, BABL AI, with the summary published by whoever deploys the tool. The employer Paramount posted a summary dated 29 June 2023 for the pymetrics Soft Skills Platform; Harver posted one dated 17 July 2025 for what is now Harver's Soft Skills Platform — same games, same three recommendation tiers, same 50th-percentile threshold. The 2025 summary reports every calculated impact ratio at or above 0.8, the lowest non-intersectional one 0.914 for Asian candidates, computed across 164,014 male and 119,242 female candidates whose gender was known. It also reports, to its credit, what it could not compute over: 191,455 candidates of unknown gender, 317,513 of unknown race or ethnicity, and 331,177 with at least one unknown. The excluded unknown-race population is roughly twice the size of the included one.

That is honest arithmetic over the people who told the system who they are, and it is not proof of platform-wide compliance. It is also not a criticism of these particular audits, which were among the more complete of their kind. An ACLU-led study at FAccT 2025 read all 116 public audits filed under the mandate between July 2023 and November 2024 and found the regime as a whole producing incomplete evaluations of bias: missing demographic data, opaque aggregation, problematic test data, metrics that do not represent how the tools are used. It warns about audit washing, capture of auditors, and discrimination-hacking, and records that the four-fifths rule is a rule of thumb rather than the legal standard. It cites the pymetrics cooperative audit among the proposed audit standards the field built on, and the capture critique among its warnings.

So the arc, stated flat. A deep, voluntary, source-code-level examination with adversarial testing and a right to publish a negative result happened once, in one summer, on questions the audited party helped choose. A shallow, mandatory, arithmetic examination that verifies reported statistics and sees no code now happens every year, forever. The record contains both, and the honest reading is that neither is the other's replacement.