Behavioral-health & crisis triage

A national VA model flags the 0.1% of patients at highest suicide risk each month — a working, coordinator-mediated program that improved appointments and safety plans without reducing suicide deaths in two evaluations, and that a 2024 investigation found under-flagged women by leaving their risk factors out of the model.

A suicide-risk model that scores routine hospital records made clinicians far more likely to screen a flagged patient — but only when its alert was made interruptive rather than passive, in a trial too small to show a single prevented suicide.

An EHR-embedded machine-learning score that flags high suicide-risk patients within about 30 minutes of a virtual mental-health intake and routes them into the same assessment-and-outreach workflow as a positive self-report screen — a carefully co-designed integration whose main effect is to add a very-low-PPV sensor to an existing human screen, published as a feasibility design with no outcome-effectiveness evaluation.

A national crisis line built an in-house model to rank its most-at-risk texters and reorder who gets a counselor first — while the same anonymized conversations became a corpus that was routed to a for-profit it owned a stake in, to train commercial software. A 2022 exposé ended the data-sharing in three days; the deeper failure was that the review body that should have gated it was reportedly never asked, and that consent was gathered at the moment of crisis via a Terms of Service.

A proprietary overdose-risk score embedded in most US prescription-monitoring programs assigns patients secret 000-999 numbers that can gate their access to pain medication — never independently validated, not regulated by the FDA, uncontestable by the patient, and, when independent researchers rebuilt a version of it in 2026, far less accurate than the vendor claims.

A self-referral triage chatbot that genuinely widened access to NHS talking therapies — with the largest gains among under-served groups — while nearly all the evidence that it works was measured, published, and acted on by the company that built it.

A rule-based CBT chatbot used by roughly 1.5 million people was deliberately retired on a published schedule — with a transcript-access window and scheduled data anonymization — because its maker judged the FDA marketing-authorization economics unsustainable, not because the tool failed clinically. The Atlas's reference point for what a responsible wind-down looks like.

An AI quality-assurance tool that scores 988 crisis counselors on their own call practice — not the callers — expanding measured review from under 3% of calls toward nearly all of them. The domain's positive, governed counterexample: the AI sits on the supervision link over the operators, with strong peer-reviewed reliability evidence but a vendor-authored, still-unpublished trial of whether its feedback actually helps.

Gaggle Safety Management, an AI service, scans everything students write on school-issued accounts — email, homework, art, the student newspaper — around the clock, flagging anything that might signal self-harm. Used by roughly 1,500 US districts covering about 6 million students, it works as a multi-hop chain: a machine flags, an off-site vendor reviewer triages, a counselor gets an alert, and an after-hours emergency flag can send police to a home. In Vancouver, Washington nearly 1 in 10 students triggered alerts in a year; in Lawrence, Kansas about two-thirds of 1,200-plus flags were deemed nonissues — and the archive of flagged suicide notes and coming-out essays was accidentally released, nearly 3,500 unredacted documents at once. The atlas's discontinuation-that-wasn't case: when nine students sued, Lawrence quit the vendor mid-lawsuit and quietly swapped in an equivalent one with no board vote, and a federal judge found the district broke open-records law trying to withhold the swap's paperwork from the student journalists it had been flagging.

Camera-based, contact-free vital-signs monitoring installed in NHS psychiatric bedrooms. In March 2026 the health ombudsman partly upheld a complaint that a trust never sought a patient's consent, gave her no information, and would not switch the camera off when she asked — and found even the revised procedure still lets staff film a refusing, capacitous patient. The defining defect is vendor capture of the evaluation link: the company selling the tool authored the trust's business case and shaped the evidence meant to vouch for it, then rebranded mid-inquiry. Contested throughout; an ICO investigation is open.

ODMAP, a nationwide rule-based early-warning network, fires overdose spike alerts from aggregate, unnamed event counts rather than scoring any individual. Its algorithm is a plain threshold, not machine learning; the hard questions live off the model and on the links - the shared store sits inside a federal drug-enforcement program where public-health and law-enforcement read access coexist, the threshold learns from the store's own reporting, and every figure is program-self-published with no independent evaluation.