Domain Atlas / Housing & homelessness services
Homebase Risk Assessment Questionnaire
New York City's Homebase homelessness-prevention program has since June 2012 routed applicant households on a 15-item Risk Assessment Questionnaire scored 0-25 (each answer worth 1-3 points), distilled by backward elimination from a Cox proportional-hazards model of shelter entry fitted to 11,105 families who applied October 2004 - June 2008 (12.8% entered shelter within three years; decile risk 1% to 37%); a total at or above 7 points routed a household to 'full' services (financial assistance, case management, legal and mediation referrals) rather than a 'brief' one-or-two-visit contact. It is a regression-derived additive point screener, not a machine-learning system: the city's statutory algorithmic-tool register files it under computation type 'Scoring', purpose 'Resource allocation', autonomy 'Monitored', frequency 'Daily', with 'Vendor(s): None' in the CY2025 entry. Against caseworker judgment, which had deemed 66.5% of applicants eligible, the instrument would have increased correct targeting of families entering shelter by 26% and cut misses by almost two-thirds at equivalent false-alarm rates. Services are delivered by seven contracted nonprofit providers across 26 neighborhood offices, and The Department of Homeless Services states the network serves more than 25,000 at-risk households a year. The causal effect evidence for the program comes from outside the deploying agency's own research office: a randomized controlled trial by Abt Associates, commissioned by the Department of Homeless Services (2010-2013, 295 families with children analyzed across eleven sites), found the share spending at least one night in shelter falling from 14.5% to 8.0%, the share applying for shelter falling from 18.2% to 9.3%, and average shelter nights falling by 22.6; an independent community-district difference-in-differences study estimated roughly 5-11% fewer family shelter entries (11.2 log points, 95% CI 3.6-18.8), a $14.2M annual budget avoiding an estimated $20-44M of shelter expenditure. The separately claimed 'prevention rate' of around 90-97% is a city performance metric with no counterfactual and is not an effect size. The threshold of 7 describes the documented pre-2023 configuration; no public source states the threshold in force after the 2023 item revision.[6]
What happened
New York City's Homebase homelessness-prevention program launched in 2004 and screened applicants on caseworker judgment. In June 2012 it began using a Risk Assessment Questionnaire instead: fifteen items, each answer worth one to three points, scored 0 to 25. The instrument came out of a study by Marybeth Shinn and colleagues of 11,105 families who applied to Homebase between October 2004 and June 2008, of whom 12.8% entered shelter within three years. A Cox proportional-hazards model was reduced by backward elimination to fifteen variables covering childhood disruption and adversity, leaseholder status, formal and informal eviction, landlord and household discord, moves in the past year, recent discharge from an institution, and prior shelter applications and stays. Shelter-entry risk ran from 1% in the lowest decile to 37% in the highest. Against caseworker judgment — which had deemed 66.5% of applicants eligible — the model would have increased correct targeting of families who entered shelter by 26% and cut misses by almost two-thirds at equivalent false-alarm rates. The authors cautioned that models are bound by time and place. A total at or above 7 points routed a household to "full" Homebase services: financial assistance, case management, legal and mediation referrals. Below it, a "brief" contact of one or two visits. This is a regression-derived additive point screener, not a machine-learning system; the city register files it under computation type "Scoring", purpose "Resource allocation", autonomy "Monitored", frequency of use "Daily".
Services are delivered by seven contracted nonprofit providers across 26 neighborhood offices, so the people who ask the questions, record the answers and assign the tier are contractor staff rather than city employees. The Department of Homeless Services states the network serves more than 25,000 at-risk households a year; the program's inflation-adjusted lifetime spend passed roughly a billion dollars by 2023. Two evaluations from outside the deploying agency's own research office found the program works. A randomized controlled trial by Abt Associates, commissioned by the Department of Homeless Services (2010-2013, 415 households randomized, 295 families with children analyzed across eleven sites), found the share of families spending at least one night in shelter falling from 14.5% to 8.0%, the share applying for shelter falling from 18.2% to 9.3%, and average shelter nights falling by 22.6. Goodman, Messeri and O'Flaherty's independent community-district difference-in-differences study estimated Homebase reduced family shelter entries by roughly 5-11% (11.2 log points, 95% CI 3.6-18.8), so a $14.2M annual budget avoided an estimated $20-44M of a roughly $400M family-shelter budget, averting an estimated 1,271-1,788 entries over four years. City materials separately claim a "prevention rate" around 90-97%; that is a performance metric with no counterfactual and is not an effect size.
Local Law 35 of 2022 (passed by the Council 15 December 2021, lapsed into law unsigned 14 January 2022) added Admin. Code sec. 3-119.5, requiring every city agency to report by 31 December every "algorithmic tool" it used one or more times during the prior calendar year — a definition that expressly reaches tools which "generate risk scores" or "determine what resources are allocated to particular groups or individuals" — with six mandatory disclosure elements, compiled by the Office of Technology and Innovation into a public report delivered to the mayor and Council speaker each 31 March. DSS has filed the RAQ in all four cycles to date. Each entry states, in the agency's own words, the June 2012 start date, the 2004-2008 training data analyzed with academic researchers, the input factors, the points-and-threshold eligibility mechanism, and the rule that "workers are able to override a limited number of model decisions with permission of a supervisor". The vendor field varies across cycles: the CY2022 entry carries a narrative description of contracted researchers rather than a vendor name, CY2023 and CY2024 carry "Vendor Name: Multiple researchers", and CY2025 carries "Vendor(s): None". The loop demonstrably fired once: after the CY2023 report was published, DSS updated its filing — the register carries the note "Update 3/27/2024 - The Department of Social Services updated their reporting to include changes made to the Homebase Risk Assessment Questionnaire after initial publication of the report" — disclosing the 2023 item revision and adding its peer-reviewed citation. The Council's Committee on Technology held an oversight hearing on the regime on 28 October 2024, at which OTI described coordinating 45 agencies plus 24 further offices; a bill to create an Office of Algorithmic Data Integrity sat on the same agenda. The register entries are unaudited agency self-reports.
The most detailed evidence about the instrument comes from the agency itself. DSS's Office of Research & Policy Innovation published a peer-reviewed re-examination (Housing Policy Debate, 2022) of 48,450 deduplicated families with children who applied between 2013 and 2016, covering 58,674 family-years, over a period in which the caseload rose from about 600 cases a month in 2013 to more than 1,500 a month in some 2015 months. It reported that 73.9% of applications scored at or above the threshold of 7; that 13.7% of families at or above it applied for DHS shelter within two years against 5.9% below it (chi-square 699.98, p<.001); and that the deployed instrument's area under the curve was 0.7387 against 0.7389 for a revised version — the revision changed items, not discriminative power. It reported two distortions against itself. First, the totals cluster at the cutoff: 4,269 cases scored 6, 10,634 scored exactly 7, and 7,756 scored 8, which the agency's own researchers wrote "suggests that Homebase staff may be focused on getting families to that threshold so they qualify for full services" — while the agency separately asserts the override valve reduces incentives for workers to misreport data to ensure eligibility. Second, the overrides: 6.1% of the 58,674 applications departed from the score with supervisor approval, 4.8% moved up to full services and 1.4% moved down to brief. Only 3.7% of the families moved up later applied to shelter, against 25.8% of the families moved down — worker discretion under-calling risk in both directions, and the paper concludes worker judgment is less accurate than the RAQ at predicting risk of homelessness on average. A ten-case note review attributed many of the downward decisions to needs beyond the program's capacity, such as families needing an apartment immediately with no funds, rather than to a judgment about risk. The paper also simulated an alternative threshold of 5 on the revised score, reporting precision rising from 13.7% to 15.2% at similar enrollment volume. The 2023 revision changed the items; no public source indicates the threshold changed.
Three government audits have examined this program, and none examined the model. The city comptroller's MG12-125A (27 June 2013, FY2012 scope, $16.9M federal plus $778,469 city funds, 10,847 clients enrolled) found no written monitoring policies, no records of initial ineligibility determinations, and every provider risk assessment announced in advance; DHS rejected the unannounced-visit recommendations. The city comptroller's January 2020 audit of HRA's oversight of the then-$53M-a-year program found 80 of 240 required provider case-file reviews performed, 2,661 of 24,938 FY2018 households (11%) returning between one and four times inside twelve months with poor tracking, $2,271,797 in provider advances unrecouped some sixteen months after closeout, and 5 of 28 visited client homes not habitable, 4 of them never fixed; it made 19 recommendations. The state comptroller's audit 2023-N-8 (issued 7 January 2026, scope July 2021 to July 2025) examined the downstream CityFHEPS rental-subsidy channel for DSS Homebase clients — 57,888 new cases and 123,762 individuals housed since 2018 through March 2025, spending rising from $176M in FY2019 to $834M in FY2024 with roughly $1.2B projected for FY2025 — and found units with hazardous violations approved, 30 of 75 sampled case records without evidence of income verification, and rents averaging $525 a month above comparables in eleven of thirty sampled statewide cases. Independent corroboration outside the agency is thinner but real: solutions journalism documented the model's operational role in 2020, and a provider-affiliated study of 9,630 families admitted to one provider's three Bronx catchment areas in FY2017-2019 confirmed that the questionnaire's housing-instability items track shelter entry, found 16% admitted two or more times, and noted more than 40% "don't know" responses in the DSS race field.
The sociotechnical reading
The structural fact worth carrying out of this case is an inversion. In most of this atlas the algorithm is the opaque component and the human system around it is the part with the paperwork. Here it is the other way round. This screener has a statutory obligation to describe itself in public every year, four consecutive filings exist, a mid-cycle amendment forced a model revision into the public record that would otherwise have left no trace, and the agency's own research office peer-review-published the instrument's accuracy, its threshold-clustering signature, and the failure profile of its own caseworkers' overrides. The human system around it has none of that. The scorers are contractor staff at 26 offices under seven providers; the city's own review cycle over them ran at 80 of 240 required file reviews and had operated for years with no written monitoring policies; repeat clients were poorly tracked; and the downstream subsidy channel the screened case opens grew past $800M a year with income-verification evidence missing from 30 of 75 sampled records. Three government audits looked at this program in thirteen years and not one of them looked at the model. So the best-documented, most-evaluated, most-publicly-described component in the pipeline is the fifteen-question point total, and the parts nobody has measured are the ones with the money and the discretion in them.
The second reading is about what a transparency loop actually buys, and this is the clearest natural experiment in the atlas because the loop demonstrably fired and then stopped. Local Law 35 compels a description on a fixed date; it does not compel a change. When DSS revised the questionnaire in 2023 the register caught it, and the amendment note is the proof. But the same agency's own paper had shown that a lower cutoff would raise precision from 13.7% to 15.2%, and the change the register discloses is to the items; no public source indicates the threshold changed either way. The only discrimination comparison anyone has published is that paper's own, run before the revision shipped: 0.7387 for the deployed instrument against 0.7389 for the revised version it simulated. An evaluation loop fired and produced a real change, and what became of the lever with the largest documented effect is not in the public record at all, because nothing in the regime requires the agency to explain what it chose not to do. Meanwhile the distortions the agency documented run through people, not code. The totals pile up at exactly the qualifying score because the score is written in a conversation between a caseworker and a family in front of them, and the same distorted records feed the next re-examination, so the cohort that produced the 2023 revision was itself sorted by the version being revised. The override channel, which exists to catch what the instrument misses, was measured and found to be reading risk worse than the instrument in both directions — and then the ten-case note review found the reason: supervisors were moving families down not because they judged them low risk but because the program could not give them what they needed. That is a capacity constraint wearing the clothes of a judgment call, and no governance lever aimed at the model reaches it. The families themselves are outside every model here. What lands on them — a shelter application avoided or not, a voucher processed or not, a unit with hazardous violations approved — is documented in the trial and audit evidence above and is never computed from any diagram.
The concepts used in this reading are defined in the Field Guide; the governance responses live in the Practice Library.