Validating the Crystal Ball: Retrospective Evaluation and Uncertainty-Aware Forecasting of Severe Housing Cost Burden in New York City

<title>Abstract</title> <p>Machine-learning forecasts of housing cost burden are increasingly used to guide resource allocation, yet their out-of-sample accuracy is rarely tested against realized outcomes. We provide the first retrospective validation of such forecasts for New York City. Building on the NTA-level ensemble panel model of Rahman et al. [1], we extend the panel to 2012-2025, re-run the original forecasting protocol exactly as published, and score its 2023–2025 borough forecasts against realized ACS 5-year estimates. The original forecasts were sys tematically optimistic: the mean absolute borough-year error was 3.4 percentage points, and every borough was under-predicted, with the bias widening from −2.0 pp in 2023 to −4.7 pp in 2025 and the Bronx diverging most sharply (−5.4 pp by 2025). We diagnose the failure as a static-baseline assumption (+3.0% rent, +2.5% income growth) that could not capture post-COVID rent acceleration and income stagnation, and we recalibrate the framework with (i) split-conformal prediction intervals, (ii) ACS margin-of-error propaga tion via bootstrap, (iii) a leave-one-group-out ablation that quantifies the contribution of building-level features (rent stabilization, NYCHA inventory, HPD complaints), and (iv) scenario analysis replacing the single static baseline. The validated 2025–2028 forecasts carry 90% uncertainty bands and imply that, even under rent–income parity, severe bur den remains above 2022 levels in every borough; under adverse rent growth, Bronx severe burden approaches 45% of renter households. The framework is fully reproducible with a calibrated synthetic panel and is directly portable to other cities with ACS and market-rent data.</p>