Opening the Black Box of Eviction Court with Document-Conditioned LLM Analysis

<title>Abstract</title> <p>Documents filed in civil court proceedings contain rich information about litigants' behavior, procedural events, and case outcomes, but the volume and heterogeneity of these records have frustrated previous efforts to study them at scale. This paper presents and validates a document-conditioned large language model (LLM) pipeline for measuring procedural and legal outcomes in eviction cases. The pipeline conditions LLM inference on the OCR-extracted text of individual court documents labeled with HTML metadata, constrains outputs to structured JSON schemas, and algorithmically reconciles document-level extractions into case-level measurements. We apply this pipeline to 195,050 documents from 8,503 eviction cases filed in Pierce County, Washington between 2022 and 2024, measuring twelve variables—including written responses, hearing attendance, legal representation, and court-ordered displacement—with 98.4% overall accuracy against a manually coded sample of 300 cases. Tasks requiring analysis of multiple documents proved most error-prone, and results broadly show that this approach is well suited to empirical research but insufficient for high-stakes legal analysis. Substantively, we depict tenants' participation in eviction proceedings in a high-volume urban court and engagement with the nation's first statewide right to counsel program. We find that the existence of such a program does not guarantee access to legal assistance—fewer than half of tenants submit a written response, and access to representation is strongly conditioned on early procedural engagement—and that the association between representation and case outcomes varies considerably across policy environments. These findings advance the growing body of empirical research on civil courts, evictions, and access to justice.</p>