SPS-LIME: Role-Aware Segment Perturbation for Explaining Long Legal Documents

<title>Abstract</title> <p>Legal text classifiers may achieve strong classification performance, but their practical value depends on whether their decisions can be explained through discourse units that correspond to recognizable forms of legal reasoning. Existing perturbation-based explanation methods commonly operate on individual tokens or sentences, which can fragment factual narratives, party arguments, and judicial reasoning that extend across multiple sentences. We introduce Semantic-Preserved Segment LIME, or SPS-LIME, a model-agnostic explanation framework that merges consecutive sentences assigned the same rhetorical role into variable-length semantic segments. These role-continuous segments serve as the units of perturbation and attribution, allowing explanations to better retain natural-language continuity and legal-semantic structure. SPS-LIME further localizes influential sentences within highly ranked segments through a coarse-to-fine explanation mechanism. We evaluate SPS-LIME on LexGLUE-SCOTUS and a document-level binary violation-classification task derived from the ECtHR corpus. Compared with LIME variants that perturb individual tokens or sentences, SPS-LIME achieves the highest mean Consistency and Comprehensiveness scores on both datasets and the lowest Sufficiency score on ECtHR. These results indicate that rhetorical-role-based segments provide a more stable and semantically continuous explanation space for long legal documents. The experiments also reveal that faithfulness to classifier behavior and legal relevance do not necessarily coincide. Influential segments identified by SPS-LIME may concentrate on factual or contextual passages rather than on rhetorical roles that explicitly express judicial reasoning or disposition. SPS-LIME therefore contributes both a discourse-aware explanation mechanism and a diagnostic framework for examining potential mismatches between classifier-sensitive content and role-based legal salience.</p>