Predicting Renal Cell Carcinoma Recurrence With Residual Regression-Augmented Language Models for Radiomic Interpretation: Retrospective Cohort Study (Preprint)

<sec> <title>BACKGROUND</title> <p>Renal cell carcinoma (RCC) recurrence after definitive therapy remains difficult to predict using conventional clinicopathologic risk models. Computed tomography (CT), electronic health record data, and radiology text may contain complementary prognostic signals, but many multimodal models are difficult to interpret.</p> </sec> <sec> <title>OBJECTIVE</title> <p>This study aimed to develop and evaluate an interpretable multimodal model that uses residual regression to identify imaging information not captured by known semantic features and to predict RCC recurrence after surgery.</p> </sec> <sec> <title>METHODS</title> <p>We conducted a retrospective cohort study using Mass General Brigham data from January 1, 2000, through September 1, 2022, with the public DeepLesion data set used for pretraining. The final RCC imaging cohort included 1615 patients. A CT contrastive language-image pretraining model aligned imaging and text embeddings; residual regression isolated an imaging component orthogonal to concept unique identifier–based text features. Baseline clinical, radiomic, natural language processing, and residual features were combined in XGBoost Cox models. Models were trained using algorithmically derived recurrence labels and evaluated using an expert-annotated subset. Discrimination was assessed at 12, 60, and 120 months, and predicted risk groups were compared using time-to-event analyses.</p> </sec> <sec> <title>RESULTS</title> <p>The ensemble model achieved area under the receiver operating characteristic curve values of 0.86 (95% CI 0.85-0.87) at 12 months, 0.78 (95% CI 0.77-0.80) at 60 months, and 0.82 (95% CI 0.81-0.83) at 120 months. The corresponding baseline-only values were 0.68 (95% CI 0.66-0.71), 0.73 (95% CI 0.72-0.75), and 0.79 (95% CI 0.78-0.81). At 60 months, ensemble-defined high- and low-risk groups had a hazard ratio of 6.34 (95% CI 2.70-14.91; P&lt;.001). Residual features mapped to interpretable imaging phenotypes, including necrosis, infiltrative growth, and vascular characteristics.</p> </sec> <sec> <title>CONCLUSIONS</title> <p>Residual regression–derived imaging features added prognostic information to clinical, radiomic, and text-derived features while retaining an interpretable link to imaging phenotypes. External and prospective validation is needed before clinical use.</p> </sec>