Self-Supervised Multi-Modal Transformer for Early Brain Tumor Prediction Using Super-Resolution MRI and ICU Time-Series Data

<title>Abstract</title> <p>Early and accurate detection of brain tumors remains one of the most challenging and consequential problems in clinical neuroscience and critical care medicine. Conventional diagnostic pipelines rely heavily on expert radiological interpretation of single-modality MRI scans, often overlooking the rich temporal and physiological information embedded in Intensive Care Unit (ICU) monitoring records. This article presents the Self-Supervised Multi-Modal Transformer (SS-MMT), a novel deep learning framework that synergistically integrates super-resolution MRI reconstruction with ICU time-series physiological signals to achieve superior early brain tumor prediction. The proposed architecture employs a masked autoencoder pre-training strategy for self-supervised representation learning, followed by cross-modal attention-based fusion of spatial MRI features and temporal ICU biomarkers. A Generative Adversarial Network (GAN)-based super-resolution module enhances low-quality clinical MRI scans to 4× resolution before feature extraction. Extensive experiments on the BraTS-2023, MIMIC-IV-ICU, and TCGA-GBM datasets comprising 3,951 subjects demonstrate that SS-MMT achieves an AUC-ROC of 0.945, F1-score of 0.931, and sensitivity of 0.924, outperforming state-of-the-art methods by up to 6.2% in AUC. Ablation studies confirm the independent contribution of each modality and the self-supervised pre-training strategy. The proposed framework offers a clinically actionable, interpretable, and computationally efficient pathway toward automated neuro-oncological triage in ICU settings.</p>