Leakage-Resistant Evaluation of Calibrated Multimodal Driver Drowsiness Detection Across Drivers and Routes

<title>Abstract</title> <p>Target leakage, adjacent-window overlap, participant mixing and evaluation on familiar driving routes can all inflate reported performance in driver-drowsiness detection. This study is a leakage-resistant secondary analysis of two author-supplied multimodal dataset families. The Driver Monitoring and Physiological Sensing (DMPS) workbooks include 225,000 observations from 15 drivers on highway, rough-terrain and dense-urban routes; the skin-conductance (SC) workbook includes 87,430 overlapping 30-s windows from 20 participants. The variables that were directly measured from the events and alternate encodings of the drowsiness target were removed prior to modeling. Logistic regression, pooled histogram-based gradient boosting (HGB) and calibrated quality-weighted modality fusion (CQMF) were assessed in the context of random-row, within-sequence time-blocked, driver-held-out and route-held-out protocols. Probabilities were calibrated on separate calibration subsets and evaluated with AUROC, AUPRC, F1, Matthews correlation coefficient, Brier score, expected calibration error, and missing-sensor stress tests. Under five-fold driver-held-out validation, pooled HGB achieved AUROC 0.9990 ± 0.0003, AUPRC 0.9819 ± 0.0041, F1 0.9342 ± 0.0094, MCC 0.9319 ± 0.0098, Brier score 0.00355 ± 0.00039, and expected calibration error 0.00149 ± 0.00121. If the whole route was left out, mean AUPRC dropped to 0.9686 and mean sensitivity fell to 0.7933, revealing a route-transfer penalty that was not seen when using random splitting. CQMF did not outperform pooled HGB (driver-held-out AUPRC 0.9062), indicating that separating the modalities lost useful cross-modal interactions. When sensor values were missing at random, pooled HGB remained the stronger model, although its AUPRC fell from 0.9819 to 0.6630 at 50% missingness. Held-out separability remained high in the SC sensitivity analysis after participants were separated; however, the supplied binary label was deterministically linked to KSS and raw acquisition streams were unavailable, which limits its value as external evidence. The results help to validate calibrated pooled multimodal modeling and illustrate the need for leakage auditing, route-held-out validation, and transparent provenance as prerequisites for credible safety claims.</p>