Predicting antidepressive treatment outcome using machine learning analysis of EEG data: the importance of independent feature selection

<p>Background: Incorrect non-independent feature selection leads to inflation of machine learning model performance. The objective was to evaluate the impact of non-independent feature selection on model performance in predicting antidepressant response and suggest methodological improvements. Methods: This reproduction study used baseline EEG data from the Canadian Biomarker Integration Network in Depression (CAN-BIND-1, NCT04162522) non-randomized open-label trial (N=121). The goal of CAN-BIND-1 was to develop biomarkers to evaluate escitalopram treatment outcome. EEG features at baseline were extracted and response was defined as 50% symptom reduction after 8 weeks. Support vector machines were used and model performance was evaluated with balanced accuracy. Analyses compared model accuracy with feature selection on the whole sample or training set only. Additional analyses aimed to improve classification accuracy with data harmonization and the inclusion of additional EEG features. Sex prediction was evaluated to benchmark machine learning model performance. Results: Acceptable balanced accuracy on the test set was obtained with non-independent feature selection on the whole dataset (i.e., 72.1%), which dropped to chance level with independent feature selection on the training set only (i.e., 52.7%). Adding additional EEG features did not improve accuracy (50.4%), while performance for sex prediction did reach preferable estimates (76.3%).Conclusions: The potential value of EEG as biomarker for treatment outcome prediction can only be estimated with independent testing to correctly evaluate model performance. Data sharing enables independent validation of studies and facilitates the credibility of research results.</p>