Validation of a Machine Learning Brain Electrical Activity-Based Index to Aid in Diagnosing Concussion Among Athletes

Abstract

Importance: An objective, reliable indicator of the presence and severity of concussive brain injury and of the readiness for the return to activity has the potential to reduce concussion-related disability.

Objective: To validate the classification accuracy of a previously derived, machine learning, multimodal, brain electrical activity-based Concussion Index in an independent cohort of athletes with concussion.

Design, setting, and participants: This prospective diagnostic cohort study was conducted at 10 clinical sites (ie, US universities and high schools) between February 4, 2017, and March 20, 2019. A cohort comprising a consecutive sample of 207 athletes aged 13 to 25 years with concussion and 373 matched athlete controls without concussion were assessed with electroencephalography, cognitive testing, and symptom inventories within 72 hours of injury, at return to play, and 45 days after return to play. Variables from the multimodal assessment were used to generate a Concussion Index at each time point. Athletes with concussion had experienced a witnessed head impact, were removed from play for 5 days or more, and had an initial Glasgow Coma Scale score of 13 to 15. Participants were excluded for known neurologic disease or history within the last year of traumatic brain injury. Athlete controls were matched to athletes with concussion for age, sex, and type of sport played.

Main outcomes and measures: Classification accuracy of the Concussion Index at time of injury using a prespecified cutoff of 70 or less (total range, 0-100, where ≤70 indicates it is likely the individual has a concussion and >70 indicates it is likely the individual does not have a concussion).

Results: Of 580 eligible participants with analyzable data, 207 had concussion (124 male participants [59.9%]; mean [SD] age, 19.4 [2.5] years), and 373 were athlete controls (187 male participants [50.1%]; mean [SD] age, 19.6 [2.2] years). The Concussion Index had a sensitivity of 86.0% (95% CI, 80.5%-90.4%), specificity of 70.8% (95% CI, 65.9%-75.4%), negative predictive value of 90.1% (95% CI, 86.1%-93.3%), positive predictive value of 62.0% (95% CI, 56.1%-67.7%), and area under receiver operator characteristic curve of 0.89. At day 0, the mean (SD) Concussion Index among athletes with concussion was significantly lower than among athletes without concussion (75.0 [14.0] vs 32.7 [27.2]; P < .001). Among athletes with concussion, there was a significant increase in the Concussion Index between day 0 and return to play, with a mean (SD) paired difference between these time points of -41.2 (27.0) (P < .001).

Conclusions and relevance: These results suggest that the multimodal brain activity-based Concussion Index has high classification accuracy for identification of the likelihood of concussion at time of injury and may be associated with the return to control values at the time of recovery. The Concussion Index has the potential to aid in the clinical diagnosis of concussion and in the assessment of athletes' readiness to return to play.

Conflict of interest statement

Conflict of Interest Disclosures: All authors were principal investigators at one of the clinical data acquisition sites participating in the research, whose institutions received contracts to support participant recruitment, consenting, and data acquisition and have no competing financial interests to disclaim. Drs Bazarian, Casa, Lopez, Schnyer, Yeargin, and Covassin reported receiving grants from BrainScope Co Inc during the conduct of the study. Dr Bazarian reported receiving personal fees from Abbott and Q30 Innovations outside the submitted work. Dr Elbin reported receiving grants and consulting fees from BrainScope Co Inc during the conduct of the study. Dr Neville reported being an equity owner of Quadrant Biosciences Inc outside the submitted work. No other disclosures were reported.

Figures

Figure 1.. Flowchart of Participant Enrollment and Study Population

EEG indicates electroencephalogram; RTP, return to play.

Figure 2.. Longitudinal Change in Concussion Index Among Athletes With or Without Concussion

Concussion index value at day 0, return to play (RTP) and RTP+45 days, in the athletes with or without concussion. The dotted line indicates the threshold for the Concussion Index, where more than 70 is not concussed and 70 or less is concussed. Vertical lines indicate the 95% CI. aSignificance of the difference on day 0 between the mean Concussion Index among athletes with concussion compared with those without concussion (P < .001), with the Concussion Index significantly lower among the athletes with concussion. bAmong athletes without concussion, significant noninferiority (equivalence) in the mean Concussion Index between day 0 and RTP+45 days was found, with the Concussion Index of 78% of athletes with concussion exceeding the 90th percentile Concussion Index of athletes without concussion.