Machine learning algorithms performed no better than regression models for prognostication in traumatic brain injury
Benjamin Y Gravesteijn, Daan Nieboer, Ari Ercole, Hester F Lingsma, David Nelson, Ben van Calster, Ewout W Steyerberg, CENTER-TBI collaborators, Cecilia Åkerlund, Krisztina Amrein, Nada Andelic, Lasse Andreassen, Audny Anke, Anna Antoni, Gérard Audibert, Philippe Azouvi, Maria Luisa Azzolini, Ronald Bartels, Pál Barzó, Romuald Beauvais, Ronny Beer, Bo-Michael Bellander, Antonio Belli, Habib Benali, Maurizio Berardino, Luigi Beretta, Morten Blaabjerg, Peter Bragge, Alexandra Brazinova, Vibeke Brinck, Joanne Brooker, Camilla Brorsson, Andras Buki, Monika Bullinger, Manuel Cabeleira, Alessio Caccioppola, Emiliana Calappi, Maria Rosa Calvi, Peter Cameron, Guillermo Carbayo Lozano, Marco Carbonara, Giorgio Chevallard, Arturo Chieregato, Giuseppe Citerio, Maryse Cnossen, Mark Coburn, Jonathan Coles, D Jamie Cooper, Marta Correia, Amra Čović, Nicola Curry, Endre Czeiter, Marek Czosnyka, Claire Dahyot-Fizelier, Helen Dawes, Véronique De Keyser, Vincent Degos, Francesco Della Corte, Hugo den Boogert, Bart Depreitere, Đula Đilvesi, Abhishek Dixit, Emma Donoghue, Jens Dreier Guy-Loup Dulière, Ari Ercole, Patrick Esser, Erzsébet Ezer Martin Fabricius, Valery L Feigin Kelly Foks, Shirin Frisvold, Alex Furmanov, Pablo Gagliardo, Damien Galanaud, Dashiell Gantner, Guoyi Gao, Pradeep George, Alexandre Ghuysen, Lelde Giga, Ben Glocker, Jagoš Golubovic, Pedro A Gomez, Johannes Gratz, Benjamin Gravesteijn, Francesca Grossi, Russell L Gruen, Deepak Gupta, Juanita A Haagsma, Iain Haitsma, Raimund Helbok, Eirik Helseth, Lindsay Horton, Jilske Huijben, Peter J Hutchinson, Bram Jacobs, Stefan Jankowski, Mike Jarrett Ji-Yao Jiang, Kelly Jones, Mladen Karan, Angelos G Kolias, Erwin Kompanje, Daniel Kondziella, Evgenios Koraropoulos, Lars-Owe Koskinen, Noémi Kovács, Alfonso Lagares, Linda Lanyon, Steven Laureys, Fiona Lecky, Rolf Lefering, Valerie Legrand, Aurelie Lejeune, Leon Levi, Roger Lightfoot, Hester Lingsma, Andrew I R Maas, Ana M Castaño-León, Marc Maegele, Marek Majdan, Alex Manara, Geoffrey Manley, Costanza Martino, Hugues Maréchal, Julia Mattern, Catherine McMahon, Béla Melegh, David Menon, Tomas Menovsky, Davide Mulazzi, Visakh Muraleedharan, Lynnette Murray, Nandesh Nair, Ancuta Negru, David Nelson, Virginia Newcombe, Daan Nieboer, Quentin Noirhomme, József Nyirádi, Otesile Olubukola, Matej Oresic, Fabrizio Ortolano, Aarno Palotie, Paul M Parizel, Jean-François Payen, Natascha Perera, Vincent Perlbarg, Paolo Persona, Wilco Peul, Anna Piippo-Karjalainen, Matti Pirinen, Horia Ples, Suzanne Polinder, Inigo Pomposo, Jussi P Posti, Louis Puybasset, Andreea Radoi, Arminas Ragauskas, Rahul Raj, Malinka Rambadagalla, Ruben Real, Jonathan Rhodes, Sylvia Richardson, Sophie Richter, Samuli Ripatti, Saulius Rocka, Cecilie Roe, Olav Roise, Jonathan Rosand, Jeffrey V Rosenfeld, Christina Rosenlund, Guy Rosenthal, Rolf Rossaint, Sandra Rossi, Daniel Rueckert, Martin Rusnák, Juan Sahuquillo, Oliver Sakowitz, Renan Sanchez-Porras, Janos Sandor, Nadine Schäfer, Silke Schmidt, Herbert Schoechl, Guus Schoonman, Rico Frederik Schou, Elisabeth Schwendenwein, Charlie Sewalt, Toril Skandsen, Peter Smielewski, Abayomi Sorinola, Emmanuel Stamatakis, Simon Stanworth, Ana Kowark, Robert Stevens, William Stewart, Ewout W Steyerberg, Nino Stocchetti, Nina Sundström, Anneliese Synnot, Riikka Takala, Viktória Tamás, Tomas Tamosuitis, Mark Steven Taylor, Braden Te Ao, Olli Tenovuo, Alice Theadom, Matt Thomas, Dick Tibboel, Marjolein Timmers, Christos Tolias, Tony Trapani, Cristina Maria Tudora, Peter Vajkoczy, Shirley Vallance, Egils Valeinis, Zoltán Vámos, Gregory Van der Steen, Joukje van der Naalt, Jeroen T J M van Dijck, Thomas A van Essen, Wim Van Hecke, Caroline van Heugten, Dominique Van Praag, Thijs Vande Vyvere, Audrey Vanhaudenhuyse, Roel P J van Wijk, Alessia Vargiolu, Emmanuel Vega, Kimberley Velt, Jan Verheyden, Paul M Vespa, Anne Vik, Rimantas Vilcinis, Victor Volovici, Nicole von Steinbüchel, Daphne Voormolen, Petar Vulekovic, Kevin K W Wang, Eveline Wiegers, Guy Williams, Lindsay Wilson, Stefan Winzeck, Stefan Wolf, Zhihui Yang, Peter Ylén, Alexander Younsi, Frederik A Zeiler, Veronika Zelinkova, Agate Ziverte, Tommaso Zoerle, Benjamin Y Gravesteijn, Daan Nieboer, Ari Ercole, Hester F Lingsma, David Nelson, Ben van Calster, Ewout W Steyerberg, CENTER-TBI collaborators, Cecilia Åkerlund, Krisztina Amrein, Nada Andelic, Lasse Andreassen, Audny Anke, Anna Antoni, Gérard Audibert, Philippe Azouvi, Maria Luisa Azzolini, Ronald Bartels, Pál Barzó, Romuald Beauvais, Ronny Beer, Bo-Michael Bellander, Antonio Belli, Habib Benali, Maurizio Berardino, Luigi Beretta, Morten Blaabjerg, Peter Bragge, Alexandra Brazinova, Vibeke Brinck, Joanne Brooker, Camilla Brorsson, Andras Buki, Monika Bullinger, Manuel Cabeleira, Alessio Caccioppola, Emiliana Calappi, Maria Rosa Calvi, Peter Cameron, Guillermo Carbayo Lozano, Marco Carbonara, Giorgio Chevallard, Arturo Chieregato, Giuseppe Citerio, Maryse Cnossen, Mark Coburn, Jonathan Coles, D Jamie Cooper, Marta Correia, Amra Čović, Nicola Curry, Endre Czeiter, Marek Czosnyka, Claire Dahyot-Fizelier, Helen Dawes, Véronique De Keyser, Vincent Degos, Francesco Della Corte, Hugo den Boogert, Bart Depreitere, Đula Đilvesi, Abhishek Dixit, Emma Donoghue, Jens Dreier Guy-Loup Dulière, Ari Ercole, Patrick Esser, Erzsébet Ezer Martin Fabricius, Valery L Feigin Kelly Foks, Shirin Frisvold, Alex Furmanov, Pablo Gagliardo, Damien Galanaud, Dashiell Gantner, Guoyi Gao, Pradeep George, Alexandre Ghuysen, Lelde Giga, Ben Glocker, Jagoš Golubovic, Pedro A Gomez, Johannes Gratz, Benjamin Gravesteijn, Francesca Grossi, Russell L Gruen, Deepak Gupta, Juanita A Haagsma, Iain Haitsma, Raimund Helbok, Eirik Helseth, Lindsay Horton, Jilske Huijben, Peter J Hutchinson, Bram Jacobs, Stefan Jankowski, Mike Jarrett Ji-Yao Jiang, Kelly Jones, Mladen Karan, Angelos G Kolias, Erwin Kompanje, Daniel Kondziella, Evgenios Koraropoulos, Lars-Owe Koskinen, Noémi Kovács, Alfonso Lagares, Linda Lanyon, Steven Laureys, Fiona Lecky, Rolf Lefering, Valerie Legrand, Aurelie Lejeune, Leon Levi, Roger Lightfoot, Hester Lingsma, Andrew I R Maas, Ana M Castaño-León, Marc Maegele, Marek Majdan, Alex Manara, Geoffrey Manley, Costanza Martino, Hugues Maréchal, Julia Mattern, Catherine McMahon, Béla Melegh, David Menon, Tomas Menovsky, Davide Mulazzi, Visakh Muraleedharan, Lynnette Murray, Nandesh Nair, Ancuta Negru, David Nelson, Virginia Newcombe, Daan Nieboer, Quentin Noirhomme, József Nyirádi, Otesile Olubukola, Matej Oresic, Fabrizio Ortolano, Aarno Palotie, Paul M Parizel, Jean-François Payen, Natascha Perera, Vincent Perlbarg, Paolo Persona, Wilco Peul, Anna Piippo-Karjalainen, Matti Pirinen, Horia Ples, Suzanne Polinder, Inigo Pomposo, Jussi P Posti, Louis Puybasset, Andreea Radoi, Arminas Ragauskas, Rahul Raj, Malinka Rambadagalla, Ruben Real, Jonathan Rhodes, Sylvia Richardson, Sophie Richter, Samuli Ripatti, Saulius Rocka, Cecilie Roe, Olav Roise, Jonathan Rosand, Jeffrey V Rosenfeld, Christina Rosenlund, Guy Rosenthal, Rolf Rossaint, Sandra Rossi, Daniel Rueckert, Martin Rusnák, Juan Sahuquillo, Oliver Sakowitz, Renan Sanchez-Porras, Janos Sandor, Nadine Schäfer, Silke Schmidt, Herbert Schoechl, Guus Schoonman, Rico Frederik Schou, Elisabeth Schwendenwein, Charlie Sewalt, Toril Skandsen, Peter Smielewski, Abayomi Sorinola, Emmanuel Stamatakis, Simon Stanworth, Ana Kowark, Robert Stevens, William Stewart, Ewout W Steyerberg, Nino Stocchetti, Nina Sundström, Anneliese Synnot, Riikka Takala, Viktória Tamás, Tomas Tamosuitis, Mark Steven Taylor, Braden Te Ao, Olli Tenovuo, Alice Theadom, Matt Thomas, Dick Tibboel, Marjolein Timmers, Christos Tolias, Tony Trapani, Cristina Maria Tudora, Peter Vajkoczy, Shirley Vallance, Egils Valeinis, Zoltán Vámos, Gregory Van der Steen, Joukje van der Naalt, Jeroen T J M van Dijck, Thomas A van Essen, Wim Van Hecke, Caroline van Heugten, Dominique Van Praag, Thijs Vande Vyvere, Audrey Vanhaudenhuyse, Roel P J van Wijk, Alessia Vargiolu, Emmanuel Vega, Kimberley Velt, Jan Verheyden, Paul M Vespa, Anne Vik, Rimantas Vilcinis, Victor Volovici, Nicole von Steinbüchel, Daphne Voormolen, Petar Vulekovic, Kevin K W Wang, Eveline Wiegers, Guy Williams, Lindsay Wilson, Stefan Winzeck, Stefan Wolf, Zhihui Yang, Peter Ylén, Alexander Younsi, Frederik A Zeiler, Veronika Zelinkova, Agate Ziverte, Tommaso Zoerle
Abstract
Objective: We aimed to explore the added value of common machine learning (ML) algorithms for prediction of outcome for moderate and severe traumatic brain injury.
Study design and setting: We performed logistic regression (LR), lasso regression, and ridge regression with key baseline predictors in the IMPACT-II database (15 studies, n = 11,022). ML algorithms included support vector machines, random forests, gradient boosting machines, and artificial neural networks and were trained using the same predictors. To assess generalizability of predictions, we performed internal, internal-external, and external validation on the recent CENTER-TBI study (patients with Glasgow Coma Scale <13, n = 1,554). Both calibration (calibration slope/intercept) and discrimination (area under the curve) was quantified.
Results: In the IMPACT-II database, 3,332/11,022 (30%) died and 5,233(48%) had unfavorable outcome (Glasgow Outcome Scale less than 4). In the CENTER-TBI study, 348/1,554(29%) died and 651(54%) had unfavorable outcome. Discrimination and calibration varied widely between the studies and less so between the studied algorithms. The mean area under the curve was 0.82 for mortality and 0.77 for unfavorable outcomes in the CENTER-TBI study.
Conclusion: ML algorithms may not outperform traditional regression approaches in a low-dimensional setting for outcome prediction after moderate or severe traumatic brain injury. Similar to regression-based prediction models, ML algorithms should be rigorously validated to ensure applicability to new populations.
Trial registration: ClinicalTrials.gov NCT02210221.
Keywords: Cohort study; Data science; Machine learning; Prediction; Prognosis; Traumatic brain injury.
Copyright © 2020 The Authors. Published by Elsevier Inc. All rights reserved.
Source: PubMed