Diagnosing the GOSE: Structural and Psychometric Properties Using Item Response Theory, a TRACK-TBI Pilot Study

Abstract

The Glasgow Outcome Scale-Extended (GOSE) was designed to assess global outcome after traumatic brain injury (TBI). Since its introduction, several empirically founded criticisms of the GOSE have been raised, including poor reliability; an insensitivity to small, but potentially meaningful, changes; a tendency to produce ceiling effects; inconsistent associations with neurocognitive, psychological, and quality-of-life measures; and an inability to assess the multi-dimensional nature of TBI outcome. The current project took a diagnostic approach to identifying the underlying causes of reported limitations by exploring the internal construct validity of the GOSE at 3 and 6 months post-injury using item response theory (IRT) techniques. Data were from the TRACK-TBI Pilot Study, a large (N = 586), prospective, multi-site project that included TBI cases of all injury severity levels. To assess the level of latent functional "impairment" captured by GOSE items independent of the assigned outcome category or GOSE total score, items were modified so that higher scores reflected greater impairment. Results showed that although the GOSE's items capture varying levels of impairment across a broad disability spectrum at 3 and 6 months, there was also evidence at each time point of item redundancy (multiple items capturing similar levels of impairment), item deficiency (lack of items capturing lower levels of impairment), and item inefficiency (items only capturing minimal impairment information). The findings illustrate the value of IRT to illuminate strengths and weaknesses of clinical outcome assessment measures and provide a framework for future measure refinement.

Trial registration: ClinicalTrials.gov NCT01565551.

Keywords: Glasgow Outcome Scale–Extended; item response theory; outcome assessment; psychometrics; traumatic brain injury.

Conflict of interest statement

Dr. Temkin reports grants from the Department of Defense, NIH, NIDILRR, and CDC during the conduct of the study. Dr. Dikmen reports grants from NIH and NIDILRR during the conduct of the study. Dr. Giacino reports grants from the Department of Defense, NIH, NIDILRR, James S. McDonnell Foundation, and other support from the Barbara Epstein Foundation during the conduct of the study. Dr. Okonkwo reports grants from NIH and the Department of Defense during the conduct of the study. Dr. Manley reports grants from the Department of Defense, NIH, and other support from One Mind, Palantir, and Johnson & Johnson Family of Companies/DePuySynthes/Codman Neuro during the conduct of the study. Dr. Nelson reports grants from NIH and the Medical College of Wisconsin's Center for Patient Care and Outcomes Research, Clinical and Translational Science Institute, and Advancing a Healthier Wisconsin Endowment during the conduct of the study.

Figures

FIG. 1.

Item response functions (IRFs) from a two-parameter item response theory (2PL-IRT) model, using four hypothetical dichotomous items within a hypothetical test. This model allows one to characterize items both in term of their “difficulty” and “discrimination.” Difficulty reflects the level of the latent construct (i.e., the point along the x-axis) at which participants have a 50% chance of endorsing an item. For example, among the three items depicted by solid lines, a lower level of the latent construct is required to endorse the lighter colored (leftmost) line, where a higher level of the construct is needed to endorse the solid black line. Discrimination reflects the slope of the line at the difficulty level of the item, where steeper slopes translate to being able to estimate individuals along the latent continuum with more precision (lower standard errors). In other words, items with high discrimination (solid lines) yield more “information” (i.e., more precise estimates) of individuals' scores on the latent construct of interest than items with lower discrimination (dashed line). For the goal of measuring TBI-related functional limitations along a wide spectrum of severity, a desirable test would contain items high in discrimination that span a wide range of difficulty (severity) levels. TBI, traumatic brain injury.

FIG. 2.

Two-parameter/graded response hybrid model item and total information functions (curves) for the five modified items of the GOSE instrument at 3 (A) and 6 (B) months. Each function represents the amount of information (precision) provided by each item across the theta (θ) continuum of functional limitations (global outcome) after TBI. The information provided by each item was similar across time, with the exception of 8a, which provided substantially more information at 6 as compared to 3 months. Across time, the GOSE as a whole (black line) yielded the most information about moderate-to-severe functional limitations (right half of the figure). Item 2b (Independence Inside the Home) was not included in this analysis because, when entered into the model with item 3a4a, both produced perfectly discriminating items (discrimination >5). However, substituting item 3a4a for item 2b yielded no difference in the item or total information curves, implying redundancy in the psychometric performance of these two items. Legend: 3a4a = Independence Outside the Home; 5b = Work; 6b = Social & Leisure Activities; 7b = Family and Friendship Disruptions; 8a = Return to Normal Life/Other Issues. GOSE, Glasgow Outcome Scale–Extended; TBI, traumatic brain injury. Color image is available online.

FIG. 3.

Scatterplot of GOSE total scores crossed by IRT theta (θ) z-scores at 3 and 6 months post-injury. Circles denote individual cases. Squares denote cases in which item recoding procedures necessary for IRT analyses introduced the potential to artificially deflate the association between traditional and IRT-based GOSE scores. These cases were not included in correlational analyses. Higher theta scores (rightward on x-axis) reflect greater latent impairment whereas lower GOSE total scores (downward on the y-axis) reflect poorer global outcome. Although the overall distribution of cases at both time points support the expected negative association between latent impairment and GOSE total scores (greater impairment ∼ lower GOSE total score), there was substantial variability of latent impairment occurring within the GOSE outcome categories 3 through 7. GOSE, Glasgow Outcome Scale–Extended; IRT, item response function.