Pairing regression and configurational analysis in health services research: modelling outcomes in an observational cohort using a split-sample design

Edward J Miech, Anthony J Perkins, Ying Zhang, Laura J Myers, Jason J Sico, Joanne Daggy, Dawn M Bravata, Edward J Miech, Anthony J Perkins, Ying Zhang, Laura J Myers, Jason J Sico, Joanne Daggy, Dawn M Bravata

Abstract

Background: Configurational methods are increasingly being used in health services research.

Objectives: To use configurational analysis and logistic regression within a single data set to compare results from the two methods.

Design: Secondary analysis of an observational cohort; a split-sample design involved randomly dividing patients into training and validation samples.

Participants and setting: Patients who had a transient ischaemic attack (TIA) in US Department of Veterans Affairs hospitals.

Measures: The patient outcome was the combined endpoint of all-cause mortality or recurrent ischaemic stroke within 1 year post-TIA. The quality-of-care outcome was the without-fail rate (proportion of patients who received all processes for which they were eligible, among seven processes).

Results: For the recurrent stroke or death outcome, configurational analysis yielded a three-pathway model identifying a set of (validation sample) patients where the prevalence was 15.0% (83/552), substantially higher than the overall sample prevalence of 11.0% (relative difference, 36%). The configurational model had a sensitivity (coverage) of 84.7% and specificity of 40.6%. The logistic regression model identified six factors associated with the combined endpoint (c-statistic, 0.632; sensitivity, 63.3%; specificity, 63.1%). None of these factors were elements of the configurational model. For the quality outcome, configurational analysis yielded a single-pathway model identifying a set of (validation sample) patients where the without-fail rate was 64.3% (231/359), nearly twice the overall sample prevalence (33.7%). The configurational model had a sensitivity (coverage) of 77.3% and specificity of 78.2%. The logistic regression model identified seven factors associated with the without-fail rate (c-statistic, 0.822; sensitivity, 80.3%; specificity, 84.2%). Two of these factors were also identified in the configurational analysis.

Conclusions: Configurational analysis and logistic regression represent different methods that can enhance our understanding of a data set when paired together. Configurational models optimise sensitivity with relatively few conditions. Logistic regression models discriminate cases from controls and provided inferential relationships between outcomes and independent variables.

Keywords: neurology; statistics & research methods; stroke.

Conflict of interest statement

Competing interests: None declared.

© Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

References

    1. Yakovchenko V, Miech EJ, Chinman MJ, et al. . Strategy configurations directly linked to higher hepatitis C virus treatment starts: an applied use of configurational comparative methods. Med Care 2020;58:e31–8. 10.1097/MLR.0000000000001319
    1. Hickman SE, Miech EJ, Stump TE, et al. . Identifying the implementation conditions associated with positive outcomes in a successful nursing facility demonstration project. Gerontologist 2020;60:1566–74. 10.1093/geront/gnaa041
    1. Palinkas LA, Mendon SJ, Hamilton AB. Innovations in mixed methods evaluations. Annu Rev Public Health 2019;40:423–42. 10.1146/annurev-publhealth-040218-044215
    1. Harris K, Kneale D, Lasserson TJ, et al. . School-based self-management interventions for asthma in children and adolescents: a mixed methods systematic review. Cochrane Database Syst Rev 2019;1:Cd011651. 10.1002/14651858.CD011651.pub2
    1. Cragun D. Configurational comparative methods. In: The handbook on implementation science. Cheltenham: Edward Elgar Publishing, 2020.
    1. Whitaker RG, Sperber N, Baumgartner M, et al. . Coincidence analysis: a new method for causal inference in implementation science. Implement Sci 2020;15:108. 10.1186/s13012-020-01070-3
    1. Cragun D, Pal T, Vadaparampil ST, et al. . Qualitative comparative analysis: a hybrid method for identifying factors associated with program effectiveness. J Mix Methods Res 2016;10:251–72. 10.1177/1558689815572023
    1. Ragin C. The comparative method: moving beyond qualitative and quantitative strategies. Berkeley, CA: University of California Press, 1987.
    1. Thiem A. Conducting configurational comparative research with qualitative comparative analysis: a hands-on tutorial for applied evaluation scholars and practitioners. Am J Evaluat 2017;38:420–33.
    1. Giménez-Espert MDC, Valero-Moreno S, Prado-Gascó VJ. Evaluation of emotional skills in nursing using regression and QCA models: a transversal study. Nurse Educ Today 2019;74:31–7. 10.1016/j.nedt.2018.11.019
    1. Ragin C, Fiss P. Intersectional inequality. University of Chicago Press, 2016.
    1. Ragin CC, Shulman D, Weinberg A, et al. . Complexity, generality, and qualitative comparative analysis. Field Methods 2003;15:323–40. 10.1177/1525822X03257689
    1. Ebbinghaus B, Visser J. When institutions matter: union growth and decline in Western Europe, 1950-1995. Eur Sociol Rev 1999;15:135–58. 10.1093/oxfordjournals.esr.a018257
    1. Grofman B, Schneider CQ. An introduction to CRISP set QCA, with a comparison to binary logistic regression. Polit Res Q 2009;62:662–72. 10.1177/1065912909338464
    1. Bravata DM, Myers LJ, Perkins AJ, et al. . Assessment of the protocol-guided rapid evaluation of veterans experiencing new transient neurological symptoms (prevent) program for improving quality of care for transient ischemic attack: a nonrandomized cluster trial. JAMA Netw Open 2020;3:e2015920. 10.1001/jamanetworkopen.2020.15920
    1. Shah KH, Metz HA, Edlow JA. Clinical prediction rules to stratify short-term risk of stroke among patients diagnosed in the emergency department with a transient ischemic attack. Ann Emerg Med 2009;53:662–73. 10.1016/j.annemergmed.2008.08.004
    1. Concato J, Peduzzi P, Holford TR, et al. . Importance of events per independent variable in proportional hazards analysis. I. background, goals, and general strategy. J Clin Epidemiol 1995;48:1495–501. 10.1016/0895-4356(95)00510-2
    1. Peduzzi P, Concato J, Feinstein AR, et al. . Importance of events per independent variable in proportional hazards regression analysis. II. accuracy and precision of regression estimates. J Clin Epidemiol 1995;48:1503–10. 10.1016/0895-4356(95)00048-8
    1. Bravata DM, Wells CK, Lo AC, et al. . Processes of care associated with acute stroke outcomes. Arch Intern Med 2010;170:804–10. 10.1001/archinternmed.2010.92
    1. Bravata DM, Myers LJ, Homoya B, et al. . The protocol-guided rapid evaluation of veterans experiencing new transient neurological symptoms (prevent) quality improvement program: rationale and methods. BMC Neurol 2019;19:294. 10.1186/s12883-019-1517-x
    1. Li J, Zhang Y, Myers LJ, et al. . Power calculation in stepped-wedge cluster randomized trial with reduced intervention sustainability effect. J Biopharm Stat 2019;29:663–74. 10.1080/10543406.2019.1633658
    1. Development VHSR . VA informatics and computing infrastructure (VINCI), 2008. Available: [Accessed 10 Dec 2020].
    1. Affairs USDoV . VA informatics and computing infrastructure (VINCI) 2008.
    1. Borzecki AM, Wong AT, Hickey EC, et al. . Can we use automated data to assess quality of hypertension care? Am J Manag Care 2004;10:473–9.
    1. Sohn M-W, Arnold N, Maynard C, et al. . Accuracy and completeness of mortality data in the Department of Veterans Affairs. Popul Health Metr 2006;4:2. 10.1186/1478-7954-4-2
    1. Man S, Zhao X, Uchino K, et al. . Comparison of acute ischemic stroke care and outcomes between comprehensive stroke centers and primary stroke centers in the United States. Circ Cardiovasc Qual Outcomes 2018;11:e004512. 10.1161/CIRCOUTCOMES.117.004512
    1. Fonarow GC, Reeves MJ, Smith EE, et al. . Characteristics, performance measures, and in-hospital outcomes of the first one million stroke and transient ischemic attack admissions in get with the Guidelines-Stroke. Circ Cardiovasc Qual Outcomes 2010;3:291–302. 10.1161/CIRCOUTCOMES.109.921858
    1. Nolan T, Berwick DM. All-or-none measurement raises the bar on performance. JAMA 2006;295:1168–70. 10.1001/jama.295.10.1168
    1. Bravata DM, Myers LJ, Homoya B, et al. . The protocol-guided rapid evaluation of veterans experiencing new transient neurological symptoms (PREVENT) quality improvement program: rationale and methods. BMC Neurol 2019;19:294. 10.1186/s12883-019-1517-x
    1. Bravata DM, Myers LJ, Cheng E, et al. . Development and validation of electronic quality measures to assess care for patients with transient ischemic attack and minor ischemic stroke. Circ Cardiovasc Qual Outcomes 2017;10. 10.1161/CIRCOUTCOMES.116.003157
    1. Bravata DM, Myers LJ, Arling G, et al. . Quality of care for veterans with transient ischemic attack and minor stroke. JAMA Neurol 2018;75:419–27. 10.1001/jamaneurol.2017.4648
    1. Kernan WN, Ovbiagele B, Black HR, et al. . Guidelines for the prevention of stroke in patients with stroke and transient ischemic attack: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 2014;45:2160–236. 10.1161/STR.0000000000000024
    1. Smith EE, Saver JL, Alexander DN, et al. . Clinical performance measures for adults hospitalized with acute ischemic stroke: performance measures for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 2014;45:3472–98. 10.1161/STR.0000000000000045
    1. Bravata DM, Myers LJ, Reeves M, et al. . Processes of care associated with risk of mortality and recurrent stroke among patients with transient ischemic attack and nonsevere ischemic stroke. JAMA Network Open 2019;2:e196716. 10.1001/jamanetworkopen.2019.6716
    1. Steyerberg EW, Harrell FE, Borsboom GJ, et al. . Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001;54:774–81. 10.1016/s0895-4356(01)00341-9
    1. CNA: causal modeling with coincidence analysis. R package version 2.1.1. [program] 2018.
    1. Cohen DJ, Sweeney SM, Miller WL, et al. . Improving smoking and blood pressure outcomes: the interplay between operational changes and local context. Ann Fam Med 2021;19:240–8. 10.1370/afm.2668
    1. Coury J, Miech EJ, Styer P, et al. . What’s the ‘secret sauce’? How implementation variation affects the success of colorectal cancer screening outreach. Implement Sci Commun 2021;2:5. 10.1186/s43058-020-00104-7
    1. Petrik AF, Green B, Schneider J, et al. . Factors influencing implementation of a colorectal cancer screening improvement program in community health centers: an applied use of configurational comparative methods. J Gen Intern Med 2020;35:815–22. 10.1007/s11606-020-06186-2
    1. Baumgartner M, Thiem A. Model ambiguities in configurational comparative research. Sociol Methods Res 2017;46:954–87. 10.1177/0049124115610351
    1. Pisters R, Lane DA, Nieuwlaat R, et al. . A novel user-friendly score (HAS-BLED) to assess 1-year risk of major bleeding in patients with atrial fibrillation. Chest 2010;138:1093–100. 10.1378/chest.10-0134
    1. Charlson ME, Pompei P, Ales KL, et al. . A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40:373–83. 10.1016/0021-9681(87)90171-8
    1. Knaus W, Wagner D, Draper E. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991;100:1619–36.
    1. Thiem A, Baumgartner M, Bol D. Still lost in translation! a correction of three misunderstandings between configurational comparativists and regressional analysts. Comp Polit Stud 2016;49:742–74. 10.1177/0010414014565892
    1. Rattray NA, Miech EJ, True G, et al. . Modeling contingency in veteran community reintegration: a mixed methods approach. J Mix Methods Res;78:15586898211059616. 10.1177/15586898211059616
    1. Baumgartner M. Qualitative comparative analysis and robust sufficiency. Qual Quant 2021:1–25. 10.1007/s11135-021-01157-z

Source: PubMed

Patrocinadores y Colaboradores

Condiciones médicas

Intervenciones de drogas

3
Suscribir