Comparing two advance care planning conversation activities to motivate advance directive completion in underserved communities across the USA: The Project Talk Trial study protocol for a cluster, randomized controlled trial

Lauren J Van Scoy, Benjamin H Levi, Cindy Bramble, William Calo, Vernon M Chinchilli, Lindsey Currin, Denise Grant, Christopher Hollenbeak, Maria Katsaros, Sara Marlin, Allison M Scott, Amy Tucci, Erika VanDyke, Emily Wasserman, Pamela Witt, Michael J Green, Lauren J Van Scoy, Benjamin H Levi, Cindy Bramble, William Calo, Vernon M Chinchilli, Lindsey Currin, Denise Grant, Christopher Hollenbeak, Maria Katsaros, Sara Marlin, Allison M Scott, Amy Tucci, Erika VanDyke, Emily Wasserman, Pamela Witt, Michael J Green

Abstract

Background: Advance care planning (ACP) is a process involving conversations between patients, loved ones, and healthcare providers that consider patient preferences for the types of medical therapies received at the end of life. Underserved populations, including Black, Hispanic, rural, and low-income communities are less likely to engage in ACP than other communities, a health inequity that results in lower-quality care and reduced hospice utilization. The purpose of this trial is to compare efficacy of two interventions intended to motivate ACP (particularly advance directive completion) for those living in underserved communities.

Methods: This 3-armed cluster, randomized controlled mixed methods design is being conducted in 75 community venues in underserved communities across the USA. The goal of the trial is to compare the efficacy of two interventions at motivating ACP. Arm 1 uses an end-of-life conversation game (Hello); Arm 2 uses a nationally utilized workshop format for ACP conversations (The Conversation Project); and Arm 3 uses an attention control game (TableTopics). Events are held in partnership with 75 local community-based host organizations and will involve 1500 participants (n=20 per event). The primary outcome is completion of a visually verified advance directive at 6 months post-event. Primary analyses compare efficacy of each intervention to each other and the control arm. Secondary mixed methods outcomes include (a) other ACP behaviors and engagement; (b) communication quality; (c) impact of sociocultural environment on ACP (via qualitative interviews); and (d) implementation and sustainability. Subgroup analyses examine outcomes for Black, Hispanic, and rural groups in particular.

Discussion: This trial will add to the evidence base behind various conversational ACP interventions, examine potential mechanisms of action for such interventions, and provide qualitative data to better understand the sociocultural environment of how community-based ACP interventions are experienced by underserved populations. Results will also provide important data for future researchers to learn whether visual verification of advance directives is necessary or whether reliance on self-reported outcomes is of comparable value. Data from this study will inform ways to effectively motivate underserved communities to participate in advance care planning.

Trial registration: ClinicalTrials.gov NCT04612738. Registered on October 12, 2020. All information from the WHO Trial Registration Data Set can be found within the protocol.

Keywords: Advance care planning; Advance directives; Health behavior; Health communication; Health games; Mixed methods; Randomized controlled trial; Terminal illness; Underserved communities.

Conflict of interest statement

The Principal Investigator (LJV) is an unpaid scientific advisor for Common Practice, LLC, which is the company that designed the Hello game. The remaining authors declare that they have no competing interests.

© 2022. The Author(s).

Figures

Fig. 1
Fig. 1
Overview of mixed methods study design. The study design involves a three-armed, cluster randomized controlled trial of two ACP interventions and an attention control. Gray boxes represent mixed methods data elements. CP = Conversation Project; CQA= Communication Quality Analysis; ACP= advance care planning
Fig. 2
Fig. 2
Overview of the Communication Quality Analysis coding procedures. A domain score is calculated for each of the six communication quality domains. As participants meet the definitions of each domain, the scores are increased. A domain score is assigned every 5 min for each of the six domains. All domains are scored 1–7 with “7” being the highest-quality score
Fig. 3
Fig. 3
Calculation of the multiple goals score. The multiple goals score is a summative measure of all 6 quality domains that is reported in a single “breadth” score. To calculate the multiple goals score, each of the six domain scores are combined into three goals scores (task, relational, and identity goals). For each of these three goals, a normative score of “0” or “1” is assigned based on whether the score is above or below the sample mean. The MGS score is calculated for each time segment by taking the sum of each of the three normative goals scores (resulting a range from 0 to 3). Once the MGS score is calculated for each 5-min time interval, scores are then averaged across time intervals

References

    1. Lennon RP, Small ML, Smith RA, Van Scoy LJ, Myrick JG, Martin MA, et al. Unique predictors of intended uptake of a COVID-19 vaccine. medRxiv. 2020;36(1):180-4.
    1. Loggers ET, Maciejewski PK, Paulk E, DeSanto-Madeya S, Nilsson M, Viswanath K, et al. Racial differences in predictors of intensive end-of-life care in patients with advanced cancer. J Clin Oncol. 2009;27(33):5559–5564. doi: 10.1200/JCO.2009.22.4733.
    1. LoPresti MA, Dement F, Gold HT. End-of-life care for people with cancer from ethnic minority groups: a systematic review. Am J Hosp Palliat Med. 2016;33(3):291–305. doi: 10.1177/1049909114565658.
    1. Mack JW, Paulk ME, Viswanath K, Prigerson HG. Racial disparities in the outcomes of communication on medical care received near death. Arch Intern Med. 2010;170(17):1533–1540. doi: 10.1001/archinternmed.2010.322.
    1. Degenholtz HB, Arnold RA, Meisel A, Lave JR. Persistence of racial disparities in advance care plan documents among nursing home residents. J Am Geriatr Soc. 2002;50(2):378–381. doi: 10.1046/j.1532-5415.2002.50073.x.
    1. Cohen LL. Racial/ethnic disparities in hospice care: a systematic review. J Palliat Med. 2008;11(5):763–768. doi: 10.1089/jpm.2007.0216.
    1. Connor S, Elwert F, Spence C, Christakis N. Racial disparity in hospice use in the United States in 2002. Palliat Med. 2008;22(3):205–213. doi: 10.1177/0269216308089305.
    1. Kwak J, Haley WE. Current research findings on end-of-life decision making among racially or ethnically diverse groups. Gerontologist. 2005;45(5):634–641. doi: 10.1093/geront/45.5.634.
    1. Sudore RL, Lum HD, You JJ, Hanson LC, Meier DE, Pantilat SZ, et al. Defining advance care planning for adults: a consensus definition from a multidisciplinary Delphi panel. J Pain Symptom Manage. 2017;53(5):821–32 e1. doi: 10.1016/j.jpainsymman.2016.12.331.
    1. Silveira MJ, Wiitala W, Piette J. Advance directive completion by elderly Americans: a decade of change. J Am Geriatr Soc. 2014;62(4):706–710. doi: 10.1111/jgs.12736.
    1. Teno JM, Gozalo PL, Bynum JPW, Leland NE, Miller SC, Morden NE, et al. Change in end-of-life care for Medicare beneficiaries: site of death, place of care, and health care transitions in 2000, 2005, and 2009. Jama. 2013;309(5):470–477. doi: 10.1001/jama.2012.207624.
    1. Detering KM, Hancock AD, Reade MC, Silvester W. The impact of advance care planning on end of life care in elderly patients: randomised controlled trial. BMJ. 2010;340:1345. doi: 10.1136/bmj.c1345.
    1. Morrison RS, Chichin E, Carter J, Burack O, Lantz M, Meier DE. The effect of a social work intervention to enhance advance care planning documentation in the nursing home. J Am Geriatr Soc. 2005;53(2):290–294. doi: 10.1111/j.1532-5415.2005.53116.x.
    1. Casarett D, Karlawish J, Morales K, Crowley R, Mirsch T, Asch DA. Improving the use of hospice services in nursing homes: a randomized controlled trial. Jama. 2005;294(2):211–217. doi: 10.1001/jama.294.2.211.
    1. Kramer BJ, Kavanaugh M, Trentham-Dietz A, Walsh M, Yonker JA. Predictors of family conflict at the end of life: The experience of spouses and adult children of persons with lung cancer. Gerontologist. 2009;50(2):215–225. doi: 10.1093/geront/gnp121.
    1. Angus DC, Barnato AE, Linde-Zwirble WT, Weissfeld LA, Watson RS, Rickert T, et al. Use of intensive care at the end of life in the United States: an epidemiologic study*. Crit Care Med. 2004;32(3):638–643. doi: 10.1097/01.CCM.0000114816.62331.08.
    1. Nicholas LH, Langa KM, Iwashyna TJ, Weir DR. Regional variation in the association between advance directives and end-of-life medicare expenditures. Jama. 2011;306(13):1447–1453. doi: 10.1001/jama.2011.1410.
    1. Zhang B, Wright AA, Huskamp HA, Nilsson ME, Maciejewski ML, Earle CC, et al. Health care costs in the last week of life: associations with end-of-life conversations. Arch Intern Med. 2009;169(5):480–488. doi: 10.1001/archinternmed.2008.587.
    1. Martin RS, Hayes B, Gregorevic K, Lim WK. The effects of advance care planning interventions on nursing home residents: a systematic review. J Am Med Dir Assoc. 2016;17(4):284–293. doi: 10.1016/j.jamda.2015.12.017.
    1. Dying in America: Improving quality and honoring individual preferences near the end of life. Washington, D.C.: National Academies Press; 2015.
    1. H.R.4449 - Patient Self Determination Act of 1990, (1990).
    1. Congress t . The Patient Protection and Affordable Care Act: Public Law. 2010. pp. 111–148.
    1. Annas GJ, Galea S. Dying healthy: public health priorities for fixed population life expectancies. Ann Intern Med. 2018;169(8):568–569. doi: 10.7326/M18-1609.
    1. Rao JK, Anderson LA, Lin FC, Laux JP. Completion of advance directives among U.S. consumers. Am J Prev Med. 2014;46(1):65–70. doi: 10.1016/j.amepre.2013.09.008.
    1. Morhaim DK, Pollack KM. End-of-life care issues: a personal, economic, public policy, and public health crisis. Am J Public Health. 2013;103(6):e8–e10. doi: 10.2105/AJPH.2013.301316.
    1. Riley GF, Lubitz JD. Long-term trends in Medicare payments in the last year of life. Health Serv Res. 2010;45(2):565–576. doi: 10.1111/j.1475-6773.2010.01082.x.
    1. Jones T, Luth EA, Lin SY, Brody AA. Advance care planning, palliative care, and end-of-life care interventions for racial and ethnic underrepresented groups: a systematic review. J Pain Symptom Manage. 2021;62(3):e248–ee60. doi: 10.1016/j.jpainsymman.2021.04.025.
    1. Jimenez G, Tan WS, Virk AK, Low CK, Car J, Ho AHY. State of advance care planning research: a descriptive overview of systematic reviews. Palliat Support Care. 2019;17(2):234–244. doi: 10.1017/S1478951518000500.
    1. Morrison RS, Meier DE, Arnold RM. What's wrong with advance care planning? Jama. 2021;326(16):1575–1576. doi: 10.1001/jama.2021.16430.
    1. Myers J, Steinberg L, Seow H. Controversies about advance care planning. Jama. 2022;327(7):684–685. doi: 10.1001/jama.2021.24733.
    1. Auriemma CL, O’Donnell H, Klaiman T, Jones J, Barbati Z, Akpek E, et al. How traditional advance directives undermine advance care planning: if you have it in writing, you do not have to worry about it. JAMA. Intern Med. 2022;182(6):682-4.
    1. Yadav KN, Gabler NB, Cooney E, Kent S, Kim J, Herbst N, et al. Approximately one in three US adults completes any type of advance directive for end-of-life care. Health Aff. 2017;36(7):1244–1251. doi: 10.1377/hlthaff.2017.0175.
    1. Resnick HE, Hickman S, Foster GL. Documentation of advance directives among home health and hospice patients: United States, 2007. Am J Hosp Palliat Care. 2012;29(1):26–35. doi: 10.1177/1049909111407627.
    1. Greenberg MR, Weiner M, Greenberg GB. Risk-reducing legal documents: controlling personal health and financial resources. Risk Anal. 2009;29(11):1578–1587. doi: 10.1111/j.1539-6924.2009.01282.x.
    1. Sanders JJ, Robinson MT, Block SD. Factors impacting advance care planning among African Americans: results of a systematic integrated review. J Palliat Med. 2016;19(2):202–227. doi: 10.1089/jpm.2015.0325.
    1. Koss CS, Baker TA. Race differences in advance directive completion: the narrowing gap between White and African American older adults. J Aging Health. 2017;29(2):324–342. doi: 10.1177/0898264316635568.
    1. Bischoff KE, Sudore R, Miao Y, Boscardin WJ, Smith AK. Advance care planning and the quality of end-of-life care in older adults. J Am Geriatr Soc. 2013;61(2):209–214. doi: 10.1111/jgs.12105.
    1. Harrison KL, Adrion ER, Ritchie CS, Sudore RL, Smith AK. Low completion and disparities in advance care planning activities among older medicare beneficiaries. JAMA Intern Med. 2016;176(12):1872–1875. doi: 10.1001/jamainternmed.2016.6751.
    1. Lyon ME, Squires L, D'Angelo LJ, Benator D, Scott RK, Greenberg IH, et al. FAmily-CEntered (FACE) advance care planning among African-American and non-African-American adults living with HIV in Washington, DC: a randomized controlled trial to increase documentation and health equity. J Pain Symptom Manage. 2019;57(3):607–616. doi: 10.1016/j.jpainsymman.2018.11.014.
    1. Sudore RL, Boscardin J, Feuz MA, McMahan RD, Katen MT, Barnes DE. Effect of the PREPARE website vs an easy-to-read advance directive on advance care planning documentation and engagement among veterans: a randomized clinical trial. JAMA Intern Med. 2017;177(8):1102–1109. doi: 10.1001/jamainternmed.2017.1607.
    1. Stevens J, Deliens L, Pype P, De Vleminck A, Pardon K. Complex advance care planning interventions for chronic serious illness: how do they work: a scoping review. BMJ Support Palliat Care. 2021;12(e3):e441-50.
    1. Houlihan MCK, Mayahara M, Swanson B, Fogg L. A review of clinical trials of advance care planning interventions adapted for limited health literacy. Palliat Support Care. 2021;20(4):593-9.
    1. Fahner JC, Beunders AJM, van der Heide A, Rietjens JAC, Vanderschuren MM, van Delden JJM, et al. Interventions guiding advance care planning conversations: a systematic review. J Am Med Dir Assoc. 2019;20(3):227–248. doi: 10.1016/j.jamda.2018.09.014.
    1. McMahan RD, Tellez I, Sudore RL. Deconstructing the complexities of advance care planning outcomes: what do we know and where do we go? A scoping review. J Am Geriatr Soc. 2020;69(1):234-44.
    1. Sudore RL. Preparing surrogates for complex decision making: the often neglected piece of the advance care planning equation. JAMA Intern Med. 2019;179(2):268–269. doi: 10.1001/jamainternmed.2018.5280.
    1. Sudore RL, Fried TR. Redefining the “planning” in advance care planning: preparing for end-of-life decision making. Ann Intern Med. 2010;153(4):256–261. doi: 10.7326/0003-4819-153-4-201008170-00008.
    1. Johnson KS, Kuchibhatla M, Tulsky JA. What explains racial differences in the use of advance directives and attitudes toward hospice care? J Am Geriatr Soc. 2008;56(10):1953–1958. doi: 10.1111/j.1532-5415.2008.01919.x.
    1. Ethnicity and attitudes towards life sustaining technology, (1999).
    1. Duffy S, Jackson F, Schim S, Ronis D, Fowler K. Cultural concepts at the end of life. Nurs Older People. 2006;18(8):10–14. doi: 10.7748/nop.18.8.10.s8.
    1. Wicher CP, Meeker MA. What influences African American end-of-life preferences? J Health Care Poor Underserved. 2012;23(1):28–58. doi: 10.1353/hpu.2012.0027.
    1. Ladd SC. Systematic review of research literature on African Americans’ end-of-life healthcare preferences. J Afr Am Stud. 2014;18(4):373–397. doi: 10.1007/s12111-013-9276-z.
    1. Fried TR, Bullock K, Iannone L, O'Leary JR. Understanding advance care planning as a process of health behavior change. J Am Geriatr Soc. 2009;57(9):1547–1555. doi: 10.1111/j.1532-5415.2009.02396.x.
    1. Schickedanz AD, Schillinger D, Landefeld CS, Knight SJ, Williams BA, Sudore RL. A clinical framework for improving the advance care planning process: start with patients’ self-identified barriers. J Am Geriatr Soc. 2009;57(1):31–39. doi: 10.1111/j.1532-5415.2008.02093.x.
    1. Ramsaroop SD, Reid MC, Adelman RD. Completing an advance directive in the primary care setting: what do we need for success? J Am Geriatr Soc. 2007;55(2):277–283. doi: 10.1111/j.1532-5415.2007.01065.x.
    1. Berger JT. What about process? Limitations in advance directives, care planning, and noncapacitated decision making. Am J Bioeth. 2010;10(4):33–34. doi: 10.1080/15265161003632948.
    1. Hello: Great Conversations Start with Hello [Available from: ]. Accessed 1 Sept 2022.
    1. Radhakrishnan K, Van Scoy LJ, Jillapalli R, Saxena S, Kim MT. Community-based game intervention to improve South Asian Indian Americans’ engagement with advanced care planning. Ethn Health. 2017;24(6):705-23.
    1. Van Scoy LJ, Green MJ, Reading JM, Scott AM, Chuang CH, Levi BH. Can playing an end-of-life conversation game motivate people to engage in advance care planning? Am J Hosp Palliat Care. 2017;34(8):754–761. doi: 10.1177/1049909116656353.
    1. Van Scoy LJ, Green MJ, Volpe R. Evaluating an advance care planning curriculum: a lecture, a game, a patient, and an essay. Medical. Sci Educ. 2019;29(2):453-62.
    1. Van Scoy LJ, Reading JM, Hopkins M, Smith B, Dillon J, Green MJ, et al. Community game day: using an end-of-life conversation game to encourage advance care planning. J Pain Symptom Manage. 2017;54(5):680–691. doi: 10.1016/j.jpainsymman.2017.07.034.
    1. Van Scoy LJ, Reading JM, Scott AM, Chuang C, Levi BH, Green MJ. Exploring the topics discussed during a conversation card game about death and dying: a content analysis. J Pain Symptom Manage. 2016;52(5):655–662. doi: 10.1016/j.jpainsymman.2016.03.021.
    1. Van Scoy LJ, Reading JM, Scott AM, Green MJ, Levi BH. Conversation game effectively engages groups of individuals in discussions about death and dying. J Palliat Med. 2016;19(6):661–667. doi: 10.1089/jpm.2015.0390.
    1. Van Scoy LJ, Watson-Martin E, Bohr TA, Levi BH, Green MJ. End-of-life conversation game increases confidence for having end-of-life conversations for chaplains-in-training. Am J Hosp Palliat Care. 2017;35(4):592-600.
    1. Van Scoy LJ, Levi BH, Witt P, Bramble C, Richardson C, Putzig I, et al. Association of participation in an end-of-life conversation game with advance care planning behavior and perspectives among African American individuals. JAMA Netw Open. 2020;3(5):e204315. doi: 10.1001/jamanetworkopen.2020.4315.
    1. Project Talk Trial website [Available from: ]. Accessed 1 Sept 2022.
    1. Creswell JW, Creswell JD. Research design: qualitative, quantitative, and mixed methods approaches. Sage: Sage publications; 2017.
    1. Richardson S, Hirsch JS, Narasimhan M, Crawford JM, McGinn T, Davidson KW, et al. Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area. Jama. 2020;323(20):2052-9.
    1. Federal Register (2019 Poverty Guidelines) Department of Health and Human Services. 2019.
    1. Weiss DS. The impact of event scale: revised. Cross-cultural assessment of psychological trauma and PTSD. Boston: Springer; 2007. p. 219–38.
    1. Rural Health Information Hub [Available from: ]. Accessed 1 Sept 2022.
    1. Van Scoy LJ, Witt PD, Bramble C, Richardson C, Putzig I, Currin L, et al. Success of a community-based delivery at recruiting individuals from underserved communities for an observational cohort study of an advance care planning intervention. J Pain Symptom Manage. 2022;63(2):e149–ee54. doi: 10.1016/j.jpainsymman.2021.09.021.
    1. Starks K. Cognitive behavioral game design: a unified model for designing serious games. Front Psychol. 2014;5:28. doi: 10.3389/fpsyg.2014.00028.
    1. The Conversation Project [October 4, 2018]. Available from: . Accessed 1 Sept 2022.
    1. Table topics: questions to start great conversations [Available from: ]. Accessed 1 Sept 2022.
    1. Green MJ, Levi BH. Development of an interactive computer program for advance care planning. Health Expect. 2009;12(1):60–69. doi: 10.1111/j.1369-7625.2008.00517.x.
    1. Living Voice: Vital Decisions, LLC; [Available from: ]. Accessed 1 Sept 2022.
    1. Shea JA, Micco E, Dean LT, McMurphy S, Schwartz JS, Armstrong K. Development of a revised health care system distrust scale. J Gen Intern Med. 2008;23(6):727–732. doi: 10.1007/s11606-008-0575-3.
    1. Sudore RL, Heyland DK, Barnes DE, Howard M, Fassbender K, Robinson CA, et al. Measuring advance care planning: optimizing the advance care planning engagement survey. J Pain Symptom Manage. 2017;53(4):669–81 e8. doi: 10.1016/j.jpainsymman.2016.10.367.
    1. Hecht ML. The conceptualization and measurement of interpersonal communication satisfaction. Hum Comm Res. 1978;4(3):253–264. doi: 10.1111/j.1468-2958.1978.tb00614.x.
    1. Hecht ML. Measures of communication satisfaction. Hum Comm Res. 1978;4(4):350. doi: 10.1111/j.1468-2958.1978.tb00721.x.
    1. Van Scoy LJ, Darnell W, Watterson T, Chinchilli VM, Wasserman EJ, Wolpaw D, et al. Adapting a small group communication quality assessment to new contexts. International. Res Higher Educ. 2018;3(2):61-75.
    1. Van Scoy LJ, Scott AM, Reading JM, Chuang CH, Chinchilli VM, Levi BH, et al. From theory to practice: measuring end-of-life communication quality using multiple goals theory. Patient Educ Couns. 2017;100(5):909–918. doi: 10.1016/j.pec.2016.12.010.
    1. Caughlin JP. A multiple goals theory of personal relationships: conceptual integration and program overview. J Soc Pers Relationships. 2010;27(6):824. doi: 10.1177/0265407510373262.
    1. Scott AM, Caughlin JP. Enacted goal attention in family conversations about end-of-life health decisions. Commun Monogr. 2014;81(3):261–284. doi: 10.1080/03637751.2014.925568.
    1. Scott AM. A longitudinal examination of enacted goal attention in end-of-life communication in families. Commun Res. 2021; (in press).
    1. Scott AM, Van Scoy LJ. What counts as “good” clinical communication in the coronavirus disease 2019 era and beyond?: ditching checklists for juggling communication goals. Chest. 2020;158(3):879–880. doi: 10.1016/j.chest.2020.05.539.
    1. Scott A. Communication about end-of-life health decisions. Commun Yearbook. 2014;38:242–277.
    1. Fleiss JL, Levin B, Paik MC. Statistical methods for rates and proportions. Hoboken: Wiley; 2003.
    1. Bakeman R, Gottman JM. Observing interaction: an introduction to sequential analysis. Cambridge university press; 1997.
    1. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. doi: 10.1186/1748-5908-4-50.
    1. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O'Neal L, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. 2019;95:103208. doi: 10.1016/j.jbi.2019.103208.
    1. Vonesh E, Chinchilli VM. Linear and nonlinear models for the analysis of repeated measurements. Boca Raton, London, New York: CRC press; 1996.
    1. VanderWeele TJ. A unification of mediation and interaction: a 4-way decomposition. Epidemiology. 2014;25(5):749–761. doi: 10.1097/EDE.0000000000000121.
    1. Morse JM. Critical analysis of strategies for determining rigor in qualitative inquiry. Qual Health Res. 2015;25(9):1212–1222. doi: 10.1177/1049732315588501.
    1. Creswell JW, Inquiry Q. Research design: choosing among five approaches. Thousand Oaks: Sage; 2007.
    1. Moustakas C. Phenomenological research methods. Thousand Oaks: Sage; 1994.
    1. Guba EG, Lincoln YS. Effective evaluation: improving the usefulness of evaluation results through responsive and naturalistic approaches. Thousand Oaks: Jossey-Bass; 1981.
    1. Guba EG. Criteria for assessing the trustworthiness of naturalistic inquiries. ECTJ. 1981;29(2):75–91. doi: 10.1007/BF02766777.
    1. Miles MB, Huberman AM, Saldana J. Qualitative data analysis. Thousand Oaks: Sage; 2013.
    1. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–1288. doi: 10.1177/1049732305276687.
    1. Dye JF, Schatz IM, Rosenberg BA, Coleman ST. Constant comparison method: a kaleidoscope of data. Qualit Rep. 2000;4(1):1–10.
    1. Curry L, Nunez-Smith M. Mixed methods in health sciences research: a practical primer. Thousand Oaks: Sage Publications; 2014.
    1. Lewin S, Glenton C, Oxman AD. Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions: methodological study. BMJ. 2009;339.
    1. O'Cathain A. A practical guide to using qualitative research with randomized controlled trials. Oxford: Oxford University Press; 2018.

Source: PubMed

3
Abonneren