Testing the implementation and sustainment facilitation (ISF) strategy as an effective adjunct to the Addiction Technology Transfer Center (ATTC) strategy: study protocol for a cluster randomized trial

Bryan R Garner, Mark Zehner, Mathew R Roosa, Steve Martino, Heather J Gotham, Elizabeth L Ball, Patricia Stilen, Kathryn Speck, Denna Vandersloot, Traci R Rieckmann, Michael Chaple, Erika G Martin, David Kaiser, James H Ford 2nd, Bryan R Garner, Mark Zehner, Mathew R Roosa, Steve Martino, Heather J Gotham, Elizabeth L Ball, Patricia Stilen, Kathryn Speck, Denna Vandersloot, Traci R Rieckmann, Michael Chaple, Erika G Martin, David Kaiser, James H Ford 2nd

Abstract

Background: Improving the extent to which evidence-based practices (EBPs)-treatments that have been empirically shown to be efficacious or effective-are integrated within routine practice is a well-documented challenge across numerous areas of health. In 2014, the National Institute on Drug Abuse funded a type 2 effectiveness-implementation hybrid trial titled the substance abuse treatment to HIV Care (SAT2HIV) Project. Aim 1 of the SAT2HIV Project tests the effectiveness of a motivational interviewing-based brief intervention (MIBI) for substance use as an adjunct to usual care within AIDS service organizations (ASOs) as part of its MIBI Experiment. Aim 2 of the SAT2HIV Project tests the effectiveness of implementation and sustainment facilitation (ISF) as an adjunct to the Addiction Technology Transfer Center (ATTC) model for training staff in motivational interviewing as part of its ISF Experiment. The current paper describes the study protocol for the ISF Experiment.

Methods: Using a cluster randomized design, case management and leadership staff from 39 ASOs across the United States were randomized to receive either the ATTC strategy (control condition) or the ATTC + ISF strategy (experimental condition). The ATTC strategy is staff-focused and includes 10 discrete strategies (e.g., provide centralized technical assistance, conduct educational meetings, provide ongoing consultation). The ISF strategy is organization-focused and includes seven discrete strategies (e.g., use an implementation advisor, organize implementation team meetings, conduct cyclical small tests of change). Building upon the exploration-preparation-implementation-sustainment (EPIS) framework, the effectiveness of the ISF strategy is examined via three staff-level measures: (1) time-to-proficiency (i.e., preparation phase outcome), (2) implementation effectiveness (i.e., implementation phase outcome), and (3) level of sustainment (i.e., sustainment phase outcome).

Discussion: Although not without limitations, the ISF experiment has several strengths: a highly rigorous design (randomized, hypothesis-driven), high-need setting (ASOs), large sample size (39 ASOs), large geographic representation (23 states and the District of Columbia), and testing along multiple phases of the EPIS continuum (preparation, implementation, and sustainment). Thus, study findings will significantly improve generalizable knowledge regarding the best preparation, implementation, and sustainment strategies for advancing EBPs along the EPIS continuum. Moreover, increasing ASO's capacity to address substance use may improve the HIV Care Continuum. Trial registration ClinicalTrials.gov: NCT03120598.

Keywords: External facilitation; Implementation strategies; Type 2 hybrid trial.

Figures

Fig. 1. Conceptual overview of the ISF…
Fig. 1. Conceptual overview of the ISF experiment within the context of the parent SAT2HIV Project.
Note: MIBI motivational interviewing-based brief intervention; ISF implementation and sustainment facilitation; UC usual care; bolded arrows represent hypothesized relationships; dashed arrows represent interactions and cross-level interactions that will be examined
Fig. 2. Flow of participating AIDS service…
Fig. 2. Flow of participating AIDS service organizations (ASOs)
Note: t time; ATTC Addiction Technology Transfer Center; ISF implementation and sustainment facilitation
Fig. 3
Fig. 3
Potential impacts of the SAT2HIV Project’s ISF experiment

References

    1. Institute of Medicine . Bridging the gap between practice and research: forging partnerships with community-based drug and alcohol treatment. Washington: National Academy Press; 1998.
    1. Institute of Medicine . Crossing the quality chasm: a new health system for the 21st century. Washington: National Academy Press; 2001.
    1. Hogan MF. The President’s New Freedom Commission: recommendations to transform mental health care in America. Psychiatr Serv. 2003;54:1467–1474. doi: 10.1176/appi.ps.54.11.1467.
    1. Institute of Medicine . Improving the quality of health care for mental and substance-use conditions: quality chasm series. Washington: National Academy Press; 2006.
    1. Garner BR. Research on the diffusion of evidence-based treatments within substance abuse treatment: a systematic review. J Subst Abuse Treat. 2009;36:376–399. doi: 10.1016/j.jsat.2008.08.004.
    1. Sweeney S, Obure CD, Maier CB, Greener R, Dehne K, Vassall A. Costs and efficiency of integrating HIV/AIDS services with other health services: a systematic review of evidence and experience. Sex Transm Infect. 2012;88:85–99. doi: 10.1136/sextrans-2011-050199.
    1. Bing EG, Burnam A, Longshore D, Fleishman JA, Sherbourne CD, London AS, et al. Psychiatric disorders and drug use among human immunodeficiency virus-infected adults in the United States. Arch Gen Psychiatry. 2001;58:721–728. doi: 10.1001/archpsyc.58.8.721.
    1. Gaynes BN, Pence BW, Eron JJ, Jr, Miller WC. Prevalence and comorbidity of psychiatric diagnoses based on reference standard in an HIV+ patient population. Psychosom Med. 2008;70:505–511. doi: 10.1097/PSY.0b013e31816aa0cc.
    1. Hartzler B, Dombrowski JC, Crane HM, Eron JJ, Geng EH, Christopher Mathews W, et al. Prevalence and predictors of substance use disorders among HIV care enrollees in the United States. AIDS Behav. 2017;21:1138–1148. doi: 10.1007/s10461-016-1584-6.
    1. Lucas GM, Cheever LW, Chaisson RE, Moore RD. Detrimental effects of continued illicit drug use on the treatment of HIV-1 infection. J Acquir Immune Defic Syndr. 2001;27:251–259. doi: 10.1097/00042560-200107010-00006.
    1. Arnsten JH, Demas PA, Grant RW, Gourevitch MN, Farzadegan H, Howard AA, et al. Impact of active drug use on antiretroviral therapy adherence and viral suppression in HIV-infected drug users. J Gen Intern Med. 2002;17:377–381. doi: 10.1007/s11606-002-0044-3.
    1. King WD, Larkins S, Hucks-Ortiz C, Wang PC, Gorbach PM, Veniegas R, et al. Factors associated with HIV viral load in a respondent driven sample in Los Angeles. AIDS Behav. 2009;13:145–153. doi: 10.1007/s10461-007-9337-1.
    1. Malta M, Strathdee SA, Magnanini MM, Bastos FI. Adherence to antiretroviral therapy for human immunodeficiency virus/acquired immune deficiency syndrome among drug users: a systematic review. Addiction. 2008;103:1242–1257. doi: 10.1111/j.1360-0443.2008.02269.x.
    1. Friedman MS, Marshal MP, Stall R, Kidder DP, Henny KD, Courtenay-Quirk C, et al. Associations between substance use, sexual risk taking and HIV treatment adherence among homeless people living with HIV. AIDS Care. 2009;21:692–700. doi: 10.1080/09540120802513709.
    1. Hendershot CS, Stoner SA, Pantalone DW, Simoni JM. Alcohol use and antiretroviral adherence: review and meta-analysis. J Acquir Immune Defic Syndr. 2009;52:180–202. doi: 10.1097/QAI.0b013e3181b18b6e.
    1. Azar MM, Springer SA, Meyer JP, Altice FL. A systematic review of the impact of alcohol use disorders on HIV treatment outcomes, adherence to antiretroviral therapy and health care utilization. Drug Alcohol Depend. 2010;112:178–193. doi: 10.1016/j.drugalcdep.2010.06.014.
    1. Palepu A, Tyndall M, Yip B, O’Shaughnessy MV, Hogg RS, Montaner JS. Impaired virologic response to highly active antiretroviral therapy associated with ongoing injection drug use. J Acquir Immune Defic Syndr. 2003;32:522–526. doi: 10.1097/00126334-200304150-00009.
    1. National Institute on Drug Abuse, Department of Health and Human Services. .
    1. NIH Research Portfolio Online Reporting Tools (RePORT). Project information. Substance abuse treatment to HIV care (SAT2HIV) (Project No. 7R01DA038146-02). .
    1. Garner BR, Gotham HJ, Tueller SJ, Ball E, Kaiser D, Stilen P, et al. Testing the effectiveness of a motivational interviewing-based brief intervention for substance use as an adjunct to usual care in community-based AIDS service organizations: Study protocol for a multisite randomized controlled trial. Addict Sci Clin Pract. (2017). 10.1186/s13722-017-0095-8.
    1. Chan AW, Tetzlaff JM, Altman DG, Laupacis A, Gotzsche PC, Krleza-Jeric K, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158:200–207. doi: 10.7326/0003-4819-158-3-201302050-00583.
    1. Chan AW, Tetzlaff JM, Gotzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7586. doi: 10.1136/bmj.e7586.
    1. Dunn C, Deroo L, Rivara FP. The use of brief interventions adapted from motivational interviewing across behavioral domains: a systematic review. Addiction. 2001;96:1725–1742. doi: 10.1046/j.1360-0443.2001.961217253.x.
    1. Lundahl BW, Kunz C, Brownell C, Tollefson D, Burke BL. A meta-analysis of motivational interviewing: twenty-five years of empirical studies. Res Soc Work Pract. 2010;20:137–160. doi: 10.1177/1049731509347850.
    1. Vasilaki EI, Hosier SG, Cox WM. The efficacy of motivational interviewing as a brief intervention for excessive drinking: a meta-analytic review. Alcohol Alcohol. 2006;41:328–335. doi: 10.1093/alcalc/agl016.
    1. Martino S, Ball SA, Nich C, Frankforter TL, Carroll KM. Community program therapist adherence and competence in motivational enhancement therapy. Drug Alcohol Depend. 2008;96:37–48. doi: 10.1016/j.drugalcdep.2008.01.020.
    1. Hill S, Kavookjian J. Motivational interviewing as a behavioral intervention to increase HAART adherence in patients who are HIV-positive: a systematic review of the literature. AIDS Care. 2012;24:583–592. doi: 10.1080/09540121.2011.630354.
    1. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38:65–76. doi: 10.1007/s10488-010-0319-7.
    1. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38:4–23. doi: 10.1007/s10488-010-0327-7.
    1. Miller WR, Yahne CE, Moyers TB, Martinez J, Pirritano M. A randomized trial of methods to help clinicians learn motivational interviewing. J Consult Clin Psychol. 2004;72:1050–1062. doi: 10.1037/0022-006X.72.6.1050.
    1. Klein KJ, Conn AB, Sorra JS. Implementing computerized technology: an organizational analysis. J Appl Psychol. 2001;86:811–824. doi: 10.1037/0021-9010.86.5.811.
    1. Helfrich CD, Weiner BJ, McKinney MM, Minasian L. Determinants of implementation effectiveness: adapting a framework for complex innovations. Med Care Res Rev. 2007;64:279–303. doi: 10.1177/1077558707299887.
    1. Klein KJ, Sorra JS. The challenge of innovation implementation. Acad Manag Rev. 1996;21:1055–1080.
    1. Weiner BJ, Lewis MA, Linnan LA. Using organization theory to understand the determinants of effective implementation of worksite health promotion programs. Health Educ Res. 2009;24:292–305. doi: 10.1093/her/cyn019.
    1. Hunter SB, Ayer L, Han B, Garner BR, Godley SH. Examining the sustainment of the adolescent-community reinforcement approach in community addiction treatment settings: protocol for a longitudinal mixed method study. Implement Sci. 2014;9:104. doi: 10.1186/s13012-014-0104-1.
    1. Squires DD, Gumbley SJ, Storti SA. Training substance abuse treatment organizations to adopt evidence-based practices: the Addiction Technology Transfer Center of New England science to service laboratory. J Subst Abuse Treat. 2008;34:293–301. doi: 10.1016/j.jsat.2007.04.010.
    1. Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC, et al. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev. 2012;69:123–157. doi: 10.1177/1077558711430690.
    1. Gustafson DH, Quanbeck AR, Robinson JM, Ford JH, 2nd, Pulvermacher A, French MT, et al. Which elements of improvement collaboratives are most effective? A cluster-randomized trial. Addiction. 2013;108:1145–1157. doi: 10.1111/add.12117.
    1. Preacher KJ, Zhang Z, Zyphur MJ. Multilevel structural equation models for assessing moderation within and across levels of analysis. Psychol Methods. 2016;21:189–205. doi: 10.1037/met0000052.
    1. Blair JM, McNaghten AD, Frazier EL, Skarbinski J, Huang P, Heffelfinger JD. Clinical and behavioral characteristics of adults receiving medical care for HIV infection—Medical Monitoring Project, United States, 2007. MMWR Surveill Summ. 2011;60:1–20.
    1. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8:139. doi: 10.1186/1748-5908-8-139.
    1. Davis D, Barrington T, Phoenix U, Gilliam A, Collins C, Cotton D, et al. Evaluation and technical assistance for successful HIV program delivery. AIDS Educ Prev. 2000;12(Suppl A):115–125.
    1. Wandersman A, Chien VH, Katz J. Toward an evidence-based system for innovation support for implementing innovations with quality: tools, training, technical assistance, and quality assurance/quality improvement. Am J Community Psychol. 2012;50:445–459. doi: 10.1007/s10464-012-9509-7.
    1. Chaple M, Sacks S. The impact of technical assistance and implementation support on program capacity to deliver integrated services. J Behav Health Serv Res. 2016;43:3–17. doi: 10.1007/s11414-014-9419-6.
    1. Farmer AP, Legare F, Turcot L, et al. Printed educational materials: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2008;3:CD004398.
    1. Giguere A, Legare F, Grimshaw J, et al. Printed educational materials: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;10:CD004398.
    1. HealtheKnowledge. On-Demand Courses. .
    1. Martino S, Garner BR, Gotham H, Speck K, Vandersloot D. Motivational Interviewing-based brief intervention (BI) protocol for HIV-Infected Clients with Risky Substance Use. Unpublished treatment manual. 2014.
    1. Godley SH, Garner BR, Smith JE, Meyers RJ, Godley MD. A large-scale dissemination and implementation model for evidence-based treatment and continuing care. Clin Psychol (New York) 2011;18:67–83.
    1. Miller WR, Moyers TB, Arciniega L, Ernst D, Forcehimes A. Training, supervision and quality monitoring of the COMBINE Study behavioral interventions. J Stud Alcohol. 2005;15(Suppl):188–195. doi: 10.15288/jsas.2005.s15.188.
    1. Schneider EC, Zaslavsky AM, Landon BE, Lied TR, Sheingold S, Cleary PD. National quality monitoring of Medicare health plans: the relationship between enrollees’ reports and the quality of clinical care. Med Care. 2001;39:1313–1325. doi: 10.1097/00005650-200112000-00007.
    1. Gibbons CJ, Carroll KM, Ball SA, Nich C, Frankforter TL, Martino S. Community program therapist adherence and competence in a motivational interviewing assessment intake session. Am J Drug Alcohol Abuse. 2010;36:342–349. doi: 10.3109/00952990.2010.500437.
    1. Martino S, Ball S, Nich C, Frankforter TL, Carroll KM. Correspondence of motivational enhancement treatment integrity ratings among therapists, supervisors, and observers. Psychother Res. 2009;19:181–193. doi: 10.1080/10503300802688460.
    1. McCormack L, Sheridan S, Lewis M, Boudewyns V, Melvin CL, Kistler C, et al. Communication and Dissemination Strategies To Facilitate the Use of Health-Related Evidence Report/Technology Assessment No. 213 (prepared by the RTI International-University of North Carolina Evidence-based Practice Center under Contract No. 290-2007-10056-I.) AHRQ Publication No. 13(14)-E003-EF. Rockville, MD: Agency for Healthcare Research and Quality; November 2013. .
    1. Madson MB, Loignon AC, Lane C. Training in motivational interviewing: a systematic review. J Subst Abuse Treat. 2009;36:101–109. doi: 10.1016/j.jsat.2008.05.005.
    1. Lane C, Hood K, Rollnick S. Teaching motivational interviewing: using role play is as effective as using simulated patients. Med Educ. 2008;42:637–644. doi: 10.1111/j.1365-2923.2007.02990.x.
    1. Jamtvedt G, Young JM, Kristoffersen DT, et al. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2006;2:CD000259.
    1. Hysong SJ. Meta-analysis: audit and feedback features impact effectiveness on care quality. Med Care. 2009;47:356–363. doi: 10.1097/MLR.0b013e3181893f6b.
    1. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012:CD000259. 10.1002/14651858.CD000259.pub3.
    1. Colquhoun HL, Brehaut JC, Sales A, Ivers N, Grimshaw J, Michie S, et al. A systematic review of the use of theory in randomized controlled trials of audit and feedback. Implement Sci. 2013;8:66. doi: 10.1186/1748-5908-8-66.
    1. Sholomskas DE, Syracuse-Siewert G, Rounsaville BJ, Ball SA, Nuro KF, Carroll KM. We don’t train in vain: a dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. J Consult Clin Psychol. 2005;73:106–115. doi: 10.1037/0022-006X.73.1.106.
    1. Motivational Interviewing Network of Trainers. Welcome to the motivational interviewing page! Fairfax, VA: Motivational Interviewing Network of Trainers; 2016. . Accessed 10 Jan 2017.
    1. Roosa M, Scripa JS, Zastowny TR, Ford JH., 2nd Using a NIATx based local learning collaborative for performance improvement. Eval Program Plan. 2011;34:390–398. doi: 10.1016/j.evalprogplan.2011.02.006.
    1. Stephan SH, Connors EH, Arora P, et al. A learning collaborative approach to training school-based health providers in evidence-based mental health treatment. Child Youth Serv Rev. 2013;35:1970–1978. doi: 10.1016/j.childyouth.2013.09.008.
    1. Haine-Schlagel R, Brookman-Frazee L, Janis B, Gordon J. Evaluating a learning collaborative to implement evidence-informed engagement strategies in community-based services for young children. Child Youth Care Forum. 2013;42:457–473. doi: 10.1007/s10566-013-9210-5.
    1. Kauth MR, Sullivan G, Blevins D, Cully JA, Landes RD, Said Q, et al. Employing external facilitation to implement cognitive behavioral therapy in VA clinics: a pilot study. Implement Sci. 2010;5:75. doi: 10.1186/1748-5908-5-75.
    1. Kilbourne AM, Abraham KM, Goodrich DE, Bowersox NW, Almirall D, Lai Z, et al. Cluster randomized adaptive implementation trial comparing a standard versus enhanced implementation intervention to improve uptake of an effective re-engagement program for patients with serious mental illness. Implement Sci. 2013;8:136. doi: 10.1186/1748-5908-8-136.
    1. Stetler CB, Legro MW, Rycroft-Malone J, Bowman C, Curran G, Guihan M, et al. Role of “external facilitation” in implementation of research findings: a qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implement Sci. 2006;1:23. doi: 10.1186/1748-5908-1-23.
    1. Cully JA, Armento ME, Mott J, Nadorff MR, Naik AD, Stanley MA, et al. Brief cognitive behavioral therapy in primary care: a hybrid type 2 patient-randomized effectiveness–implementation design. Implement Sci. 2012;7:64. doi: 10.1186/1748-5908-7-64.
    1. Miller K, Dunn D. Using past performance to improve future practice: a framework for method evaluation and improvement. In: Zupancic J, Wojtkowski WG, Wojtkowski W, Wrycza S, editors. Evolution and Challenges in System Development. US: Springer; 1999. pp. 99–107.
    1. Miller WR, Rose GS. Motivational interviewing and decisional balance: contrasting responses to client ambivalence. Behav Cogn Psychother. 2015;43:129–141. doi: 10.1017/S1352465813000878.
    1. Ford JH, 2nd, Green CA, Hoffman KA, Wisdom JP, Riley KJ, Bergmann L, et al. Process improvement needs in substance abuse treatment: admissions walk-through results. J Subst Abuse Treat. 2007;33:379–389. doi: 10.1016/j.jsat.2007.02.003.
    1. Weiner BJ, Belden CM, Bergmire DM, Johnston M. The meaning and measurement of implementation climate. Implement Sci. 2011;6:78. doi: 10.1186/1748-5908-6-78.
    1. Rapp CA, Etzel-Wise D, Marty D, Coffman M, Carlson L, Asher D, et al. Evidence-based practice implementation strategies: results of a qualitative study. Community Ment Health J. 2008;44:213–224. doi: 10.1007/s10597-007-9109-4.
    1. Zwarentein M, Goldman J, Reeves S. Interprofessional collaboration: effects of practice-based interventions on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2009;3:CD000072.
    1. Hysong SJ, Best RG, Pugh JA. Clinical practice guideline implementation strategy patterns in Veterans Affairs primary care clinics. Health Serv Res. 2007;42:84–103. doi: 10.1111/j.1475-6773.2006.00610.x.
    1. Pare G, Sicotte C, Poba-Nzaou P, Balouzakis G. Clinicians’ perceptions of organizational readiness for change in the context of clinical information system projects: insights from two cross-sectional surveys. Implement Sci. 2011;6:15. doi: 10.1186/1748-5908-6-15.
    1. Weiner BJ, Amick H, Lee SY. Conceptualization and measurement of organizational readiness for change: a review of the literature in health services research and other fields. Med Care Res Rev. 2008;65:379–436. doi: 10.1177/1077558708317802.
    1. Weiner BJ. A theory of organizational readiness for change. Implement Sci. 2009;4:67. doi: 10.1186/1748-5908-4-67.
    1. Lehman WE, Greener JM, Simpson DD. Assessing organizational readiness for change. J Subst Abuse Treat. 2002;22:197–209. doi: 10.1016/S0740-5472(02)00233-7.
    1. Helfrich CD, Li YF, Sharp ND, Sales AE. Organizational readiness to change assessment (ORCA): development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework. Implement Sci. 2009;4:38. doi: 10.1186/1748-5908-4-38.
    1. Kajermo KN, Bostrom AM, Thompson DS, Hutchinson AM, Estabrooks CA, Wallin L. The BARRIERS scale—the barriers to research utilization scale: a systematic review. Implement Sci. 2010;5:32. doi: 10.1186/1748-5908-5-32.
    1. Marshall T, Solomon P, Steber SA. Implementing best practice models by using a consensus-building process. Adm Policy Ment Health. 2001;29:105–116. doi: 10.1023/A:1014300329581.
    1. Margerum RD. Collaborative planning—building consensus and building a distinct model for practice. J Plan Educ Res. 2002;21:237–253. doi: 10.1177/0739456X0202100302.
    1. Nembhard IM. Learning and improving in quality improvement collaboratives: which collaborative features do participants value most? Health Serv Res. 2009;44:359–378. doi: 10.1111/j.1475-6773.2008.00923.x.
    1. Nadeem E, Olin SS, Hill LC, et al. Understanding the components of quality improvement collaboratives: A systematic literature review. Milbank. 2013;91:354–394. doi: 10.1111/milq.12016.
    1. Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf. 2014;23:290–298. doi: 10.1136/bmjqs-2013-001862.
    1. Raudenbush SW, et al. Optimal design software for multi-level and longitudinal research (Version 3.01) [Software]. 2011.
    1. . Find HIV testing sites and care services—locator map. .
    1. POZ Magazine. Health services directory. Find your local HIV/AIDS health care and service organizations. Smart + Strong. . Accessed 7 Apr 2015.
    1. Stout RL, Wirtz PW, Carbonari JP, Del Boca FK. Ensuring balanced distribution of prognostic factors in treatment outcome research. J Stud Alcohol. 1994;12(Suppl):70–75. doi: 10.15288/jsas.1994.s12.70.
    1. Charpentier PA. Urn Randomization Program gRand [computer program] 1.10. New Haven: Yale University; 2003.
    1. Martino S, Ball SA, Gallon SL, Hall D, Garcia M, Ceperich S, et al. Motivational interviewing assessment: supervisory tools for enhancing proficiency. Salem, OR: Northwest Frontier Addiction Technology Transfer Center, Oregon Health and Science University; 2006. .
    1. Garner BR, Godley SH, Dennis ML, Hunter B, Bair C, Godley MD. Using pay for performance to improve treatment implementation for adolescent substance use disorders: results from a cluster randomized trial. Arch Pediatr Adolesc Med. 2012;166:938–944. doi: 10.1001/archpediatrics.2012.802.
    1. Figueredo AJ, McKnight PE, McKnight KM, Sidani S. Multivariate modeling of missing data within and across assessment waves. Addiction. 2000;95(Suppl 3):S361–S380. doi: 10.1046/j.1360-0443.95.11s3.6.x.
    1. Little RJA, Rubin DB. The analysis of social science data with missing values. Sociol Meth Res. 1989;18:292–326. doi: 10.1177/0049124189018002004.
    1. Raudenbush SW, Bryk AS, Congdon R. HLM 7.01 for Windows [Computer software] Stokie: Scientific Software International, Inc.; 2013.
    1. . HIV/AIDS Care Continuum. 2015. . Accessed 11 Jan 2017.
    1. Altice FL, Kamarulzaman A, Soriano VV, Schechter M, Friedland GH. Treatment of medical, psychiatric, and substance-use comorbidities in people infected with HIV who use drugs. Lancet. 2010;376:367–387. doi: 10.1016/S0140-6736(10)60829-X.
    1. Gwadz M, de Guzman R, Freeman R, Kutnick A, Silverman E, Leonard NR, et al. Exploring how substance use impedes engagement along the HIV care continuum: a qualitative study. Front Public Health. 2016;4:62.
    1. Martino S, Haeseler F, Belitsky R, Pantalon M, Fortin AH. Teaching brief motivational interviewing to year three medical students. Med Educ. 2007;41:160–167. doi: 10.1111/j.1365-2929.2006.02673.x.
    1. Haeseler F, Fortin AH, Pfeiffer C, Walters C, Martino S. Assessment of a motivational interviewing curriculum for year 3 medical students using a standardized patient case. Patient Educ Couns. 2011;84:27–30. doi: 10.1016/j.pec.2010.10.029.
    1. Prochaska JO, Velicer WF, Rossi JS, Goldstein MG, Marcus BH, Rakowski W, et al. Stages of change and decisional balance for 12 problem behaviors. Health Psychol. 1994;13:39–46. doi: 10.1037/0278-6133.13.1.39.
    1. Prestwich A, Lawton R, Conner M. The use of implementation intentions and the decision balance sheet in promoting exercise behaviour. Psychol Health. 2003;18:707–721. doi: 10.1080/08870440310001594493.
    1. McFarlane WR, McNary S, Dixon L, Hornby H, Cimett E. Predictors of dissemination of family psychoeducation in community mental health centers in Maine and Illinois. Psychiatr Serv. 2001;52:935–942. doi: 10.1176/appi.ps.52.7.935.
    1. Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci. 2014;9:7. doi: 10.1186/1748-5908-9-7.
    1. Jacobs SR, Weiner BJ, Bunger AC. Context matters: measuring implementation climate among individuals and groups. Implement Sci. 2014;9:46. doi: 10.1186/1748-5908-9-46.
    1. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. doi: 10.1186/1748-5908-4-50.

Source: PubMed

3
Suscribir