"Scaling-out" evidence-based interventions to new populations or new health care delivery systems

Gregory A Aarons, Marisa Sklar, Brian Mustanski, Nanette Benbow, C Hendricks Brown, Gregory A Aarons, Marisa Sklar, Brian Mustanski, Nanette Benbow, C Hendricks Brown

Abstract

Background: Implementing treatments and interventions with demonstrated effectiveness is critical for improving patient health outcomes at a reduced cost. When an evidence-based intervention (EBI) is implemented with fidelity in a setting that is very similar to the setting wherein it was previously found to be effective, it is reasonable to anticipate similar benefits of that EBI. However, one goal of implementation science is to expand the use of EBIs as broadly as is feasible and appropriate in order to foster the greatest public health impact. When implementing an EBI in a novel setting, or targeting novel populations, one must consider whether there is sufficient justification that the EBI would have similar benefits to those found in earlier trials.

Discussion: In this paper, we introduce a new concept for implementation called "scaling-out" when EBIs are adapted either to new populations or new delivery systems, or both. Using existing external validity theories and multilevel mediation modeling, we provide a logical framework for determining what new empirical evidence is required for an intervention to retain its evidence-based standard in this new context. The motivating questions are whether scale-out can reasonably be expected to produce population-level effectiveness as found in previous studies, and what additional empirical evaluations would be necessary to test for this short of an entirely new effectiveness trial. We present evaluation options for assessing whether scaling-out results in the ultimate health outcome of interest.

Conclusion: In scaling to health or service delivery systems or population/community contexts that are different from the setting where the EBI was originally tested, there are situations where a shorter timeframe of translation is possible. We argue that implementation of an EBI in a moderately different setting or with a different population can sometimes "borrow strength" from evidence of impact in a prior effectiveness trial. The collection of additional empirical data is deemed necessary by the nature and degree of adaptations to the EBI and the context. Our argument in this paper is conceptual, and we propose formal empirical tests of mediational equivalence in a follow-up paper.

Keywords: Delivery system fixed; Effectiveness; Evidence-based intervention; External validity; Implementation science; Intervention adaptation; Mediational equivalence; Multilevel mediation modeling; Population fixed; Scaling-out; Scaling-up.

Conflict of interest statement

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable

Competing interests

GAA is an Associate Editor and CHB is on the Editorial Board of Implementation Science. All decisions on this paper were made by another editor. NB receives salary support from a subcontract from the University of Chicago that is supported by Gilead, the maker of PrEP, which is mentioned in this paper. The authors declare that they have no other competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures

Fig. 1
Fig. 1
Schematic of scaling-out and implementation and effectiveness domains for evaluation

References

    1. Department of Health and Human Services. Dissemination and implementation research in health (R01) NIH funding opportunity: PAR-16-238. NIH grant funding opportunities; 2016. p. 2017.
    1. Campbell DT. Factors relevant to the validity of experiments in social settings. Psychol Bull. 1957;54:297–312. doi: 10.1037/h0040950.
    1. Cook TD, Campbell DT, Day A. Quasi-experimentation: design & analysis issues for field settings. Boston: Houghton Mifflin; 1979.
    1. Cronbach LJ, Shapiro K. Designing evaluations of educational and social programs. San Francisco: Jossey-Bass; 1982.
    1. Cook TD. Social prevention and the social sciences: theoretical controversies, research problems, and evaluation strategies. Berlin: Walter de Gruyter; 1991. Meta-analysis: its potential for causal description and causal explanation within program evaluation; pp. 245–285.
    1. Brown CH, Beardslee W. Realizing population-level improvements for all children’s cognitive, affective, and behavioral health. Am J Prev Med. 2016;51:S101–S105. doi: 10.1016/j.amepre.2016.07.017.
    1. Klingner JK, Boardman AG, McMaster KL. What does it take to scale-up and sustain evidence-based practices? Except Child. 2013;79:195–211. doi: 10.1177/001440291307900205.
    1. Tomlinson M, Rotheram-Borus MJ, Swartz L, Tsai AC. Scaling-up mHealth: where is the evidence? PLoS Med. 2013;10:e1001382. doi: 10.1371/journal.pmed.1001382.
    1. Klein RA, Ratliff KA, Vianello M, Adams RB, Bahnik S, Bernstein MJ, Bocian K, Brandt MJ, Brooks B, Brumbaugh CC, et al. Investigating variation in replicability: a "many labs" replication project. Soc Psychol. 2014;45:142–152. doi: 10.1027/1864-9335/a000178.
    1. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health Ment Health Serv Res. 2011;38:65–76. doi: 10.1007/s10488-010-0319-7.
    1. Matt G, Brewer A, Sklar M. External validity. In: Baker EL, Peterson PL, McGaw B, editors. International encyclopedia of education, vol. 6. Third ed. Oxford: Academic Press; 2010. p. 521–8.
    1. Mohr DC, Schueller SM, Riley WT, Brown CH, Cuijpers P, Duan N, Kwasny MJ, Stiles-Shields C, Cheung K. Trials of intervention principles: evaluation methods for evolving behavioral intervention technologies. J Med Internet Res. 2015;17:e166. doi: 10.2196/jmir.4391.
    1. MacKinnon DP. Introduction to statistical mediation analysis. New York: Routledge; 2008.
    1. Wellek S. Testing statistical hypotheses of equivalence. Boca Raton: CRC Press; 2010.
    1. Piquero AR, Jennings WG, Diamond B, Farrington DP, Tremblay RE, Welsh BC, Gonzalez JMR. A meta-analysis update on the effects of early family/parent training programs on antisocial behavior and delinquency. J Exp Criminol. 2016;12:229–248. doi: 10.1007/s11292-016-9256-0.
    1. Sandler IN, Wolchik SA, Cruden G, Mahrer NE, Ahn S, Brincks A, Brown CH. Overview of meta-analyses of the prevention of mental health, substance use, and conduct problems. Annu Rev Clin Psychol. 2014;10:243–273. doi: 10.1146/annurev-clinpsy-050212-185524.
    1. Institute of Medicine . National Research Council: preventing mental, emotional, and behavioral disorders among young people: progress and possibilities. Washington, DC: The National Academies Press; 2009.
    1. Brody GH, Murry VM, Kogan SM, Gerrard M, Gibbons FX, Molgaard V, Brown AC, Anderson T, Chen Y-f, Luo Z. The strong African American families program: a cluster-randomized prevention trial of long-term effects and a mediational model. J Consult Clin Psychol. 2006;74:356–366. doi: 10.1037/0022-006X.74.2.356.
    1. Forgatch MS, Patterson GR, Gewirtz AH. Looking forward: the promise of widespread implementation of parent training programs. Perspect Psychol Sci. 2013;8:682–694. doi: 10.1177/1745691613503478.
    1. Hurlburt M, Aarons GA, Fettes D, Willging C, Gunderson L, Chaffin MJ. Interagency collaborative team model for capacity building to scale-up evidence-based practice. Child Youth Serv Rev. 2014;39:160–168. doi: 10.1016/j.childyouth.2013.10.005.
    1. Pantin H, Prado G, Lopez B, Huang S, Tapia MI, Schwartz SJ, Sabillon E, Brown CH, Branchini J. A randomized controlled trial of Familias Unidas for Hispanic adolescents with behavior problems. Psychosom Med. 2009;71:987–995. doi: 10.1097/PSY.0b013e3181bb2913.
    1. Prado G, Huang S, Maldonado-Molina M, Bandiera F, Schwartz SJ, de la Vega P, Brown CH, Pantin H. An empirical test of ecodevelopmental theory in predicting HIV risk behaviors among Hispanic youth. Health Educ Behav. 2010;37:97–114. doi: 10.1177/1090198109349218.
    1. Prado G, Pantin H, Huang S, Cordova D, Tapia MI, Velazquez MR, Calfee M, Malcolm S, Arzon M, Villamar J, et al. Effects of a family intervention in reducing HIV risk behaviors among high-risk Hispanic adolescents: a randomized controlled trial. Arch Pediatr Adolesc Med. 2012;166:127–133. doi: 10.1001/archpediatrics.2011.189.
    1. Perrino T, Pantin H, Prado G, Huang S, Brincks A, Howe G, Beardslee W, Sandler I, Brown CH. Preventing internalizing symptoms among Hispanic adolescents: a synthesis across Familias Unidas trials. Prev Sci. 2014;15:917–928. doi: 10.1007/s11121-013-0448-9.
    1. Huang S, MacKinnon DP, Perrino T, Gallo CG, Cruden G, Brown CH. A statistical method for synthesizing mediation analyses using the product of coefficient approach across multiple trials. Stat Methods Appl. 2016;25:565–579. doi: 10.1007/s10260-016-0354-y.
    1. Patient Protection and Affordable Care Act: USC § 18001 et seq.; 2010.
    1. Leslie LK, Mehus CJ, Hawkins JD, Boat T, McCabe MA, Barkin S, Perrin EC, Metzler CW, Prado G, Tait VF. Primary health care: potential home for family-focused preventive interventions. Am J Prev Med. 2016;51:S106–S118. doi: 10.1016/j.amepre.2016.05.014.
    1. Kemper AR, Mabry-Hernandez IR, Grossman DC. U.S. preventive services task force approach to child cognitive and behavioral health. Am J Prev Med. 2016;51:S119–S123. doi: 10.1016/j.amepre.2016.05.016.
    1. Asarnow JR, Rozenman M, Wiblin J, Zeltzer L. Integrated medical-behavioral care compared with usual primary care for child and adolescent behavioral health: a meta-analysis. JAMA Pediatr. 2015;169:929–937. doi: 10.1001/jamapediatrics.2015.1141.
    1. U.S. Public Health Service . In: Preexposure prophylaxis for the prevention of HIV infection in the United States - 2014: a clinical practice guideline. Services USDoHaH, editor. Atlanta: Centers for Disease Control and Prevention; 2014.
    1. Weiss G. Implementing PrEP in STD clinics: findings from a 2015 assessment of local health department engagement in PrEP implementation. In: 2016 National STD Prevention Conference. Atlanta: CDC; 2016.
    1. Barrera M, Berkel C, Castro FG. Directions for the advancement of culturally adapted preventive interventions: local adaptations, engagement, and sustainability. Prev Sci. 2016;18:640–8. doi: 10.1007/s11121-016-0705-9.
    1. Bernal GE, Domenech Rodríguez MM. Cultural adaptations: tools for evidence-based practice with diverse populations. Washington, DC: American Psychological Association; 2012.
    1. Wingood GM, DiClemente RJ. The ADAPT-ITT model: a novel method of adapting evidence-based HIV interventions. JAIDS J Acquir Immune Defic Syndr. 2008;47:S40–S46. doi: 10.1097/QAI.0b013e3181605df1.
    1. Culp L, Caucci L. State adolescent consent laws and implications for HIV pre-exposure prophylaxis. Am J Prev Med. 2013;44:S119–S124. doi: 10.1016/j.amepre.2012.09.044.
    1. Mustanski B, Fisher CB. HIV rates are increasing in gay/bisexual teens: IRB barriers to research must be resolved to bend the curve. Am J Prev Med. 2016;51:249–252. doi: 10.1016/j.amepre.2016.02.026.
    1. Hosek S, Celum C, Wilson CM, Kapogiannis B, Delany-Moretlwe S, Bekker L-G. Preventing HIV among adolescents with oral PrEP: observations and challenges in the United States and South Africa. J Int AIDS Soc. 2016;19(7(Suppl 6)):21107.
    1. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89:1322–1327. doi: 10.2105/AJPH.89.9.1322.
    1. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50:217–226. doi: 10.1097/MLR.0b013e3182408812.
    1. Chambers DA, Glasgow R, Stange K. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8:117. doi: 10.1186/1748-5908-8-117.
    1. Proctor E, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8:139–150. doi: 10.1186/1748-5908-8-139.
    1. Berkel C, Mauricio AM, Schoenfelder E, Sandler IN. Putting the pieces together: an integrated model of program implementation. Prev Sci. 2011;12:23–33. doi: 10.1007/s11121-010-0186-1.
    1. Berkel C, Sandler IN, Wolchik SA, Brown CH, Gallo CG, Chiapa A, Mauricio AM, Jones S. Prevention science. 2016. “home practice is the program”: Parents' practice of program skills as predictors of outcomes in the new beginnings program effectiveness trial.
    1. Prado G, Cordova D, Cano N, Arzon M, Pantin H, Brown CH. Drug abuse preventive interventions for Hispanic youth: State of the science and implications for future research. In: Drug Use Trajectories Among Minority Youth. Dordrecht: Springer; 2016. p. 347–65.
    1. Estrada Y, Rosen A, Huang S, Tapia M, Sutton M, Willis L, Quevedo A, Condo C, Vidot DC, Pantin H. Efficacy of a brief intervention to reduce substance use and human immunodeficiency virus infection risk among Latino youth. J Adolesc Health. 2015;57:651–657. doi: 10.1016/j.jadohealth.2015.07.006.
    1. Brincks A, Perrino T, Howe G, Pantin H, Prado G, Huang S, Cruden G, Brown CH. Preventing youth internalizing symptoms through the Familias Unidas intervention: examining variation in response. Prev Sci. 2016. .
    1. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health Ment Health Serv Res. 2011;38:4–23. doi: 10.1007/s10488-010-0327-7.
    1. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:1. doi: 10.1186/1748-5908-4-50.
    1. Aarons GA, Green AE, Palinkas LA, Self-Brown S, Whitaker DJ, Lutzker JR, Silovsky JF, Hecht DB, Chaffin MJ. Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implement Sci. 2012;7:32. doi: 10.1186/1748-5908-7-32.
    1. Aarons GA, Palinkas LA. Implementation of evidence-based practice in child welfare: service provider perspectives. Adm Policy Ment Health Ment Health Serv Res. 2007;34:411–419. doi: 10.1007/s10488-007-0121-3.
    1. Backer TE. The failure of success: challenges of disseminating effective substance abuse prevention programs. J Community Psychol. 2000;28:363–373. doi: 10.1002/(SICI)1520-6629(200005)28:3<363::AID-JCOP10>;2-T.
    1. Kendall PC, Gosch E, Furr JM, Sood E. Flexibility within fidelity. J Am Acad Child Adolesc Psychiatry. 2008;47:987–993. doi: 10.1097/CHI.0b013e31817eed2f.
    1. Rier DA, Indyk D. Flexible rigidity: supporting HIV treatment adherence in a rapidly-changing treatment environment. Soc Work Health Care. 2006;42:133–150. doi: 10.1300/J010v42n03_09.
    1. Larsen T, Samdal O. Implementing second step: balancing fidelity and program adaptation. J Educ Psychol Consult. 2007;17:1–29. doi: 10.1080/10474410709336588.
    1. Dimeff LA, Koerner KE. Dialectical behavior therapy in clinical practice: applications across disorders and settings. New York: Guilford Press; 2007.
    1. McIntosh VV, Jordan J, McKenzie JM, Luty SE, Carter FA, Carter JD, Frampton CM, Joyce PR. Measuring therapist adherence in psychotherapy for anorexia nervosa: scale adaptation, psychometric properties, and distinguishing psychotherapies. Psychother Res. 2005;15:339–344. doi: 10.1080/10503300500091124.
    1. Dariotis JK, Bumbarger BK, Duncan LG, Greenberg MT. How do implementation efforts relate to program adherence? Examining the role of organizational, implementer, and program factors. J Community Psychol. 2008;36:744–760. doi: 10.1002/jcop.20255.
    1. Castro FG, Barrera M, Jr, Martinez CR., Jr The cultural adaptation of prevention interventions: resolving tensions between fidelity and fit. Prev Sci. 2004;5:41–45. doi: 10.1023/B:.
    1. Kelly JA, Heckman TG, Stevenson LY, Williams PN, Ertl T, Hays RB, Leonard NR, O'Donnell L, Terry MA, Sogolow ED, Neumann MS. Transfer of research-based HIV prevention interventions to community service providers: fidelity and adaptation. AIDS Educ Prev. 2000;12:87–98.
    1. Stirman SW, Gutner CA, Langdon K, Graham JR. Bridging the gap between research and practice in mental health service settings: An overview of developments in implementation theory and research. Behav Ther. 2016;47(6):920–36.
    1. Kemp L. Adaption and fidelity: a recipe analogy for achieving both in population scale implementation. Prev Sci. 2016;17:429–438. doi: 10.1007/s11121-016-0642-7.
    1. Chambers DA, Norton WE. The adaptome: advancing the science of intervention adaptation. Am J Prev Med. 2016;51:S124–S131. doi: 10.1016/j.amepre.2016.05.011.
    1. Stirman SW, Miller CJ, Toder K, Calloway A. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implement Sci. 2013;8:65. doi: 10.1186/1748-5908-8-65.
    1. Arvey SR, Fernandez ME. Identifying the core elements of effective community health worker programs: a research agenda. Am J Public Health. 2012;102:1633–1637. doi: 10.2105/AJPH.2012.300649.
    1. Beckmann J, Hagemann U, Bahri P, Bate A, Boyd IW, Dal Pan GJ, Edwards BD, Edwards IR, Hartigan-Go K, Lindquist M, et al. Teaching pharmacovigilance: the WHO-ISoP core elements of a comprehensive modular curriculum. Drug Saf. 2014;37:743–759. doi: 10.1007/s40264-014-0216-1.
    1. MacNeela P, Morris R, Scott A, Treacy MP, Hyde A. Seen as core: a Delphi consensus study of essential elements of mental health nursing care in Ireland. Arch Psychiatr Nurs. 2010;24:339–348. doi: 10.1016/j.apnu.2010.02.001.
    1. Brown CH, Brincks A, Huang S, Perrino T, Cruden G, Pantin H, Howe G, Young JF, Beardslee W, Montag S, Sandler I: Two-year impact of prevention programs on adolescent depression: an integrative data analysis approach. Prev Sci. In press. Pre-print: 10.1007/s11121-016-0737-1.
    1. Hawkins JD, Oesterle S, Brown EC, Abbott RD, Catalano RF. Youth problem behaviors 8 years after implementing the communities that care prevention system: a community-randomized trial. JAMA Pediatr. 2014;168:122–129. doi: 10.1001/jamapediatrics.2013.4009.
    1. Hawkins JD, Catalano RF., Jr . Communities that care: action for drug abuse prevention. San Francisco: Jossey-Bass; 1992.
    1. Fagan AA, Hanson K, Hawkins JD, Arthur MW. Translational research in action: implementation of the communities that care prevention system in 12 communities. J Community Psychol. 2009;37:809–829. doi: 10.1002/jcop.20332.
    1. Brown EC, Hawkins JD, Arthur MW, Briney JS, Fagan AA. Prevention service system transformation using communities that care. J Community Psychol. 2011;39:183–201. doi: 10.1002/jcop.20426.
    1. Brown E, Hawkins JD, Rhew I, Shapiro V, Abbott R, Oesterle S, Arthur M, Briney J, Catalano R. Prevention system mediation of communities that care effects on youth outcomes. Prev Sci. 2014;15:623–632. doi: 10.1007/s11121-013-0413-7.
    1. Ezeanolue EE, Obiefune MC, Yang W, Obaro SK, Ezeanolue CO, Ogedegbe GG. Comparative effectiveness of congregation- versus clinic-based approach to prevention of mother-to-child HIV transmission: study protocol for a cluster randomized controlled trial. Implement Sci. 2013;8:62. doi: 10.1186/1748-5908-8-62.
    1. MacKinnon DP. Introduction to statistical mediation analysis. Mahwah, NJ: Lawrence Erlbaum Associates; 2008.
    1. MacKinnon DP, Kisbu-Sakarya Y, Gottschall AC. 16 developments in mediation analysis. The Oxford Handbook of Quantitative Methods in Psychology: Vol 2: Statistical Analysis. 2013;2:338.
    1. Anselmi L, Binyaruka P, Borghi J. Understanding causal pathways within health systems policy evaluation through mediation analysis: an application to payment for performance (P4P) in Tanzania. Implement Sci. 2017;12:10. doi: 10.1186/s13012-016-0540-1.
    1. VanderWeele TJ. Mediation analysis: a practitioner's guide. Annu Rev Public Health. 2016;37:17–32. doi: 10.1146/annurev-publhealth-032315-021402.
    1. Imai K, Keele L, Tingley D, Yamamoto T. Unpacking the black box of causality: learning about causal mechanisms from experimental and observational studies. Am Pol Sci Rev. 2011;105:765–789. doi: 10.1017/S0003055411000414.
    1. Wang C-P, Jo B, Brown CH. Causal inference in longitudinal comparative effectiveness studies with repeated measures of a continuous intermediate variable. Stat Med. 2014;33:3509–3527. doi: 10.1002/sim.6120.
    1. Gonzalez O, MacKinnon DP. A bifactor approach to model multifaceted constructs in statistical mediation analysis. Educ Psychol Meas. 2016. .
    1. Czaja SJ, Valente TW, Nair SN, Villamar J, Brown CH. Characterizing implementation strategies using a systems engineering survey and interview tool: a comparison across 10 prevention programs for drug abuse and HIV sexual risk behaviors. Implement Sci. 2016;11:70. doi: 10.1186/s13012-016-0433-3.
    1. Weiner BJ, Belden CM, Bergmire DM, Johnston M. The meaning and measurement of implementation climate. Implement Sci. 2011;6:78. doi: 10.1186/1748-5908-6-78.
    1. Eaton J, McCay L, Semrau M, Chatterjee S, Baingana F, Araya R, Ntulo C, Thornicroft G, Saxena S. Scale-up of services for mental health in low-income and middle-income countries. Lancet. 2011;378:1592–1603. doi: 10.1016/S0140-6736(11)60891-X.
    1. Paina L, Peters DH. Understanding pathways for scaling-up health services through the lens of complex adaptive systems. Health Policy Plan. 2012;27:365–373. doi: 10.1093/heapol/czr054.
    1. Mangham LJ, Hanson K. Scaling-up in international health: what are the key issues? Health Policy Plan. 2010;25:85–96. doi: 10.1093/heapol/czp066.
    1. Chamberlain P, Brown CH, Saldana L. Observational measure of implementation progress in community based settings: the stages of implementation completion (SIC) Implement Sci. 2011;6:116. doi: 10.1186/1748-5908-6-116.
    1. Saldana L, Chamberlain P, Wang W, Brown CH. Predicting program start-up using the stages of implementation measure. Adm Policy Ment Health Ment Health Serv Res. 2012;39:419–425. doi: 10.1007/s10488-011-0363-y.
    1. Brown CH, Chamberlain P, Saldana L, Padgett C, Wang W, Cruden G. Evaluation of two implementation strategies in 51 child county public service systems in two states: results of a cluster randomized head-to-head implementation trial. Implement Sci. 2014;9:134. doi: 10.1186/s13012-014-0134-8.
    1. Wang D, Ogihara M, Gallo CG, Villamar J, Smith JD, Vermeer W, Cruden G, Benbow N, Brown CH. Automatic classification of communication logs into implementation stages via text analysis. Implement Sci. 2016;11:119. doi: 10.1186/s13012-016-0483-6.
    1. Brown CH, Mohr DC, Gallo CG, Mader C, Palinkas L, Wingood G, Prado G, Kellam SG, Pantin H, Poduska J, et al. A computational future for preventing HIV in minority communities: how advanced technology can improve implementation of effective programs. J Acquir Immune Defic Syndr. 2013;63:S72–S84. doi: 10.1097/QAI.0b013e31829372bd.
    1. Perrino T, Beardslee W, Bernal G, Brincks A, Cruden G, Howe G, Murry V, Pantin H, Prado G, Sandler I. Toward scientific equity for the prevention of depression and depressive symptoms in vulnerable youth. Prev Sci. 2015;16:642–651. doi: 10.1007/s11121-014-0518-7.

Source: PubMed

Sponsor e collaboratori

Condizioni mediche

Interventi farmacologici

3
Sottoscrivi