Development and Evaluation of a Mobile App Designed to Increase HIV Testing and Pre-exposure Prophylaxis Use Among Young Men Who Have Sex With Men in the United States: Open Pilot Trial

Katie B Biello, Jonathan Hill-Rorie, Pablo K Valente, Donna Futterman, Patrick S Sullivan, Lisa Hightow-Weidman, Kathryn Muessig, Julian Dormitzer, Matthew J Mimiaga, Kenneth H Mayer, Katie B Biello, Jonathan Hill-Rorie, Pablo K Valente, Donna Futterman, Patrick S Sullivan, Lisa Hightow-Weidman, Kathryn Muessig, Julian Dormitzer, Matthew J Mimiaga, Kenneth H Mayer

Abstract

Background: HIV disproportionately affects young men who have sex with men (YMSM) in the United States. Uptake of evidence-based prevention strategies, including routine HIV testing and use of pre-exposure prophylaxis (PrEP), is suboptimal in this population. Novel methods for reaching YMSM are required.

Objective: The aim of this study is to describe the development and evaluate the feasibility and acceptability of the MyChoices app, a mobile app designed to increase HIV testing and PrEP use among YMSM in the United States.

Methods: Informed by the social cognitive theory, the MyChoices app was developed using an iterative process to increase HIV testing and PrEP uptake among YMSM. In 2017, beta theater testing was conducted in two US cities to garner feedback (n=4 groups; n=28 YMSM). These findings were used to refine MyChoices, which was then tested for initial acceptability and usability in a technical pilot (N=11 YMSM). Baseline and 2-month postbaseline assessments and exit interviews were completed. Transcripts were coded using a deductive approach, and thematic analysis was used to synthesize data; app acceptability and use data were also reported.

Results: The MyChoices app includes personalized recommendations for HIV testing frequency and PrEP use; information on types of HIV tests and PrEP; ability to search for nearby HIV testing and PrEP care sites; and ability to order free home HIV and sexually transmitted infection test kits, condoms, and lube. In theater testing, YMSM described that MyChoices appears useful and that they would recommend it to peers. Participants liked the look and feel of the app and believed that the ability to search for and be pinged when near an HIV testing site would be beneficial. Some suggested that portions of the app felt repetitive and preferred using casual language rather than formal or medicalized terms. Following theater testing, the MyChoices app was refined, and participants in the technical pilot used the app, on average, 8 (SD 5.0; range 2-18) times over 2 months, with an average duration of 28 (SD 38.9) minutes per session. At the 2-month follow-up, the mean System Usability Scale (0-100) score was 71 (ie, above average; SD 11.8). Over 80% (9/11) of the participants reported that MyChoices was useful and 91% (10/11) said that they would recommend it to a friend. In exit interviews, there was a high level of acceptability for the content, interface, and features.

Conclusions: These data show the initial acceptability and user engagement of the MyChoices app. If future studies demonstrate efficacy in increasing HIV testing and PrEP uptake, the app is scalable to reach YMSM across the United States.

Trial registration: Clinicaltrials.gov NCT03179319; https://ichgcp.net/clinical-trials-registry/NCT03179319.

International registered report identifier (irrid): RR2-10.2196/10694.

Keywords: HIV; mHealth; men who have sex with men; mobile apps; mobile phone; pilot study; pre-exposure prophylaxis.

Conflict of interest statement

Conflicts of Interest: None declared.

©Katie B Biello, Jonathan Hill-Rorie, Pablo K Valente, Donna Futterman, Patrick S Sullivan, Lisa Hightow-Weidman, Kathryn Muessig, Julian Dormitzer, Matthew J Mimiaga, Kenneth H Mayer. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 24.03.2021.

Figures

Figure 1
Figure 1
MyChoices app screenshots.

References

    1. HIV Surveillance Report, 2018 (Updated Edition) Centers for Disease Control and Prevention. 2020. May, [2020-06-05]. .
    1. Zanoni BC, Mayer KH. The Adolescent and Young Adult HIV Cascade of Care in the United States: Exaggerated Health Disparities. AIDS Patient Care and STDs. 2014 Mar;28(3):128–135. doi: 10.1089/apc.2013.0345.
    1. Wejnert C, Hess KL, Rose CE, Balaji A, Smith JC, Paz-Bailey G. Age-Specific Race and Ethnicity Disparities in HIV Infection and Awareness Among Men Who Have Sex With Men—20 US Cities, 2008–2014. J Infect Dis. 2015 Oct 20;213(5):776–783. doi: 10.1093/infdis/jiv500.
    1. Kapogiannis B, Koenig L, Xu J, Mayer K, Loeb J, Greenberg L, Monte Dina, Banks-Shields Marinna, Fortenberry J Dennis, Adolescent Medicine Trials Network for HIV/AIDS Interventions The HIV Continuum of Care for Adolescents and Young Adults Attending 13 Urban US HIV Care Centers of the NICHD-ATN-CDC-HRSA SMILE Collaborative. J Acquir Immune Defic Syndr. 2020 May 01;84(1):92–100. doi: 10.1097/QAI.0000000000002308.
    1. Monitoring selected national HIV prevention and care objectives by using HIV surveillance data: United States and 6 dependent areas, 2017. Centers for Disease Control and Prevention. 2019. Jun, [2020-06-05]. .
    1. Li Z, Purcell DW, Sansom SL, Hayes D, Hall HI. HIV Transmission Along the Continuum of Care — United States, 2016. MMWR Morb. Mortal. Wkly. Rep. 2019 Mar 22;68(11):267–272. doi: 10.15585/mmwr.mm6811e1.
    1. Grant Robert M, Lama Javier R, Anderson Peter L, McMahan Vanessa, Liu Albert Y, Vargas Lorena, Goicochea Pedro, Casapía Martín, Guanira-Carranza Juan Vicente, Ramirez-Cardich Maria E, Montoya-Herrera Orlando, Fernández Telmo, Veloso Valdilea G, Buchbinder Susan P, Chariyalertsak Suwat, Schechter Mauro, Bekker Linda-Gail, Mayer Kenneth H, Kallás Esper Georges, Amico K Rivet, Mulligan Kathleen, Bushman Lane R, Hance Robert J, Ganoza Carmela, Defechereux Patricia, Postle Brian, Wang Furong, McConnell J Jeff, Zheng Jia-Hua, Lee Jeanny, Rooney James F, Jaffe Howard S, Martinez Ana I, Burns David N, Glidden David V, iPrEx Study Team Preexposure chemoprophylaxis for HIV prevention in men who have sex with men. N Engl J Med. 2010 Dec 30;363(27):2587–99. doi: 10.1056/NEJMoa1011205.
    1. Owens DK, Davidson KW, Krist AH, Barry MJ, Cabana M, Caughey AB, Curry SJ, Doubeni CA, Epling JW, Kubik M, Landefeld CS, Mangione CM, Pbert L, Silverstein M, Simon MA, Tseng C, Wong JB. Preexposure Prophylaxis for the Prevention of HIV Infection. JAMA. 2019 Jun 11;321(22):2203. doi: 10.1001/jama.2019.6390.
    1. US Preventive Services Task Force. Owens Douglas K, Davidson Karina W, Krist Alex H, Barry Michael J, Cabana Michael, Caughey Aaron B, Curry Susan J, Doubeni Chyke A, Epling John W, Kubik Martha, Landefeld C Seth, Mangione Carol M, Pbert Lori, Silverstein Michael, Simon Melissa A, Tseng Chien-Wen, Wong John B. Screening for HIV Infection: US Preventive Services Task Force Recommendation Statement. JAMA. 2019 Jun 18;321(23):2326–2336. doi: 10.1001/jama.2019.6587.
    1. Phillips A, Cambiano V, Miners A, Lampe F, Rodger A, Nakagawa F, Brown Alison, Gill O Noel, De Angelis Daniela, Elford Jonathan, Hart Graham, Johnson Anne M, Lundgren Jens D, Collins Simon, Delpech Valerie. Potential impact on HIV incidence of higher HIV testing rates and earlier antiretroviral therapy initiation in MSM. AIDS. 2015 Sep 10;29(14):1855–62. doi: 10.1097/QAD.0000000000000767.
    1. MacFadden DR, Tan DH, Mishra S. Optimizing HIV pre-exposure prophylaxis implementation among men who have sex with men in a large urban centre: a dynamic modelling study. J Int AIDS Soc. 2016 Sep 23;19(1):20791. doi: 10.7448/IAS.19.1.20791.
    1. Cohen Myron S, Chen Ying Q, McCauley Marybeth, Gamble Theresa, Hosseinipour Mina C, Kumarasamy Nagalingeswaran, Hakim James G, Kumwenda Johnstone, Grinsztejn Beatriz, Pilotto Jose H S, Godbole Sheela V, Mehendale Sanjay, Chariyalertsak Suwat, Santos Breno R, Mayer Kenneth H, Hoffman Irving F, Eshleman Susan H, Piwowar-Manning Estelle, Wang Lei, Makhema Joseph, Mills Lisa A, de Bruyn Guy, Sanne Ian, Eron Joseph, Gallant Joel, Havlir Diane, Swindells Susan, Ribaudo Heather, Elharrar Vanessa, Burns David, Taha Taha E, Nielsen-Saines Karin, Celentano David, Essex Max, Fleming Thomas R, HPTN 052 Study Team Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med. 2011 Aug 11;365(6):493–505. doi: 10.1056/NEJMoa1105243.
    1. McCormack S, Dunn DT, Desai M, Dolling DI, Gafos M, Gilson R, Sullivan AK, Clarke A, Reeves I, Schembri G, Mackie N, Bowman C, Lacey CJ, Apea V, Brady M, Fox J, Taylor S, Antonucci S, Khoo SH, Rooney J, Nardone A, Fisher M, McOwan A, Phillips AN, Johnson AM, Gazzard B, Gill ON. Pre-exposure prophylaxis to prevent the acquisition of HIV-1 infection (PROUD): effectiveness results from the pilot phase of a pragmatic open-label randomised trial. The Lancet. 2016 Jan;387(10013):53–60. doi: 10.1016/S0140-6736(15)00056-2.
    1. DiNenno EA, Prejean J, Irwin K, Delaney KP, Bowles K, Martin T, Tailor A, Dumitru G, Mullins MM, Hutchinson AB, Lansky A. Recommendations for HIV Screening of Gay, Bisexual, and Other Men Who Have Sex with Men — United States, 2017. MMWR Morb. Mortal. Wkly. Rep. 2017 Aug 11;66(31):830–832. doi: 10.15585/mmwr.mm6631a3.
    1. Sanchez TH, Sineath RC, Kahle EM, Tregear SJ, Sullivan PS. The Annual American Men's Internet Survey of Behaviors of Men Who Have Sex With Men in the United States: Protocol and Key Indicators Report 2013. JMIR Public Health Surveill. 2015 Apr 17;1(1):e3. doi: 10.2196/publichealth.4314.
    1. Krakower DS, Mimiaga MJ, Rosenberger JG, Novak DS, Mitty JA, White JM, Mayer KH. Limited Awareness and Low Immediate Uptake of Pre-Exposure Prophylaxis among Men Who Have Sex with Men Using an Internet Social Networking Site. PLoS One. 2012 Mar 28;7(3):e33119. doi: 10.1371/journal.pone.0033119.
    1. Bauermeister J, Meanley S, Pingel E, Soler J, Harper G. PrEP awareness and perceived barriers among single young men who have sex with men. Curr HIV Res. 2013 Oct 31;11(7):520–7. doi: 10.2174/1570162x12666140129100411.
    1. Shover CL, Javanbakht M, Shoptaw S, Bolan RK, Lee S, Parsons JT, Rendina J, Gorbach PM. HIV Preexposure Prophylaxis Initiation at a Large Community Clinic: Differences Between Eligibility, Awareness, and Uptake. Am J Public Health. 2018 Oct;108(10):1408–1417. doi: 10.2105/ajph.2018.304623.
    1. Huang YA, Zhu W, Smith DK, Harris N, Hoover KW. HIV Preexposure Prophylaxis, by Race and Ethnicity - United States, 2014-2016. MMWR Morb Mortal Wkly Rep. 2018 Oct 19;67(41):1147–1150. doi: 10.15585/mmwr.mm6741a3. doi: 10.15585/mmwr.mm6741a3.
    1. Coy KC, Hazen RJ, Kirkham HS, Delpino A, Siegler AJ. Persistence on HIV preexposure prophylaxis medication over a 2-year period among a national sample of 7148 PrEP users, United States, 2015 to 2017. J Int AIDS Soc. 2019 Feb 18;22(2):e25252. doi: 10.1002/jia2.25252.
    1. Doblecki-Lewis S, Liu A, Feaster D, Cohen S, Elion R, Bacon O, Coleman Megan, Cardenas Gabriel, Kolber Michael A. Patterns and Correlates of Participant Retention in a Multi-City Pre-Exposure Prophylaxis Demonstration Project. J Acquir Immune Defic Syndr. 2018 Sep 01;79(1):62–69. doi: 10.1097/QAI.0000000000001724.
    1. Serota D, Rosenberg E, Sullivan P, Thorne A, Rolle C, Del Rio Carlos, Cutro Scott, Luisi Nicole, Siegler Aaron J, Sanchez Travis H, Kelley Colleen F. Pre-exposure Prophylaxis Uptake and Discontinuation Among Young Black Men Who Have Sex With Men in Atlanta, Georgia: A Prospective Cohort Study. Clin Infect Dis. 2020 Jul 27;71(3):574–582. doi: 10.1093/cid/ciz894.
    1. Hosek S, Rudy B, Landovitz R, Kapogiannis B, Siberry G, Rutledge B, Liu Nancy, Brothers Jennifer, Mulligan Kathleen, Zimet Gregory, Lally Michelle, Mayer Kenneth H, Anderson Peter, Kiser Jennifer, Rooney James F, Wilson Craig M, Adolescent Trials Network (ATN) for HIVAIDS Interventions An HIV Preexposure Prophylaxis Demonstration Project and Safety Study for Young MSM. J Acquir Immune Defic Syndr. 2017 Jan 01;74(1):21–29. doi: 10.1097/QAI.0000000000001179.
    1. Arnett J. Emerging adulthood: The winding road from the late teens through the twenties. New York, New York: Oxford University Press; 2014.
    1. Gruber H, Voneche J, editors. The Essential Piaget. New York, New York: Basic Books; 1977.
    1. Horvath KJ, Rosser BS, Remafedi G. Sexual Risk Taking Among Young Internet-Using Men Who Have Sex With Men. Am J Public Health. 2008 Jun;98(6):1059–1067. doi: 10.2105/ajph.2007.111070.
    1. Teva I, Bermúdez MP, Buela-Casal G. Sexual Sensation Seeking, Social Stress, and Coping Styles as Predictors of HIV/STD Risk Behaviors in Adolescents. Youth & Society. 2009 Nov 23;42(2):255–277. doi: 10.1177/0044118x09353572.
    1. Smartphone access nearly ubiquitous among teens, while having a home computer varies by income. Pew Research Center. 2018. [2020-06-05].
    1. Muessig KE, Nekkanti M, Bauermeister J, Bull S, Hightow-Weidman LB. A systematic review of recent smartphone, Internet and Web 2.0 interventions to address the HIV continuum of care. Curr HIV/AIDS Rep. 2015 Mar 28;12(1):173–90. doi: 10.1007/s11904-014-0239-3.
    1. Catalani C, Philbrick W, Fraser H, Mechael ,P, Israelski DM. Open AIDS J. 2013 Sep 20;7(1):17–41. doi: 10.2174/1874613620130812003.
    1. Maloney KM, Bratcher A, Wilkerson R, Sullivan PS. Electronic and other new media technology interventions for HIV care and prevention: a systematic review. J Int AIDS Soc. 2020 Jan;23(1):e25439. doi: 10.1002/jia2.25439.
    1. Biello KB, Marrow E, Mimiaga MJ, Sullivan P, Hightow-Weidman L, Mayer KH. A Mobile-Based App (MyChoices) to Increase Uptake of HIV Testing and Pre-Exposure Prophylaxis by Young Men Who Have Sex With Men: Protocol for a Pilot Randomized Controlled Trial. JMIR Res Protoc. 2019 Jan 07;8(1):e10694. doi: 10.2196/10694.
    1. Sullivan PS, Driggers R, Stekler JD, Siegler A, Goldenberg T, McDougal SJ, Caucutt J, Jones J, Stephenson R. Usability and Acceptability of a Mobile Comprehensive HIV Prevention App for Men Who Have Sex With Men: A Pilot Study. JMIR Mhealth Uhealth. 2017 Mar 09;5(3):e26. doi: 10.2196/mhealth.7199.
    1. Goldenberg T, McDougal SJ, Sullivan PS, Stekler JD, Stephenson R. Preferences for a Mobile HIV Prevention App for Men Who Have Sex With Men. JMIR Mhealth Uhealth. 2014 Oct 29;2(4):e47. doi: 10.2196/mhealth.3745.
    1. Goldenberg T, McDougal SJ, Sullivan PS, Stekler JD, Stephenson R. Building a Mobile HIV Prevention App for Men Who Have Sex With Men: An Iterative and Community-Driven Process. JMIR Public Health Surveill. 2015 Nov 16;1(2):e18. doi: 10.2196/publichealth.4449.
    1. Bandura A. The anatomy of stages of change. Am J Health Promot. 1997;12(1):8–10. doi: 10.4278/0890-1171-12.1.8.
    1. Bandura A. Health promotion by social cognitive means. Health Educ Behav. 2004 Apr 30;31(2):143–64. doi: 10.1177/1090198104263660.
    1. Hightow-Weidman LB, Muessig K, Rosenberg E, Sanchez T, LeGrand S, Gravens L, Sullivan PS. University of North Carolina/Emory Center for Innovative Technology (iTech) for Addressing the HIV Epidemic Among Adolescents and Young Adults in the United States: Protocol and Rationale for Center Development. JMIR Res Protoc. 2018 Aug 03;7(8):e10365. doi: 10.2196/10365.
    1. Lee S, Kapogiannis BG, Allison S. Improving the Youth HIV Prevention and Care Continuums: The Adolescent Medicine Trials Network for HIV/AIDS Interventions. JMIR Res Protoc. 2019 Mar 26;8(3):e12050. doi: 10.2196/12050.
    1. Biello KB, Hosek S, Drucker MT, Belzer M, Mimiaga MJ, Marrow E, Coffey-Esquivel J, Brothers J, Mayer KH. Preferences for Injectable PrEP Among Young U.S. Cisgender Men and Transgender Women and Men Who Have Sex with Men. Arch Sex Behav. 2018 Oct 19;47(7):2101–2107. doi: 10.1007/s10508-017-1049-7.
    1. Kazemi DM, Borsari B, Levine MJ, Shehab M, Nelson M, Dooley B, Stinson B, Fang F, Li S. Real-time demonstration of a mHealth app designed to reduce college students hazardous drinking. Psychol Serv. 2019 May;16(2):255–259. doi: 10.1037/ser0000310.
    1. Making Health Communication Programs Work : a Planner's Guide. U.S. Dept. of Health and Human Services, Public Health Service, National Institutes of Health, [Office of Cancer Communications, National Cancer Institute]; 2019. [2020-06-05]. .
    1. Hightow-Weidman LB, Muessig K, Rosenberg E, Sanchez T, LeGrand S, Gravens L, Sullivan PS. University of North Carolina/Emory Center for Innovative Technology (iTech) for Addressing the HIV Epidemic Among Adolescents and Young Adults in the United States: Protocol and Rationale for Center Development. JMIR Res Protoc. 2018 Aug 03;7(8):e10365. doi: 10.2196/10365.
    1. Ryan GW, Bernard HR. Techniques to Identify Themes. Field Methods. 2016 Jul 24;15(1):85–109. doi: 10.1177/1525822X02239569.
    1. Johnson BD, Dunlap E, Benoit E. Organizing "mountains of words" for data analysis, both qualitative and quantitative. Subst Use Misuse. 2010 Apr 11;45(5):648–70. doi: 10.3109/10826081003594757.
    1. Brooke J. SUS-A quick and dirty usability scale. In: Jordan P, Thomas B, McClelland I, Weerdmeester B, editors. Usability evaluation in industry. London: CRC Press; 1996. pp. 4–7.
    1. System Usability Scale (SUS) U.S. General Services Administration. 2020. [2020-06-05]. .
    1. Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies. 2009;4(3):114–123. doi: 10.5555/2835587.2835589.
    1. Bailey J, Mann S, Wayal S, Abraham C, Murray E. Digital media interventions for sexual health promotion-opportunities and challenges: a great way to reach people, particularly those at increased risk of sexual ill health. BMJ. 2015 Mar 03;350(mar03 8):h1099–h1099. doi: 10.1136/bmj.h1099.
    1. Fontenot HB, Rosenberger JG, McNair KT, Mayer KH, Zimet G. Perspectives and preferences for a mobile health tool designed to facilitate HPV vaccination among young men who have sex with men. Hum Vaccin Immunother. 2019 Jan 30;15(7-8):1815–1823. doi: 10.1080/21645515.2019.1568156.
    1. Saberi P, Siedle-Khan R, Sheon N, Lightfoot M. The Use of Mobile Health Applications Among Youth and Young Adults Living with HIV: Focus Group Findings. AIDS Patient Care STDS. 2016 Jun;30(6):254–60. doi: 10.1089/apc.2016.0044.
    1. Muessig KE, Pike EC, Fowler B, LeGrand S, Parsons JT, Bull SS, Wilson PA, Wohl DA, Hightow-Weidman LB. Putting prevention in their pockets: developing mobile phone-based HIV interventions for black men who have sex with men. AIDS Patient Care STDS. 2013 Apr;27(4):211–22. doi: 10.1089/apc.2012.0404.
    1. Mulawa MI, LeGrand S, Hightow-Weidman LB. eHealth to Enhance Treatment Adherence Among Youth Living with HIV. Curr HIV/AIDS Rep. 2018 Aug 29;15(4):336–349. doi: 10.1007/s11904-018-0407-y.
    1. Holloway IW, Winder TJ, Lea Charles Herbert, Tan D, Boyd D, Novak D. Technology Use and Preferences for Mobile Phone-Based HIV Prevention and Treatment Among Black Young Men Who Have Sex With Men: Exploratory Research. JMIR Mhealth Uhealth. 2017 Apr 13;5(4):e46. doi: 10.2196/mhealth.6436.
    1. Mitchell KJ, Ybarra ML, Korchmaros JD, Kosciw JG. Accessing sexual health information online: use, motivations and consequences for youth with different sexual orientations. Health Educ Res. 2014 Feb 16;29(1):147–57. doi: 10.1093/her/cyt071.
    1. Wartella E, Rideout V, Montague H, Beaudoin-Ryan L, Lauricella A. Teens, Health and Technology: A National Survey. MaC. 2016 Jun 16;4(3):13–23. doi: 10.17645/mac.v4i3.515.
    1. Gray NJ, Klein JD, Noyce PR, Sesselberg TS, Cantrill JA. Health information-seeking behaviour in adolescence: the place of the internet. Soc Sci Med. 2005 Apr;60(7):1467–78. doi: 10.1016/j.socscimed.2004.08.010.
    1. Sbaffi L, Rowley J. Trust and Credibility in Web-Based Health Information: A Review and Agenda for Future Research. J Med Internet Res. 2017 Jun 19;19(6):e218. doi: 10.2196/jmir.7579.
    1. Hosek S, Pettifor A. HIV Prevention Interventions for Adolescents. Curr HIV/AIDS Rep. 2019 Feb 1;16(1):120–128. doi: 10.1007/s11904-019-00431-y.
    1. Wilson PA, Meyer IH, Antebi-Gruszka N, Boone MR, Cook SH, Cherenack EM. Profiles of Resilience and Psychosocial Outcomes among Young Black Gay and Bisexual Men. Am J Community Psychol. 2016 Mar 28;57(1-2):144–57. doi: 10.1002/ajcp.12018.
    1. Vincent W, Peterson JL, Storholm ED, Huebner DM, Neilands TB, Calabrese SK, Rebchook GM, Tan JY, Pollack L, Kegeles SM. A Person-Centered Approach to HIV-Related Protective and Risk Factors for Young Black Men Who Have Sex with Men: Implications for Pre-exposure Prophylaxis and HIV Treatment as Prevention. AIDS Behav. 2019 Oct 12;23(10):2803–2815. doi: 10.1007/s10461-019-02630-5.
    1. Halkitis PN, Figueroa RP. Sociodemographic characteristics explain differences in unprotected sexual behavior among young HIV-negative gay, bisexual, and other YMSM in New York City. AIDS Patient Care STDS. 2013 Mar;27(3):181–90. doi: 10.1089/apc.2012.0415.
    1. Allison S, Bauermeister JA, Bull S, Lightfoot M, Mustanski B, Shegog R, Levine D. The intersection of youth, technology, and new media with sexual health: moving the research agenda forward. J Adolesc Health. 2012 Sep;51(3):207–12. doi: 10.1016/j.jadohealth.2012.06.012.
    1. Grov C, Breslow AS, Newcomb ME, Rosenberger JG, Bauermeister JA. Gay and bisexual men's use of the Internet: research from the 1990s through 2013. J Sex Res. 2014 Apr 22;51(4):390–409. doi: 10.1080/00224499.2013.871626.
    1. Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research. J Psychiatr Res. 2011 May;45(5):626–9. doi: 10.1016/j.jpsychires.2010.10.008.
    1. Onken LS, Carroll KM, Shoham V, Cuthbert BN, Riddle M. Reenvisioning Clinical Science: Unifying the Discipline to Improve the Public Health. Clin Psychol Sci. 2014 Jan 01;2(1):22–34. doi: 10.1177/2167702613497932.

Source: PubMed

Спонсоры и соавторы

Заболевания

Медикаментозные вмешательства

Подписаться