Usability of a Technology-Based Bystander Bullying Intervention for Middle School Students in Rural, Low-Income Communities: Mixed Methods Study

Diana M Doumas, Aida Midgett, Valerie Myers, Mary Klein Buller, Diana M Doumas, Aida Midgett, Valerie Myers, Mary Klein Buller

Abstract

Background: Students who are targets of bullying and who witness bullying are at high risk for negative mental health outcomes. Bystander training is essential to reduce bullying and the negative associated consequences for targets and bystanders. Resources necessary for program delivery, however, pose significant barriers for schools, particularly those in rural, low-income communities. Technology-based programs can reduce health disparities for students in these communities through cost-effective, easy-to-disseminate programming.

Objective: The aim of this study is to conduct usability testing of a bystander bullying web app prototype, STAC-T (technology-based STAC, which stands for the 4 bystander strategies Stealing the Show, Turning it Over, Accompanying Others, and Coaching Compassion) as an initial step in the development of a full-scale STAC-T intervention. Objectives include assessing usability and acceptability of the STAC-T prototype, understanding school needs and barriers to program implementation, and assessing differences in usability between school personnel and students.

Methods: A sample of 16 participants, including school personnel and students recruited from 3 middle schools in rural, low-income communities, completed usability testing followed by a qualitative interview. Descriptive statistics, 2-tailed independent sample t tests, and consensual qualitative research were used to assess usability and program satisfaction and to extract themes related to acceptability, feasibility, needs, barriers, and feedback for intervention development.

Results: Usability testing indicated that the app was easy to use, acceptable, and feasible. Both school personnel (mean rating 89.6, SD 5.1) and students (mean rating 91.8, SD 7.0) rated the app well above the standard cutoff score for above-average usability (ie, 68), and both school personnel (mean rating 5.83, SD 0.41) and students (mean rating 6.10, SD 0.57) gave the app high user-friendliness ratings (0-7 scale, with 7 as high user-friendliness). The overall ratings also suggested that school personnel and students were satisfied with the program. Of the 6 school personnel who said they would recommend the program, 1 (17%), 4 (66%), and 1 (17%) rated the program as 3, 4, and 5 stars, respectively; 80% (8/10) of students said they would recommend the program; and 60% (6/10) and 40% (4/10) rated the program as 4 stars and 5 stars, respectively. Qualitative data revealed that school personnel and students found the STAC-T app to be useful, user-friendly, and relevant, while providing feedback related to the importance of digital learning activities that engage the user. Data from school personnel also indicated positive perceptions regarding program feasibility and probability of program adoption, with the most significant barrier being cost, suggesting the importance of considering the financial resources available to schools in rural, low-income communities when setting the price point for the full-scale STAC-T intervention.

Conclusions: This study provides support for the full-scale development of the STAC-T app and provides key information for revision to enhance used engagement.

Trial registration: ClinicalTrials.gov NCT04681495; https://ichgcp.net/clinical-trials-registry/NCT04681495.

Keywords: STAC-T; low-income; middle school; mobile phone; rural; technology-based bullying intervention; usability testing.

Conflict of interest statement

Conflicts of Interest: MKB is the owner of Klein Buendel, Inc, the organization that received the National Institutes of Health grant award for this project. As it is a small business technology transfer, a concerted effort to commercialize a final product that may result from this and subsequent National Institutes of Health–funded research will be made, and the organization may receive royalties or revenue from the commercialization. DMD and AM may also receive royalties or revenue from the commercialization.

©Diana M Doumas, Aida Midgett, Valerie Myers, Mary Klein Buller. Originally published in JMIR Formative Research (https://formative.jmir.org), 26.10.2021.

Figures

Figure 1
Figure 1
Samples from the STAC-T (technology-based Stealing the Show, Turning it Over, Accompanying Others, Coaching Compassion) prototype.

References

    1. Student reports of bullying: results from the 2017 school crime supplement to the national crime victimization survey. U.S. Department of Education - NCES 2019-054. 2019. [2021-09-20]. .
    1. Injury prevention and control: #StopBullying. Centers for Disease Control and Prevention, National Center for Injury Prevention and Control. [2021-09-21]. .
    1. Ching CB, Lee H, Mason MD, Clayton DB, Thomas JC, Pope JC, Adams MC, Brock JW, Tanaka ST. Bullying and lower urinary tract symptoms: why the pediatric urologist should care about school bullying. J Urol. 2015 Feb;193(2):650–4. doi: 10.1016/j.juro.2014.08.103.S0022-5347(14)04358-4
    1. van Geel M, Goemans A, Vedder PH. The relation between peer victimization and sleeping problems: a meta-analysis. Sleep Med Rev. 2016 Jun;27:89–95. doi: 10.1016/j.smrv.2015.05.004.S1087-0792(15)00074-X
    1. Fekkes M, Pijpers FI, Verloove-Vanhorick S. Bullying behavior and associations with psychosomatic complaints and depression in victims. J Pediatr. 2004 Jan;144(1):17–22. doi: 10.1016/j.jpeds.2003.09.025.S0022-3476(03)00610-3
    1. Hase C, Goldberg S, Smith D, Stuck A, Campain J. Impacts of traditional bullying and cyberbullying on the mental health of middle school and high school students. Psychol Schs. 2015 May 22;52(6):607–17. doi: 10.1002/pits.21841. doi: 10.1002/pits.21841.
    1. Rivers I, Poteat V, Noret N, Ashurst N. Observing bullying at school: the mental health implications of witness status. Sch Psychol Q. 2009 Dec;24(4):211–23. doi: 10.1037/a0018164. doi: 10.1037/a0018164.
    1. Jack AH, Egan V. Childhood bullying, paranoid thinking and the misappraisal of social threat: trouble at school. School Ment Health. 2018 Nov 22;10(1):26–34. doi: 10.1007/s12310-017-9238-z. 9238
    1. Juvonen J, Graham S, Schuster MA. Bullying among young adolescents: the strong, the weak, and the troubled. Pediatrics. 2003 Dec 03;112(6 Pt 1):1231–7. doi: 10.1542/peds.112.6.1231.
    1. Siegel RS, La Greca AM, Harrison HM. Peer victimization and social anxiety in adolescents: prospective and reciprocal relationships. J Youth Adolesc. 2009 Sep 22;38(8):1096–109. doi: 10.1007/s10964-009-9392-1.
    1. Storch EA, Masia-Warner C. The relationship of peer victimization to social anxiety and loneliness in adolescent females. J Adolesc. 2004 Jun;27(3):351–62. doi: 10.1016/j.adolescence.2004.03.003.S0140197104000351
    1. Fisher BW, Gardella JH, Teurbe-Tolon AR. Peer cybervictimization among adolescents and the associated internalizing and externalizing problems: a meta-analysis. J Youth Adolesc. 2016 Sep 22;45(9):1727–43. doi: 10.1007/s10964-016-0541-z.10.1007/s10964-016-0541-z
    1. Kowalski RM, Giumetti GW, Schroeder AN, Lattanner MR. Bullying in the digital age: a critical review and meta-analysis of cyberbullying research among youth. Psychol Bull. 2014 Jul;140(4):1073–137. doi: 10.1037/a0035618.2014-04307-001
    1. Sticca F, Perren S. Is cyberbullying worse than traditional bullying? Examining the differential roles of medium, publicity, and anonymity for the perceived severity of bullying. J Youth Adolesc. 2013 May 27;42(5):739–50. doi: 10.1007/s10964-012-9867-3.
    1. Dulmus CN, Theriot MT, Sowers KM, Blackburn JA. Student reports of peer bullying victimization in a rural school. Stress Trauma Crisis. 2004 Jan;7(1):1–16. doi: 10.1080/15434610490281093.
    1. Leadbeater B, Sukhawathanakul P, Smith A, Thompson R, Gladstone E, Sklar N. Bullying and victimization in rural schools: risks, reasons, and responses. J Rural Community Dev. 2013;8(1):31–47.
    1. Nansel TR, Overpeck M, Pilla RS, Ruan WJ, Simons-Morton B, Scheidt P. Bullying behaviors among US youth: prevalence and association with psychosocial adjustment. J Am Med Assoc. 2001 Apr 25;285(16):2094–100. doi: 10.1001/jama.285.16.2094. joc01746
    1. Azeredo C, Rinaldi A, de Moraes Cl, Levy R, Menezes P. School bullying: a systematic review of contextual-level risk factors in observational studies. Aggr Violent Behav. 2015 May;22:65–76. doi: 10.1016/j.avb.2015.04.006. doi: 10.1016/j.avb.2015.04.006.
    1. Evans C, Smokowski P, Cotter K. Cumulative bullying victimization: an investigation of the dose–response relationship between victimization and the associated mental health outcomes, social supports, and school experiences of rural adolescents. Child Youth Serv Rev. 2014 Sep;44:256–64. doi: 10.1016/j.childyouth.2014.06.021. doi: 10.1016/j.childyouth.2014.06.021.
    1. Tippett N, Wolke D. Socioeconomic status and bullying: a meta-analysis. Am J Public Health. 2014 Jun;104(6):e48–59. doi: 10.2105/AJPH.2014.301960.
    1. Smokowski PR, Cotter KL, Robertson C, Guo S. Demographic, psychological, and school environment correlates of bullying victimization and school hassles in rural youth. J Criminol. 2013 Jun 11;2013:1–13. doi: 10.1155/2013/137583.
    1. Midgett A, Doumas DM. Witnessing bullying at school: the association between being a bystander and anxiety and depressive symptoms. School Ment Health. 2019 Jan 25;11(3):454–63. doi: 10.1007/s12310-019-09312-6.
    1. Doumas DM, Midgett A. Witnessing cyberbullying and internalizing symptoms among middle school students. Eur J Investig Health Psychol Educ. 2020 Oct 04;10(4):957–66. doi: 10.3390/ejihpe10040068. ejihpe10040068
    1. Doumas D, Midgett A. The association between witnessing cyberbullying and depressive symptoms and social anxiety among elementary school students. Psychol Schs. 2020 Dec 15;58(3):622–37. doi: 10.1002/pits.22467. doi: 10.1002/pits.22467.
    1. Doumas DM, Midgett A. The relationship between witnessing cyberbullying and depressive symptoms and social anxiety among middle school students: is witnessing school bullying a moderator? J Child Adolesc Couns. 2021 Aug 18;:1–12. doi: 10.1080/23727810.2021.1934369.
    1. Wright MF, Wachs S, Harper BD. The moderation of empathy in the longitudinal association between witnessing cyberbullying, depression, and anxiety. Cyber Psychol. 2018 Dec 29;12(4):11419. doi: 10.5817/cp2018-4-6.
    1. Janson G, Carney J, Hazler R, Oh I. Bystanders' reactions to witnessing repetitive abuse experiences. J Couns Dev. 2009;87(3):319–26. doi: 10.1002/j.1556-6678.2009.tb00113.x.
    1. Hutchinson M. Exploring the impact of bullying on young bystanders. Educ Psychol Pract. 2012 Nov 08;28(4):425–42. doi: 10.1080/02667363.2012.727785. doi: 10.1080/02667363.2012.727785.
    1. Lambe LJ, Hudson CC, Craig WM, Pepler DJ. Does defending come with a cost? Examining the psychosocial correlates of defending behaviour among bystanders of bullying in a Canadian sample. Child Abuse Negl. 2017 Mar;65:112–23. doi: 10.1016/j.chiabu.2017.01.012.S0145-2134(17)30012-1
    1. Wu W, Luu S, Luh D. Defending behaviors, bullying roles, and their associations with mental health in junior high school students: a population-based study. BMC Public Health. 2016 Oct 10;16(1):1066. doi: 10.1186/s12889-016-3721-6. 10.1186/s12889-016-3721-6
    1. DeSmet A, Bastiaensens S, Van Cleemput K, Poels K, Vandebosch H, Cardon G, De Bourdeaudhuij I. Deciding whether to look after them, to like it, or leave it: a multidimensional analysis of predictors of positive and negative bystander behavior in cyberbullying among adolescents. Comput Hum Behav. 2016 Apr;57:398–415. doi: 10.1016/j.chb.2015.12.051. doi: 10.1016/j.chb.2015.12.051.
    1. Gaffney H, Ttofi M, Farrington D. Evaluating the effectiveness of school-bullying prevention programs: an updated meta-analytical review. Aggr Violent Behav. 2019 Mar;45:111–33. doi: 10.1016/j.avb.2018.07.001. doi: 10.1016/j.avb.2018.07.001.
    1. Salmivalli C, Voeten M. Connections between attitudes, group norms, and behaviour in bullying situations. Int J Behav Dev. 2016 Jun 30;28(3):246–58. doi: 10.1080/01650250344000488.
    1. Lenhart A, Madden M, Smit A, Purcell K, Zickuhr K. Teens, kindness and cruelty on social network sites. Pew Research Center. 2011. [2021-09-21].
    1. Doumas DM, Midgett A, Watts AD. The impact of a brief, bullying bystander intervention on internalizing symptoms: is gender a moderator of intervention effects? Sch Psychol Int. 2019 Feb 14;40(3):275–93. doi: 10.1177/0143034319830149.
    1. Midgett A, Doumas DM. The impact of a brief bullying bystander intervention on depressive symptoms. J Couns Dev. 2019 Jun 13;97(3):270–80. doi: 10.1002/jcad.12267.
    1. Midgett A, Doumas D, Trull R. Evaluation of a brief, school-based bullying bystander intervention for elementary school students. Prof Sch Couns. 2018 Feb 15;20(1):172–83. doi: 10.5330/1096-2409-20.1.172.
    1. Midgett A, Doumas D, Trull R, Johnston A. A randomized controlled study evaluating a brief, bystander bullying intervention with junior high school students. J Sch Couns. 2017;15(9):1–34.
    1. Watts AD, Doumas DM, Midgett A. Efficacy of a brief, school‐based bystander bullying intervention on high school students' alcohol use. J Addict Offender Couns. 2019 Oct 07;40(2):66–83. doi: 10.1002/jaoc.12066.
    1. Macháčková H, Dedkova L, Sevcikova A, Cerna A. Bystanders' support of cyberbullied schoolmates. J Community Appl Soc Psychol. 2012 Dec 27;23(1):25–36. doi: 10.1002/casp.2135.
    1. Sjögren B, Thornberg R, Wänström L, Gini G. Bystander behaviour in peer victimisation: moral disengagement, defender self-efficacy and student-teacher relationship quality. Res Pap Educ. 2020 Feb 12;36(5):588–610. doi: 10.1080/02671522.2020.1723679.
    1. Bussey K, Fitzpatrick S, Raman A. The role of moral disengagement and self-efficacy in cyberbullying. J Sch Violence. 2014 Sep 30;14(1):30–46. doi: 10.1080/15388220.2014.954045.
    1. Reinke WM, Stormont M, Herman KC, Puri R, Goel N. Supporting children's mental health in schools: teacher perceptions of needs, roles, and barriers. Sch Psychol Q. 2011 Mar;26(1):1–13. doi: 10.1037/a0022714.
    1. NetSupport DNA - IT asset management for schools. NetSupport DNA. [2021-09-21].
    1. Duffin E. E-learning and digital education - Statistics and facts. Statista. 2020. [2021-09-21].
    1. Survey questionnaires results: students' computer access and use. National Assessment of Educational Progress (NAEP) 2015. [2021-09-21].
    1. Riddell R. Is E-Rate doing enough to effectively expand broadband in rural schools? K-12 Dive. 2018. [2021-09-21].
    1. Rachfal C, Gilroy A. Broadband internet access and the digital divide: federal assistance programs. Congressional Research Service. 2019. [2021-09-21]. .
    1. USDA launches new program to create high-speed internet e-connectivity in rural America. U.S. Department of Agriculture. 2018. [2021-09-21]. .
    1. Department of education releases resource to help schools improve ventilation systems to prevent COVID-19. U.S. Department of Education. 2021. [2021-07-23]. .
    1. Midgett A, Doumas D, Sears D, Lundquist A, Hausheer R. A bystander bullying psychoeducation program with middle school students: a preliminary report. Prof Couns. 2015 Dec;5(4):486–500. doi: 10.15241/am.5.4.486.
    1. Midgett A, Doumas DM, Johnston AD. Establishing school counselors as leaders in bullying curriculum delivery. Prof Sch Couns. 2018 May 28;21(1):1–9. doi: 10.1177/2156759x18778781.
    1. Midgett A, Doumas DM, Trull R, Johnson J. Training students who occasionally bully to be peer advocates: is a bystander intervention effective in reducing bullying behavior? J Child Adolesc Couns. 2017 Jan 19;3(1):1–13. doi: 10.1080/23727810.2016.1277116.
    1. Midgett A, Doumas DM. Acceptability and short-term outcomes of a brief, bystander intervention program to decrease bullying in an ethnically blended school in low-income community. Contemp School Psychol. 2020 Sep 18;24(4):508–17. doi: 10.1007/s40688-020-00321-w.
    1. Midgett A, Doumas D, Moran M, Gallo L. Cultural adaptation of a school-based bullying intervention. J Educ Psycholo Res. 2020:2–60.
    1. Moran M, Midgett A, Doumas DM, Moody S, Porchia S. A mixed method evaluation of a culturally adapted, brief, bullying bystander intervention for middle school students. J Child Adolesc Couns. 2019;5(3):221–38. doi: 10.1080/23727810.2019.1669372.
    1. Moran M, Midgett A, Doumas DM. Evaluation of a brief, bystander bullying intervention (STAC) for ethnically blended middle schools in low-income communities. Prof Sch Couns. 2020 Jul 20;23(1):2156759X2094064. doi: 10.1177/2156759x20940641.
    1. Midgett A, Doumas DM, Peralta C, Bond L, Flay B. Impact of a brief, bystander bullying prevention program on depressive symptoms and passive suicidal ideation: a program evaluation model for school personnel. J Prev Health Promot. 2020 Jul 26;1(1):80–103. doi: 10.1177/2632077020942959.
    1. Midgett A, Doumas D, Myers V, Moody S, Doud A. Technology-based bullying intervention for rural schools: perspectives on needs, challenges, and design. Rural Ment Health. 2021 Jan;45(1):14–30. doi: 10.1037/rmh0000151. doi: 10.1037/rmh0000151.
    1. DiClemente CC, Marinilli AS, Singh M, Bellino LE. The role of feedback in the process of health behavior change. Am J Health Behav. 2001;25(3):217–27.
    1. Miller DT, Prentice DA. Changing norms to change behavior. Annu Rev Psychol. 2016;67:339–61. doi: 10.1146/annurev-psych-010814-015013.
    1. Wichansky AM. Usability testing in 2000 and beyond. Ergonomics. 2000 Jul 10;43(7):998–1006. doi: 10.1080/001401300409170.
    1. Lyon AR, Bruns EJ. User-centered redesign of evidence-based psychosocial interventions to enhance implementation-hospitable soil or better seeds? JAMA Psychiatry. 2019 Jan 01;76(1):3–4. doi: 10.1001/jamapsychiatry.2018.3060.2712978
    1. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011 Mar;38(2):65–76. doi: 10.1007/s10488-010-0319-7.
    1. Han SS, Weiss B. Sustainability of teacher implementation of school-based mental health programs. J Abnorm Child Psychol. 2005 Dec;33(6):665–79. doi: 10.1007/s10802-005-7646-2.
    1. Ritterband LM, Gonder-Frederick LA, Cox DJ, Clifton AD, West RW, Borowitz SM. Internet interventions: in review, in use, and into the future. Prof Psychol Res Pr. 2003;34(5):527–34. doi: 10.1037/0735-7028.34.5.527.
    1. Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JE. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res. 2012;14(6):e152. doi: 10.2196/jmir.2104. v14i6e152
    1. Oinas-Kukkonen H, Harjumaa M. Persuasive systems design: key issues, process model, and system features. Commun Assoc Inf Syst. 2009;24(1):28. doi: 10.17705/1cais.02428.
    1. Oinas-Kukkonen H, Harjumaa M. A systematic framework for designing and evaluating persuasive systems. In: Oinas-Kukkonen H, Hasle H, Harjumaa M, Segerståhl K, Øhrstrøm P, editors. Persuasive Technology. Berlin, Heidelberg: Springer; 2008. pp. 164–76.
    1. Abras C, Maloney-Krichmar D, Preece J. User-centered design. In: Bainbridge W, editor. Encyclopedia of Human-Computer Interaction. Vol 37. Thousand Oaks, CA: SAGE Publications; 2004. pp. 445–56.
    1. Norman D, Draper S. User Centered System Design: New Perspectives on Human-Computer Interaction. Hillsdale, NJ: Lawrence Erlbaum Associates Inc; 1986.
    1. Gustafson K, Branch R. What is instructional design? In: Reiser RA, Dempsey JV, editors. Trends and Issues in Instructional Design and Technology. Columbus, OH: Merril/Prentice Hall; 2002.
    1. Molenda M. In search of the elusive ADDIE Model. Perf Improv. 2015 Feb 19;54(2):40–2. doi: 10.1002/pfi.21461.
    1. Peterson C. Bringing ADDIE to life: instructional design at its best. J Educ Multimedia Hypermedia. 2003;12(3):227–41. doi: 10.1007/978-0-387-09506-6_3.
    1. Mazzara ML. Society for Technical Communication. Hanover, Germany: TIB University Library; 2001. Application of theory: minimalism and user centered design; pp. 327–31.
    1. Nicholson S. A user-centered theoretical framework for meaningful gamification. Proceedings of the Games+Learning+Society Conference 8.0; Games+Learning+Society Conference 8.0; June 13-15, 2012; Madison, WI. 2012.
    1. Pagulayan R, Keeker K, Fuller T, Wixon D, Romero R, Gunn DV. User-centered design in games. In: Sears A, Jacko JA, editors. Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications. Mahwah, NJ: Lawrence Erlbaum Associaties, Inc; 2002. pp. 883–906.
    1. Cafazzo JA, Casselman M, Hamming N, Katzman DK, Palmert MR. Design of an mHealth app for the self-management of adolescent type 1 diabetes: a pilot study. J Med Internet Res. 2012;14(3):e70. doi: 10.2196/jmir.2058. v14i3e70
    1. Graafland M, Dankbaar M, Mert A, Lagro J, De WL, Schuit S, Schaafstal A, Schijven M. How to systematically assess serious games applied to health care. JMIR Serious Games. 2014;2(2):e11. doi: 10.2196/games.3825. v2i2e11
    1. Hieftje K. A tablet game for risk reduction and HIV prevention in adolescents. J Mob Technol Med. 2012 Dec;1(4S):8. doi: 10.7309/jmtm.32.
    1. Brooke J. SUS: A 'Quick and Dirty' usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, McClelland AL, editors. Usability Evaluation in Industry. London: Taylor and Francis; 1996.
    1. Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Studies. 2009;4(3):114–23.
    1. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006 Jan;3(2):77–101. doi: 10.1191/1478088706qp063oa.
    1. Nowell LS, Norris JM, White DE, Moules NJ. Thematic analysis: striving to meet the trustworthiness criteria. Int J Qual Methods. 2017 Oct 02;16(1):1–13. doi: 10.1177/1609406917733847.
    1. Lehto T, Oinas-Kukkonen H. Persuasive features in web-based alcohol and smoking interventions: a systematic review of the literature. J Med Internet Res. 2011 Jul 22;13(3):e46. doi: 10.2196/jmir.1559. v13i3e46
    1. Rural America at a glance. U.S. Department of Agriculture. 2020. [2021-09-21]. .
    1. Casilli C, Knight E. 7 things you should know about badges. EDUCAUSE. 2012. [2021-09-21]. .
    1. Miller AS, Cafazzo JA, Seto E. A game plan: gamification design principles in mHealth applications for chronic disease management. Health Informatics J. 2014 Jul 1;:184–93. doi: 10.1177/1460458214537511.1460458214537511

Source: PubMed

Спонсоры и соавторы

Заболевания

Медикаментозные вмешательства

Подписаться