Feasibility of an Autism-Focused Augmented Reality Smartglasses System for Social Communication and Behavioral Coaching

Runpeng Liu, Joseph P Salisbury, Arshya Vahabzadeh, Ned T Sahin, Runpeng Liu, Joseph P Salisbury, Arshya Vahabzadeh, Ned T Sahin

Abstract

Background: Autism spectrum disorder (ASD) is a childhood-onset neurodevelopmental disorder with a rapidly rising prevalence, currently affecting 1 in 68 children, and over 3.5 million people in the United States. Current ASD interventions are primarily based on in-person behavioral therapies that are both costly and difficult to access. These interventions aim to address some of the fundamental deficits that clinically characterize ASD, including deficits in social communication, and the presence of stereotypies, and other autism-related behaviors. Current diagnostic and therapeutic approaches seldom rely on quantitative data measures of symptomatology, severity, or condition trajectory.

Methods: Given the current situation, we report on the Brain Power System (BPS), a digital behavioral aid with quantitative data gathering and reporting features. The BPS includes customized smartglasses, providing targeted personalized coaching experiences through a family of gamified augmented-reality applications utilizing artificial intelligence. These applications provide children and adults with coaching for emotion recognition, face directed gaze, eye contact, and behavioral self-regulation. This preliminary case report, part of a larger set of upcoming research reports, explores the feasibility of the BPS to provide coaching in two boys with clinically diagnosed ASD, aged 8 and 9 years.

Results: The coaching intervention was found to be well tolerated and rated as being both engaging and fun. Both males could easily use the system, and no technical problems were noted. During the intervention, caregivers reported improved non-verbal communication, eye contact, and social engagement during the intervention. Both boys demonstrated decreased symptoms of ASD, as measured by the aberrant behavior checklist at 24-h post-intervention. Specifically, both cases demonstrated improvements in irritability, lethargy, stereotypy, hyperactivity/non-compliance, and inappropriate speech.

Conclusion: Smartglasses using augmented reality may have an important future role in helping address the therapeutic needs of children with ASD. Quantitative data gathering from such sensor-rich systems may allow for digital phenotyping and the refinement of social communication constructs of the research domain criteria. This report provides evidence for the feasibility, usability, and tolerability of one such specialized smartglasses system.

Keywords: augmented reality; autism spectrum disorder; education; feasibility; smartglasses; stimulant; tolerability; virtual reality.

Figures

Figure 1
Figure 1
Brain Power System (partial side view).
Figure 2
Figure 2
User A: pre- and post-intervention aberrant behavior checklist (ABC) subscale scores.
Figure 3
Figure 3
User B: pre- and post-intervention aberrant behavior checklist (ABC) subscale scores.

References

    1. Baio J, Developmental Disabilities Monitoring Network Surveillance Year 2010 Principal Investigators; Centers for Disease Control and Prevention (CDC) . Prevalence of autism spectrum disorder among children aged 8 years: Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2010. MMWR Surveill Summ (2014) 63(2):1–21.
    1. Buescher AV, Cidav Z, Knapp M, Mandell DS. Costs of autism spectrum disorders in the United Kingdom and the United States. JAMA Pediatr (2014) 168(8):721–8.10.1001/jamapediatrics.2014.210
    1. Lavelle TA, Weinstein MC, Newhouse JP, Munir K, Kuhlthau KA, Prosser LA. Economic burden of childhood autism spectrum disorders. Pediatrics (2014) 133(3):e520–9.10.1542/peds.2013-0763
    1. Amaral D, Geschwind D, Dawson G, editors. Autism Spectrum Disorders. Oxford: University Press; (2011).
    1. Zablotsky B, Pringle BA, Colpe LJ, Kogan MD, Rice C, Blumberg SJ. Service and treatment use among children diagnosed with autism spectrum disorders. J Dev Behav Pediatr (2015) 36(2):98–105.10.1097/DBP.0000000000000127
    1. Association AP. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®). Arlington: American Psychiatric Association Publishing; (2013).
    1. Wong C, Odom SL, Hume KA, Cox AW, Fettig A, Kucharczyk S, et al. Evidence-based practices for children, youth, and young adults with autism spectrum disorder: a comprehensive review. J Autism Dev Disord (2015) 45(7):1951–66.10.1007/s10803-014-2351-z
    1. Tanaka JW, Sung A. The “eye avoidance” hypothesis of autism face processing. J Autism Dev Disord (2016) 46(5):1538–52.10.1007/s10803-013-1976-7
    1. Odom SL, Thompson JL, Hedges S, Boyd BA, Dykstra JR, Duda MA, et al. Technology-aided interventions and instruction for adolescents with autism spectrum disorder. J Autism Dev Disord (2015) 45(12):3805–19.10.1007/s10803-014-2320-6
    1. Ramdoss S, Lang R, Fragale C, Britt C, O’Reilly M, Sigafoos J, et al. Use of computer-based interventions to promote daily living skills in individuals with intellectual disabilities: a systematic review. J Dev Phys Disabil (2012) 24(2):197–215.10.1007/s10882-011-9259-8
    1. Ramdoss S, Lang R, Mulloy A, Franco J, O’Reilly M, Didden R, et al. Use of computer-based interventions to teach communication skills to children with autism spectrum disorders: a systematic review. J Behav Educ (2011) 20(1):55–76.10.1007/s10864-010-9112-7
    1. Ramdoss S, Machalicek W, Rispoli M, Mulloy A, Lang R, O’Reilly M. Computer-based interventions to improve social and emotional skills in individuals with autism spectrum disorders: a systematic review. Dev Neurorehabil (2012) 15(2):119–35.10.3109/17518423.2011.651655
    1. Wainer AL, Ingersoll BR. The use of innovative computer technology for teaching social communication to individuals with autism spectrum disorders. Res Autism Spectr Disord (2011) 5(1):96–107.10.1016/j.rasd.2010.08.002
    1. Tanaka JW, Wolf JM, Klaiman C, Koenig K, Cockburn J, Herlihy L, et al. Using computerized games to teach face recognition skills to children with autism spectrum disorder: the let’s face it! program. J Child Psychol Psychiatry (2010) 51(8):944–52.10.1111/j.1469-7610.2010.02258.x
    1. Ayres KM, Maguire A, McClimon D. Acquisition and generalization of chained tasks taught with computer based video instruction to children with autism. Educ Train Dev Disabil (2009) 44(4):493–508.
    1. Burke RV, Andersen MN, Bowen SL, Howard MR, Allen KD. Evaluation of two instruction methods to increase employment options for young adults with autism spectrum disorders. Res Dev Disabil (2010) 31(6):1223–33.10.1016/j.ridd.2010.07.023
    1. Pennington RC. Computer-assisted instruction for teaching academic skills to students with autism spectrum disorders: a review of literature. Focus Autism Other Dev Disabl (2010) 25(4):239–48.10.1177/1088357610378291
    1. King AM, Thomeczek M, Voreis G, Scott V. iPad® use in children and young adults with autism spectrum disorder: an observational study. Child Lang Teach Ther (2014) 30(2):159–73.10.1177/0265659013510922
    1. Shane HC, Albert PD. Electronic screen media for persons with autism spectrum disorders: results of a survey. J Autism Dev Disord (2008) 38(8):1499–508.10.1007/s10803-007-0527-5
    1. Mayes SD, Calhoun SL. Analysis of WISC-III, Stanford-Binet: IV, and academic achievement test scores in children with autism. J Autism Dev Disord (2003) 33(3):329–41.10.1023/A:1024462719081
    1. Schlosser RW, Blischak DM. Is there a role for speech output in interventions for persons with autism? A review. Focus Autism Other Dev Disabl (2001) 16(3):170–8.10.1177/108835760101600305
    1. Tincani M, Boutot E. Technology and autism: current practices and future directions. In: Edyburn D, Higgins K, Boon R, editors. Handbook of Special Education Technology Research and Practice. Whitefish Bay, WI: Knowledge by Design; (2005). p. 413–21.
    1. Health NIoM. Behavioral Assessment Methods for RDoC Constructs. Bethesda, MD: National Institute of Mental Health Services DoHaH; (2016).
    1. Insel T, Cuthbert B, Garvey M, Heinssen R, Pine DS, Quinn K, et al. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatry (2010) 167:748–51.10.1176/appi.ajp.2010.09091379
    1. Escobedo L, Tentori M, Quintana E, Favela J, Garcia-Rosas D. Using augmented reality to help children with autism stay focused. IEEE Pervasive Comput (2014) 13(1):38–46.10.1109/MPRV.2014.19
    1. da Silva CA, Fernandes AR, Grohmann AP. STAR: speech therapy with augmented reality for children with autism spectrum disorders. In: Cordeiro J, Hammoudi S, Maciaszek L, Camp O, Filipe J, editors. Enterprise Information Systems. ICEIS 2014. Lecture Notes in Business Information Processing, Vol. 227 Cham: Springer; (2015).
    1. Escobedo L, Nguyen DH, Boyd L, Hirano S, Rangel A, Garcia-Rosas D, et al., editors. MOSOCO: a mobile assistive tool to support children with autism practicing social skills in real-life situations. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems ACM (2012).
    1. McDuff D, Mahmoud A, Mavadati M, Amr M, Turcot J, Kaliouby Re, editors. AFFDEX SDK: a cross-platform real-time multi-face expression recognition toolkit. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems ACM (2016).
    1. Aman MG, Singh NN, Stewart AW, Field CJ. The aberrant behavior checklist: a behavior rating scale for the assessment of treatment effects. Am J Ment Defic (1985) 89(5):485–91.
    1. Rutter M, Bailey A, Lord C. The Social Communication Questionnaire: Manual. Torrance: Western Psychological Services; (2003).
    1. Chandler S, Charman T, Baird G, Simonoff E, Loucas T, Meldrum D, et al. Validation of the social communication questionnaire in a population cohort of children with autism spectrum disorders. J Am Acad Child Adolesc Psychiatry (2007) 46(10):1324–32.10.1097/chi.0b013e31812f7d8d
    1. de Haas B, Schwarzkopf DS, Alvarez I, Lawson RP, Henriksson L, Kriegeskorte N, et al. Perception and processing of faces in the human brain is tuned to typical feature locations. J Neurosci (2016) 36(36):9289–302.10.1523/JNEUROSCI.4131-14.2016
    1. Riby DM, Hancock PJ. Viewing it differently: social scene perception in Williams syndrome and autism. Neuropsychologia (2008) 46(11):2855–60.10.1016/j.neuropsychologia.2008.05.003
    1. Klin A, Jones W, Schultz R, Volkmar F, Cohen D. Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Arch Gen Psychiatry (2002) 59(9):809–16.10.1001/archpsyc.59.9.809
    1. Corden B, Chilvers R, Skuse D. Avoidance of emotionally arousing stimuli predicts social-perceptual impairment in Asperger’s syndrome. Neuropsychologia (2008) 46(1):137–47.10.1016/j.neuropsychologia.2007.08.005
    1. Tottenham N, Hertzig ME, Gillespie-Lynch K, Gilhooly T, Millner AJ, Casey BJ. Elevated amygdala response to faces and gaze aversion in autism spectrum disorder. Soc Cogn Affect Neurosci (2014) 9(1):106–17.10.1093/scan/nst050
    1. Kaartinen M, Puura K, Makela T, Rannisto M, Lemponen R, Helminen M, et al. Autonomic arousal to direct gaze correlates with social impairments among children with ASD. J Autism Dev Disord (2012) 42(9):1917–27.10.1007/s10803-011-1435-2
    1. Moriuchi JM, Klin A, Jones W. Mechanisms of diminished attention to eyes in autism. Am J Psychiatry (2017) 174(1):26–35.10.1176/appi.ajp.2016.15091222
    1. Ekman P, Rolls E, Perrett D, Ellis H. Facial expressions of emotion: an old controversy and new findings [and discussion]. Philos Trans R Soc Lond B Biol Sci (1992) 335(1273):63–9.10.1098/rstb.1992.0008
    1. Berggren S, Fletcher-Watson S, Milenkovic N, Marschik PB, Bolte S, Jonsson U. Emotion recognition training in autism spectrum disorder: a systematic review of challenges related to generalizability. Dev Neurorehabil (2017) 10:1–14.10.1080/17518423.2017.1305004
    1. Trevisan DA, Birmingham E. Are emotion recognition abilities related to everyday social functioning in ASD? A meta-analysis. Res Autism Spectr Disord (2016) 32:24–42.10.1016/j.rasd.2016.08.004
    1. Halladay AK, Bishop S, Constantino JN, Daniels AM, Koenig K, Palmer K, et al. Sex and gender differences in autism spectrum disorder: summarizing evidence gaps and identifying emerging areas of priority. Mol Autism (2015) 6(1):36.10.1186/s13229-015-0019-y

Source: PubMed

Sponsoren und Mitarbeiter

Krankheiten

Drogeninterventionen

3
Abonnieren