Feasibility and Impact of Integrating an Artificial Intelligence-Based Diagnosis Aid for Autism Into the Extension for Community Health Outcomes Autism Primary Care Model: Protocol for a Prospective Observational Study

Kristin Sohl, Rachel Kilian, Alicia Brewer Curran, Melissa Mahurin, Valeria Nanclares-Nogués, Stuart Liu-Mayo, Carmela Salomon, Jennifer Shannon, Sharief Taraman, Kristin Sohl, Rachel Kilian, Alicia Brewer Curran, Melissa Mahurin, Valeria Nanclares-Nogués, Stuart Liu-Mayo, Carmela Salomon, Jennifer Shannon, Sharief Taraman

Abstract

Background: The Extension for Community Health Outcomes (ECHO) Autism Program trains clinicians to screen, diagnose, and care for children with autism spectrum disorder (ASD) in primary care settings. This study will assess the feasibility and impact of integrating an artificial intelligence (AI)-based ASD diagnosis aid (the device) into the existing ECHO Autism Screening Tool for Autism in Toddlers and Young Children (STAT) diagnosis model. The prescription-only Software as a Medical Device, designed for use in children aged 18 to 72 months at risk for developmental delay, produces ASD diagnostic recommendations after analyzing behavioral features from 3 distinct inputs: a caregiver questionnaire, 2 short home videos analyzed by trained video analysts, and a health care provider questionnaire. The device is not a stand-alone diagnostic and should be used in conjunction with clinical judgment.

Objective: This study aims to assess the feasibility and impact of integrating an AI-based ASD diagnosis aid into the ECHO Autism STAT diagnosis model. The time from initial ECHO Autism clinician concern to ASD diagnosis is the primary end point. Secondary end points include the time from initial caregiver concern to ASD diagnosis, time from diagnosis to treatment initiation, and clinician and caregiver experience of device use as part of the ASD diagnostic journey.

Methods: Research participants for this prospective observational study will be patients suspected of having ASD (aged 18-72 months) and their caregivers and up to 15 trained ECHO Autism clinicians recruited by the ECHO Autism Communities research team from across rural and suburban areas of the United States. Clinicians will provide routine clinical care and conduct best practice ECHO Autism diagnostic evaluations in addition to prescribing the device. Outcome data will be collected via a combination of electronic questionnaires, reviews of standard clinical care records, and analysis of device outputs. The expected study duration is no more than 12 months. The study was approved by the institutional review board of the University of Missouri-Columbia (institutional review board-assigned project number 2075722).

Results: Participant recruitment began in April 2022. As of June 2022, a total of 41 participants have been enrolled.

Conclusions: This prospective observational study will be the first to evaluate the use of a novel AI-based ASD diagnosis aid as part of a real-world primary care diagnostic pathway. If device integration into primary care proves feasible and efficacious, prolonged delays between the first ASD concern and eventual diagnosis may be reduced. Streamlining primary care ASD diagnosis could potentially reduce the strain on specialty services and allow a greater proportion of children to commence early intervention during a critical neurodevelopmental window.

Trial registration: ClinicalTrials.gov NCT05223374; https://ichgcp.net/clinical-trials-registry/NCT05223374.

International registered report identifier (irrid): PRR1-10.2196/37576.

Keywords: Software as a Medical Device; artificial intelligence; autism spectrum disorder; diagnosis; machine learning; mobile phone; primary care.

Conflict of interest statement

Conflicts of Interest: KS has received consulting fees from Cognoa, is on the Medical Advisory Board for Quadrant Biosciences, and provides research support for Autism Speaks. SL-M, JS, CS, and ST are employees of Cognoa and have Cognoa stock options. ST additionally receives consulting fees for Cognito Therapeutics, volunteers as a board member of the American Academy of Pediatrics-Orange County Chapter and American Academy of Pediatrics California, is a paid adviser for MI10 LLC, and owns stock for NTX, Inc, and HandzIn.

©Kristin Sohl, Rachel Kilian, Alicia Brewer Curran, Melissa Mahurin, Valeria Nanclares-Nogués, Stuart Liu-Mayo, Carmela Salomon, Jennifer Shannon, Sharief Taraman. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 19.07.2022.

Figures

Figure 1
Figure 1
Overview of device use. ASD: autism spectrum disorder; HCP: health care provider.
Figure 2
Figure 2
Study flow. ASD: autism spectrum disorder; ECHO: Extension for Community Health Outcomes; HCP: health care provider.

References

    1. Maenner Matthew J, Shaw Kelly A, Bakian Amanda V, Bilder Deborah A, Durkin Maureen S, Esler Amy, Furnier Sarah M, Hallas Libby, Hall-Lande Jennifer, Hudson Allison, Hughes Michelle M, Patrick Mary, Pierce Karen, Poynter Jenny N, Salinas Angelica, Shenouda Josephine, Vehorn Alison, Warren Zachary, Constantino John N, DiRienzo Monica, Fitzgerald Robert T, Grzybowski Andrea, Spivey Margaret H, Pettygrove Sydney, Zahorodny Walter, Ali Akilah, Andrews Jennifer G, Baroud Thaer, Gutierrez Johanna, Hewitt Amy, Lee Li-Ching, Lopez Maya, Mancilla Kristen Clancy, McArthur Dedria, Schwenk Yvette D, Washington Anita, Williams Susan, Cogswell Mary E. Prevalence and Characteristics of Autism Spectrum Disorder Among Children Aged 8 Years - Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2018. MMWR Surveill Summ. 2021 Dec 03;70(11):1–16. doi: 10.15585/mmwr.ss7011a1.
    1. Ben Itzchak E, Zachor DA. Who benefits from early intervention in autism spectrum disorders? Res Autism Spectrum Disorders. 2011 Jan;5(1):345–50. doi: 10.1016/j.rasd.2010.04.018.
    1. Flanagan H, Perry A, Freeman N. Effectiveness of large-scale community-based Intensive Behavioral Intervention: a waitlist comparison study exploring outcomes and predictors. Res Autism Spectrum Disorders. 2012 Apr;6(2):673–82. doi: 10.1016/j.rasd.2011.09.011.
    1. Smith T, Klorman R, Mruzek DW. Predicting outcome of community-based early intensive behavioral intervention for children with autism. J Abnorm Child Psychol. 2015 Oct 17;43(7):1271–82. doi: 10.1007/s10802-015-0002-2.
    1. Dawson G, Rogers S, Munson J, Smith M, Winter J, Greenson J, Donaldson A, Varley J. Randomized, controlled trial of an intervention for toddlers with autism: the Early Start Denver Model. Pediatrics. 2010 Jan;125(1):e17–23. doi: 10.1542/peds.2009-0958. peds.2009-0958
    1. Baker-Ericzén M, Stahmer A, Burns A. Child demographics associated with outcomes in a community-based pivotal response training program. J Positive Behav Interv. 2016 Aug 13;9(1):52–60. doi: 10.1177/10983007070090010601.
    1. Mazurek MO, Kanne SM, Miles JH. Predicting improvement in social–communication symptoms of autism spectrum disorders using retrospective treatment data. Res Autism Spectrum Disorder. 2012 Jan;6(1):535–45. doi: 10.1016/j.rasd.2011.07.014.
    1. MacDonald R, Parry-Cruwys D, Dupere S, Ahearn W. Assessing progress and outcome of early intensive behavioral intervention for toddlers with autism. Res Dev Disabil. 2014 Dec;35(12):3632–44. doi: 10.1016/j.ridd.2014.08.036.S0891-4222(14)00389-8
    1. Vivanti G, Dissanayake C, Victorian ASELCC Team Outcome for children receiving the early start denver model before and after 48 months. J Autism Dev Disord. 2016 Jul 28;46(7):2441–9. doi: 10.1007/s10803-016-2777-6.10.1007/s10803-016-2777-6
    1. Elsabbagh M, Divan G, Koh Y, Kim YS, Kauchali S, Marcín C, Montiel-Nava C, Patel V, Paula CS, Wang C, Yasamy MT, Fombonne E. Global prevalence of autism and other pervasive developmental disorders. Autism Res. 2012 Jun 11;5(3):160–79. doi: 10.1002/aur.239. doi: 10.1002/aur.239.
    1. Gordon-Lipkin E, Foster J, Peacock G. Whittling down the wait time: exploring models to minimize the delay from initial concern to diagnosis and treatment of autism spectrum disorder. Pediatr Clin North Am. 2016 Oct;63(5):851–9. doi: 10.1016/j.pcl.2016.06.007. S0031-3955(16)41030-8
    1. Hyman S, Levy S, Myers S, Council on Children With Disabilities‚ Section on Developmental and Behavioral Pediatrics Identification, evaluation, and management of children with autism spectrum disorder. Pediatrics. 2020 Jan;145(1):e20193447. doi: 10.1542/peds.2019-3447.peds.2019-3447
    1. Bridgemohan C, Bauer N, Nielsen B, DeBattista A, Ruch-Ross H, Paul L, Roizen N. A workforce survey on developmental-behavioral pediatrics. Pediatrics. 2018 Mar;141(3):e20172164. doi: 10.1542/peds.2017-2164.peds.2017-2164
    1. Baio J, Wiggins L, Christensen DL, Maenner MJ, Daniels J, Warren Z, Kurzius-Spencer M, Zahorodny W, Robinson Rosenberg C, White T, Durkin MS, Imm P, Nikolaou L, Yeargin-Allsopp M, Lee LC, Harrington R, Lopez M, Fitzgerald RT, Hewitt A, Pettygrove S, Constantino JN, Vehorn A, Shenouda J, Hall-Lande J, Van Naarden Braun K, Dowling NF. Prevalence of autism spectrum disorder among children aged 8 years - autism and developmental disabilities monitoring network, 11 sites, United States, 2014. MMWR Surveill Summ. 2018 Apr 27;67(6):1–23. doi: 10.15585/mmwr.ss6706a1.
    1. van 't Hof M, Tisseur C, van Berckelear-Onnes I, van Nieuwenhuyzen A, Daniels A, Deen M, Hoek H, Ester W. Age at autism spectrum disorder diagnosis: a systematic review and meta-analysis from 2012 to 2019. Autism. 2021 May;25(4):862–73. doi: 10.1177/1362361320971107.
    1. Pierce K, Gazestani VH, Bacon E, Barnes CC, Cha D, Nalabolu S, Lopez L, Moore A, Pence-Stophaeros S, Courchesne E. Evaluation of the diagnostic stability of the early autism spectrum disorder phenotype in the general population starting at 12 months. JAMA Pediatr. 2019 Jun 01;173(6):578–87. doi: 10.1001/jamapediatrics.2019.0624. 2732144
    1. Delobel-Ayoub M, Ehlinger V, Klapouszczak D, Maffre T, Raynaud J, Delpierre C, Arnaud C. Socioeconomic disparities and prevalence of autism spectrum disorders and intellectual disability. PLoS One. 2015 Nov 5;10(11):e0141964. doi: 10.1371/journal.pone.0141964. PONE-D-15-00583
    1. Oswald D, Haworth S, Mackenzie B, Willis J. Parental report of the diagnostic process and outcome: ASD compared with other developmental disabilities. Focus Autism Other Dev Disabl. 2015 Jun 09;32(2):152–60. doi: 10.1177/1088357615587500.
    1. Wiggins LD, Durkin M, Esler A, Lee L, Zahorodny W, Rice C, Yeargin-Allsopp M, Dowling NF, Hall-Lande J, Morrier MJ, Christensen D, Shenouda J, Baio J. Disparities in documented diagnoses of autism spectrum disorder based on demographic, individual, and service factors. Autism Res. 2020 Mar 23;13(3):464–73. doi: 10.1002/aur.2255.
    1. Shattuck PT, Durkin M, Maenner M, Newschaffer C, Mandell DS, Wiggins L, Lee L, Rice C, Giarelli E, Kirby R, Baio J, Pinto-Martin J, Cuniff C. Timing of identification among children with an autism spectrum disorder: findings from a population-based surveillance study. J Am Academy Child Adolescent Psychiatry. 2009 May;48(5):474–83. doi: 10.1097/chi.0b013e31819b3848.
    1. Monteiro SA, Dempsey J, Berry LN, Voigt RG, Goin-Kochel RP. Screening and referral practices for autism spectrum disorder in primary pediatric care. Pediatrics. 2019 Oct 12;144(4):e20183326. doi: 10.1542/peds.2018-3326.peds.2018-3326
    1. Rhoades RA, Scarpa A, Salley B. The importance of physician knowledge of autism spectrum disorder: results of a parent survey. BMC Pediatr. 2007 Nov 20;7(1):37. doi: 10.1186/1471-2431-7-37. 1471-2431-7-37
    1. Carbone P, Norlin C, Young P. Improving early identification and ongoing care of children with autism spectrum disorder. Pediatrics. 2016 Jun;137(6):e20151850. doi: 10.1542/peds.2015-1850.peds.2015-1850
    1. Fenikilé T, Ellerbeck K, Filippi M, Daley C. Barriers to autism screening in family medicine practice: a qualitative study. Prim Health Care Res Dev. 2014 Nov 4;16(04):356–66. doi: 10.1017/s1463423614000449.
    1. Self T, Parham D, Rajagopalan J. Autism spectrum disorder early screening practices. Commun Disorder Q. 2014 Dec 30;36(4):195–207. doi: 10.1177/1525740114560060.
    1. Mazurek MO, Harkins C, Menezes M, Chan J, Parker RA, Kuhlthau K, Sohl K. Primary care providers' perceived barriers and needs for support in caring for children with autism. J Pediatr. 2020 Jun;221:240–5.e1. doi: 10.1016/j.jpeds.2020.01.014.S0022-3476(20)30027-5
    1. Lai M, Kassee C, Besney R, Bonato S, Hull L, Mandy W, Szatmari P, Ameis SH. Prevalence of co-occurring mental health diagnoses in the autism population: a systematic review and meta-analysis. Lancet Psychiatry. 2019 Oct;6(10):819–29. doi: 10.1016/s2215-0366(19)30289-5.
    1. Falkmer T, Anderson K, Falkmer M, Horlin C. Diagnostic procedures in autism spectrum disorders: a systematic literature review. Eur Child Adolesc Psychiatry. 2013 Jun 16;22(6):329–40. doi: 10.1007/s00787-013-0375-0.
    1. Lord C, Rutter M, Le Couteur A. Autism diagnostic interview-revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. J Autism Dev Disord. 1994 Oct;24(5):659–85. doi: 10.1007/bf02172145.
    1. (ADOS®-2) Autism Diagnostic Observation Schedule™, Second Edition. WPS Publish. [2021-12-14]. .
    1. Kaufman NK. Rethinking "gold standards" and "best practices" in the assessment of autism. Appl Neuropsychol Child. 2022 Aug 27;11(3):529–40. doi: 10.1080/21622965.2020.1809414.
    1. Software as a Medical Device (SaMD) U.S. Food & Drug Administration. [2021-12-12]. .
    1. Abbas H, Garberson F, Liu-Mayo S, Glover E, Wall DP. Multi-modular AI approach to streamline autism diagnosis in young children. Sci Rep. 2020 Mar 19;10(1):5014. doi: 10.1038/s41598-020-61213-w. doi: 10.1038/s41598-020-61213-w.10.1038/s41598-020-61213-w
    1. Abbas H, Garberson F, Glover E, Wall D. Machine learning approach for early detection of autism by combining questionnaire and home video screening. J Am Med Inform Assoc. 2018 Aug 01;25(8):1000–7. doi: 10.1093/jamia/ocy039. 4993666
    1. Levy S, Duda M, Haber N, Wall DP. Sparsifying machine learning models identify stable subsets of predictive features for behavioral detection of autism. Mol Autism. 2017 Dec 19;8(1):65. doi: 10.1186/s13229-017-0180-6. 180
    1. Tariq Q, Daniels J, Schwartz J, Washington P, Kalantarian H, Wall D. Mobile detection of autism through machine learning on home video: a development and prospective validation study. PLoS Med. 2018 Nov;15(11):e1002705. doi: 10.1371/journal.pmed.1002705. PMEDICINE-D-18-01991
    1. Wall DP, Dally R, Luyster R, Jung J, Deluca TF. Use of artificial intelligence to shorten the behavioral diagnosis of autism. PLoS One. 2012 Aug 27;7(8):e43855. doi: 10.1371/journal.pone.0043855. PONE-D-11-13905
    1. Kosmicki JA, Sochat V, Duda M, Wall DP. Searching for a minimal set of behaviors for autism detection through feature selection-based machine learning. Transl Psychiatry. 2015 Mar 24;5(2):e514. doi: 10.1038/tp.2015.7. doi: 10.1038/tp.2015.7.tp20157
    1. Megerian JT, Dey S, Melmed RD, Coury DL, Lerner M, Nicholls CJ, Sohl K, Rouhbakhsh R, Narasimhan A, Romain J, Golla S, Shareef S, Ostrovsky A, Shannon J, Kraft C, Liu-Mayo S, Abbas H, Gal-Szabo DE, Wall DP, Taraman S. Evaluation of an artificial intelligence-based medical device for diagnosis of autism spectrum disorder. NPJ Digit Med. 2022 May 05;5(1):57. doi: 10.1038/s41746-022-00598-6. doi: 10.1038/s41746-022-00598-6.10.1038/s41746-022-00598-6
    1. Cognoa receives FDA marketing authorization for first-of-its-kind Autism Diagnosis Aid. Cognoa. [2021-10-10].
    1. Mazurek M, Brown R, Curran A, Sohl K. ECHO Autism. Clin Pediatr (Phila) 2017 Mar;56(3):247–56. doi: 10.1177/0009922816648288.0009922816648288
    1. Sohl K, Mazurek M, Brown R. ECHO Autism: using technology and mentorship to bridge gaps, increase access to care, and bring best practice autism care to primary care. Clin Pediatr (Phila) 2017 Jun;56(6):509–11. doi: 10.1177/0009922817691825.
    1. Mazurek M, Curran A, Burnette C, Sohl K. ECHO Autism STAT: accelerating early access to autism diagnosis. J Autism Dev Disord. 2019 Jan;49(1):127–37. doi: 10.1007/s10803-018-3696-5.10.1007/s10803-018-3696-5
    1. Guan X, Zwaigenbaum L, Sonnenberg L. Building capacity for community pediatric autism diagnosis: a systemic review of physician training programs. J Dev Behav Pediatr. 2021 Dec 15;43(1):44–54. doi: 10.1097/dbp.0000000000001042.
    1. Bishop-Fitzpatrick L, Kind AJ. A scoping review of health disparities in autism spectrum disorder. J Autism Dev Disord. 2017 Nov 29;47(11):3380–91. doi: 10.1007/s10803-017-3251-9. 10.1007/s10803-017-3251-9
    1. Nazneen N, Rozga A, Romero M, Findley AJ, Call NA, Abowd GD, Arriaga RI. Supporting parents for in-home capture of problem behaviors of children with developmental disabilities. Pers Ubiquit Comput. 2011 May 1;16(2):193–207. doi: 10.1007/s00779-011-0385-1.
    1. Constantino JN, Abbacchi AM, Saulnier C, Klaiman C, Mandell DS, Zhang Y, Hawks Z, Bates J, Klin A, Shattuck P, Molholm S, Fitzgerald R, Roux A, Lowe JK, Geschwind DH. Timing of the diagnosis of autism in African American children. Pediatrics. 2020 Sep 24;146(3):e20193629. doi: 10.1542/peds.2019-3629. peds.2019-3629
    1. Yingling ME, Hock RM, Bell BA. Time-lag between diagnosis of autism spectrum disorder and onset of publicly-funded early intensive behavioral intervention: do race-ethnicity and neighborhood matter? J Autism Dev Disord. 2018 Feb 28;48(2):561–71. doi: 10.1007/s10803-017-3354-3.10.1007/s10803-017-3354-3
    1. Mobile Fact Sheet. Pew Research Center. [2021-12-11].

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe