Diagnostic Efficacy and Therapeutic Decision-making Capacity of an Artificial Intelligence Platform for Childhood Cataracts in Eye Clinics: A Multicentre Randomized Controlled Trial

Haotian Lin, Ruiyang Li, Zhenzhen Liu, Jingjing Chen, Yahan Yang, Hui Chen, Zhuoling Lin, Weiyi Lai, Erping Long, Xiaohang Wu, Duoru Lin, Yi Zhu, Chuan Chen, Dongxuan Wu, Tongyong Yu, Qianzhong Cao, Xiaoyan Li, Jing Li, Wangting Li, Jinghui Wang, Mingmin Yang, Huiling Hu, Li Zhang, Yang Yu, Xuelan Chen, Jianmin Hu, Ke Zhu, Shuhong Jiang, Yalin Huang, Gang Tan, Jialing Huang, Xiaoming Lin, Xinyu Zhang, Lixia Luo, Yuhua Liu, Xialin Liu, Bing Cheng, Danying Zheng, Mingxing Wu, Weirong Chen, Yizhi Liu, Haotian Lin, Ruiyang Li, Zhenzhen Liu, Jingjing Chen, Yahan Yang, Hui Chen, Zhuoling Lin, Weiyi Lai, Erping Long, Xiaohang Wu, Duoru Lin, Yi Zhu, Chuan Chen, Dongxuan Wu, Tongyong Yu, Qianzhong Cao, Xiaoyan Li, Jing Li, Wangting Li, Jinghui Wang, Mingmin Yang, Huiling Hu, Li Zhang, Yang Yu, Xuelan Chen, Jianmin Hu, Ke Zhu, Shuhong Jiang, Yalin Huang, Gang Tan, Jialing Huang, Xiaoming Lin, Xinyu Zhang, Lixia Luo, Yuhua Liu, Xialin Liu, Bing Cheng, Danying Zheng, Mingxing Wu, Weirong Chen, Yizhi Liu

Abstract

Background: CC-Cruiser is an artificial intelligence (AI) platform developed for diagnosing childhood cataracts and providing risk stratification and treatment recommendations. The high accuracy of CC-Cruiser was previously validated using specific datasets. The objective of this study was to compare the diagnostic efficacy and treatment decision-making capacity between CC-Cruiser and ophthalmologists in real-world clinical settings.

Methods: This multicentre randomized controlled trial was performed in five ophthalmic clinics in different areas across China. Pediatric patients (aged ≤ 14 years) without a definitive diagnosis of cataracts or history of previous eye surgery were randomized (1:1) to receive a diagnosis and treatment recommendation from either CC-Cruiser or senior consultants (with over 5 years of clinical experience in pediatric ophthalmology). The experts who provided a gold standard diagnosis, and the investigators who performed slit-lamp photography and data analysis were blinded to the group assignments. The primary outcome was the diagnostic performance for childhood cataracts with reference to cataract experts' standards. The secondary outcomes included the evaluation of disease severity and treatment determination, the time required for the diagnosis, and patient satisfaction, which was determined by the mean rating. This trial is registered with ClinicalTrials.gov (NCT03240848).

Findings: Between August 9, 2017 and May 25, 2018, 350 participants (700 eyes) were randomly assigned for diagnosis by CC-Cruiser (350 eyes) or senior consultants (350 eyes). The accuracies of cataract diagnosis and treatment determination were 87.4% and 70.8%, respectively, for CC-Cruiser, which were significantly lower than 99.1% and 96.7%, respectively, for senior consultants (p < 0.001, OR = 0.06 [95% CI 0.02 to 0.19]; and p < 0.001, OR = 0.08 [95% CI 0.03 to 0.25], respectively). The mean time for receiving a diagnosis from CC-Cruiser was 2.79 min, which was significantly less than 8.53 min for senior consultants (p < 0.001, mean difference 5.74 [95% CI 5.43 to 6.05]). The patients were satisfied with the overall medical service quality provided by CC-Cruiser, typically with its time-saving feature in cataract diagnosis.

Interpretation: CC-Cruiser exhibited less accurate performance comparing to senior consultants in diagnosing childhood cataracts and making treatment decisions. However, the medical service provided by CC-Cruiser was less time-consuming and achieved a high level of patient satisfaction. CC-Cruiser has the capacity to assist human doctors in clinical practice in its current state.

Funding: National Key R&D Program of China (2018YFC0116500) and the Key Research Plan for the National Natural Science Foundation of China in Cultivation Project (91846109).

Keywords: Artificial intelligence; Childhood cataracts; Multicentre randomized controlled trial; Ophthalmology.

Figures

Fig. 1
Fig. 1
Trial profile. AI = artificial intelligence.

References

    1. Gulshan V., Peng L., Coram M. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316(22):2402–2410.
    1. Kim D.H., Kim H., Kwak S. The settings, pros and cons of the new surgical robot da Vinci Xi system for transoral robotic surgery (TORS): a comparison with the popular da Vinci Si system. Surg Laparosc Endosc Percutan Tech. 2016;26(5):391–396.
    1. Curioni-Fontecedro A. A new era of oncology through artificial intelligence. ESMO Open. 2017;2(2)
    1. Hamet P., Tremblay J. Artificial intelligence in medicine. Metabolism. 2017;69S:S36–S40.
    1. Somashekhar S.P., Sepulveda M.J., Puglielli S. Watson for oncology and breast cancer treatment recommendations: agreement with an expert multidisciplinary tumor board. Ann Oncol. 2018;29(2):418–423.
    1. Esteva A., Kuprel B., Novoa R.A. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115–118.
    1. Printz C. Artificial intelligence platform for oncology could assist in treatment decisions. Cancer. 2017;123(6):905.
    1. Wang C., Qi X., Chen X., Yu Q., Xing L. The establishment of China standardized residency training system. Zhonghua Yi Xue Za Zhi. 2015;95(14):1041–1043.
    1. Jiang Y., Luo L., Congdon N., Wang S., Liu Y. Who will be wielding the lancet for China's patients in the future? The Lancet. 2016;388(10054):1952–1954.
    1. Wang Y.X. On the training of young doctors in China. Quant Imaging Med Surg. 2015;5(1):182–185.
    1. Azad N., Amos S., Milne K., Power B. Telemedicine in a rural memory disorder clinic-remote management of patients with dementia. Can Geriatr J. 2012;15(4):96–100.
    1. Castaneda C., Nalley K., Mannion C. Clinical decision support systems for improving diagnostic accuracy and achieving precision medicine. J Clin Bioinforma. 2015;5:4.
    1. Patel V.L., Shortliffe E.H., Stefanelli M. The coming of age of artificial intelligence in medicine. Artif Intell Med. 2009;46(1):5–17.
    1. Hashmi S. 'Coming of Age' of artificial intelligence: evolution of survivorship care through information technology. Bone Marrow Transplant. 2016;51(1):41–42.
    1. Raju M., Pagidimarri V., Barreto R., Kadam A., Kasivajjala V., Aswath A. Development of a deep learning algorithm for automatic diagnosis of diabetic retinopathy. Stud Health Technol Inform. 2017;245:559–563.
    1. Burlina P.M., Joshi N., Pekala M., Pacheco K.D., Freund D.E., Bressler N.M. Automated grading of age-related macular degeneration from color fundus images using deep convolutional neural networks. JAMA Ophthalmol. 2017;135(11):1170–1176.
    1. Ting D.S.W., Cheung C.Y., Lim G. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA. 2017;318(22):2211–2223.
    1. Poplin R., Varadarajan A.V., Blumer K. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nature Biomedical Engineering. 2018;2(3):158–164.
    1. Kermany D.S., Goldbaum M., Cai W. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell. 2018;172(5):1122–31 e9.
    1. Schieppati A., Henter J-I., Daina E., Aperia A. Why rare diseases are an important medical and social issue. The Lancet. 2008;371(9629):2039–2041.
    1. Taruscio D., Vittozzi L., Stefanov R. 2010. National plans and strategies on rare diseases in Europe.
    1. Remuzzi G., Garattini S. 2008. Rare diseases: what's next?
    1. Lenhart P, Courtright P, Edward Wilson M, et al. Global challenges in the management of congenital cataract: proceedings of the 4th international congenital cataract symposium held on March 7, 2014, New York, New York; 2015.
    1. Medsinge A., Nischal K.K. Pediatric cataract: challenges and future directions. Clin Ophthalmol. 2015;9:77–90.
    1. Long E., Lin H., Liu Z. An artificial intelligence platform for the multihospital collaborative management of congenital cataracts. Nature Biomedical Engineering. 2017;1(2):0024.
    1. Rodger M., Ramsay T., Fergusson D. Diagnostic randomized controlled trials: the final frontier. Trials. 2012;13:137.
    1. Chen Y.L., Yang K.H. Consort 2010. Lancet. 2010;376(9737):230.
    1. Augestad K.M., Berntsen G., Lassen K. Standards for reporting randomized controlled trials in medical informatics: a systematic review of CONSORT adherence in RCTs on clinical decision support. J Am Med Inform Assoc. 2012;19(1):13–21.
    1. Abdul Latif L., Daud Amadera J.E., Pimentel D., Pimentel T., Fregni F. Sample size calculation in physical medicine and rehabilitation: a systematic review of reporting, characteristics, and results in randomized controlled trials. Arch Phys Med Rehabil. 2011;92(2):306–315.
    1. Bacchetti P., Leung J.M. Sample size calculations in clinical research. Anesthesiology. 2002;97(4):1028–1029. [author reply 9–32]
    1. Liang K.-Y., Zeger S.L. Longitudinal data analysis using generalized linear models. Biometrika. 1986;73(1):13–22.
    1. Tufail A., Rudisill C., Egan C. Automated diabetic retinopathy image assessment software: diagnostic accuracy and cost-effectiveness compared with human graders. Ophthalmology. 2017;124(3):343–351.
    1. Gossec L., Cantagrel A., Soubrier M. An e-health interactive self-assessment website (Sanoia((R))) in rheumatoid arthritis. A 12-month randomized controlled trial in 320 patients. Joint Bone Spine. 2018;85(6):709–714.
    1. Wakeman B., MacDonald I.M., Ginjaar I., Tarleton J., Babu D. Extraocular muscle hypertrophy in myotonia congenita: mutation identified in the SCN4A gene (V445M) J AAPOS. 2009;13(5):526–527.

Source: PubMed

Patrocinadores y Colaboradores

Condiciones médicas

Intervenciones de drogas

3
Suscribir