Evaluating the Performance of Various Machine Learning Algorithms to Detect Subclinical Keratoconus

Ke Cao, Karin Verspoor, Srujana Sahebjada, Paul N Baird, Ke Cao, Karin Verspoor, Srujana Sahebjada, Paul N Baird

Abstract

Purpose: Keratoconus (KC) represents one of the leading causes of corneal transplantation worldwide. Detecting subclinical KC would lead to better management to avoid the need for corneal grafts, but the condition is clinically challenging to diagnose. We wished to compare eight commonly used machine learning algorithms using a range of parameter combinations by applying them to our KC dataset and build models to better differentiate subclinical KC from non-KC eyes.

Methods: Oculus Pentacam was used to obtain corneal parameters on 49 subclinical KC and 39 control eyes, along with clinical and demographic parameters. Eight machine learning methods were applied to build models to differentiate subclinical KC from control eyes. Dominant algorithms were trained with all combinations of the considered parameters to select important parameter combinations. The performance of each model was evaluated and compared.

Results: Using a total of eleven parameters, random forest, support vector machine and k-nearest neighbors had better performance in detecting subclinical KC. The highest area under the curve of 0.97 for detecting subclinical KC was achieved using five parameters by the random forest method. The highest sensitivity (0.94) and specificity (0.90) were obtained by the support vector machine and the k-nearest neighbor model, respectively.

Conclusions: This study showed machine learning algorithms can be applied to identify subclinical KC using a minimal parameter set that are routinely collected during clinical eye examination.

Translational relevance: Machine learning algorithms can be built using routinely collected clinical parameters that will assist in the objective detection of subclinical KC.

Keywords: artificial intelligence; keratoconus; machine learning; subclinical keratoconus.

Conflict of interest statement

Disclosure: K. Cao, None; K. Verspoor, None; S. Sahebjada, None; P.N. Baird, None

Copyright 2020 The Authors.

Figures

Figure 1.
Figure 1.
The 10-fold cross validation for analysis of test data. Twenty rhombuses are randomly partitioned into 10 subsets, with two rhombuses in each subset. Of the 10 subsets, one subset is retained as the validation data, and the remaining nine subsets are used to train the model. This cross-validation process is then repeated 10 times. In summary, cross-validation combines measures of 10 fitness and provide an average.
Figure 2.
Figure 2.
Training machine learning models with different parameter sets. All possible combination of 11 parameters, from combination of two (e.g., gender and age, gender and SE) parameters to combination of 11 parameters, were used as input respectively to train machine learning models to differentiate subclinical KC eyes from control eyes.

References

    1. Sharif R, Bak-Nielsen S, Hjortdal J, Karamichos D, Progress in retinal and eye research. 2018; 67: 150–167.
    1. Reeves SW, Ellwein LB, Kim T, Constantine R, Lee PP. Keratoconus in the Medicare population. Cornea. 2009; 28: 40–42.
    1. Torres Netto EA, Al-Otaibi WM, Hafezi NL, et al. .. Prevalence of keratoconus in paediatric patients in Riyadh, Saudi Arabia. Br J Ophthalmol. 2018; 102: 1436–1441.
    1. Kennedy RH, Bourne WM, Dyer JA. A 48-year clinical and epidemiologic study of keratoconus. J Ophthal. 1986; 101: 267–273.
    1. Godefrooij DA, De Wit GA, Uiterwaal CS, Imhof SM, Wisse RP. Age-specific incidence and prevalence of keratoconus: a nationwide registration study. Am J ophthal. 2017; 175: 169–172.
    1. Serdarogullari H, Tetikoglu M, Karahan H, Altin F, Elcioglu M. Research v. Prevalence of keratoconus and subclinical keratoconus in subjects with astigmatism using pentacam derived parameters. J Ophthalmic Vis Res. 2013; 8: 213.
    1. Hashemi H, Beiranvand A, Khabazkhoob M, et al. .. Prevalence of keratoconus in a population-based study in Shahroud. Cornea. 2013; 32: 1441–1445.
    1. Hashemi H, Khabazkhoob M, Yazdani N, Ostadimoghaddam H, Norouzirad R, Amanzadeh K, et al. .. The prevalence of keratoconus in a young population in Mashhad, Iran. Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians (Optometrists). 2014; 34: 519–527.
    1. Hashemi H, Heydarian S, Yekta A, et al. .. High prevalence and familial aggregation of keratoconus in an Iranian rural population: a population-based study. Ophthal Physiol Optics. 2018; 38: 447–455.
    1. Millodot M, Shneor E, Albou S, Atlani E, Gordon-Shaag A. Prevalence and associated factors of keratoconus in Jerusalem: a cross-sectional study. Ophthal Epidemiol. 2011; 18: 91–97.
    1. Shneor E, Millodot M, Gordon-Shaag A, et al. .. Prevalence of keratoconus among young Arab students in Israel. 2014; 3: 9.
    1. Hashemi H, Heydarian S, Hooshmand E, et al. .. The prevalence and risk factors for keratoconus: a systematic review and meta-analysis. Cornea. 2020; 39: 263–70.
    1. Olivares Jimenez JL, Guerrero Jurado JC, Bermudez Rodriguez FJ, Serrano Laborda D. Keratoconus: age of onset and natural history. Optom Vision Sci. 1997; 74: 147–151.
    1. Röck T, Bartz-Schmidt KU, Röck DJPo. Trends in corneal transplantation at the University Eye Hospital in Tübingen, Germany over the last 12 years: 2004–2015. PloS One. 2018; 13: e0198793.
    1. Williams KA, Keane MC, Coffey NE, Jones VJ, Mills RA, Coster DJ The Australian Corneal Graft Registry 2018 Report. 2018.
    1. Tan P, Mehta JS. Collagen crosslinking for keratoconus. J Ophthalmic Vis Res. 2011; 6: 153–154.
    1. Godefrooij DA, Mangen MJ, Chan E, O'Brart DPS, Imhof SM, de Wit GA, et al. .. Cost-effectiveness analysis of corneal collagen crosslinking for progressive keratoconus. Ophthalmology. 2017; 124: 1485–1495.
    1. Accardo PA, Pensiero S. Neural network-based system for early keratoconus detection from corneal topography. J Biomed Informatics. 2002; 35: 151–159.
    1. Arbelaez MC, Versaci F, Vestri G, Barboni P, Savini G. Use of a support vector machine for keratoconus and subclinical keratoconus detection by topographic and tomographic data. Ophthalmology. 2012; 119: 2231–2238.
    1. Issarti I, Consejo A, Jimenez-Garcia M, Hershko S, Koppen C, Rozema JJ. Computer aided diagnosis for suspect keratoconus detection. Computers Biol Med. 2019; 109: 33–42.
    1. Kovacs I, Mihaltz K, Kranitz K, et al. .. Accuracy of machine learning classifiers using bilateral data from a Scheimpflug camera for identifying eyes with preclinical signs of keratoconus. J Cataract Refractive Surg. 2016; 42: 275–283.
    1. Lopes BT, Ramos IC, Salomao MQ, Guerra FP, Schallhorn SC, Schallhorn JM, et al. .. Enhanced Tomographic Assessment to Detect Corneal Ectasia Based on Artificial Intelligence. Am J Ophthal. 2018; 195: 223–32.
    1. Ruiz Hidalgo I, Rodriguez P, Rozema JJ, Ni Dhubhghaill S, Zakaria N, Tassignon MJ, et al. .. Evaluation of a machine-learning classifier for keratoconus detection based on Scheimpflug tomography. Cornea. 2016; 35: 827–832.
    1. Saad A, Gatinel D. Topographic and tomographic properties of forme fruste keratoconus corneas. Invest Ophthal Visual Sci. 2010; 51: 5546–5555.
    1. Smadja D, Touboul D, Cohen A, et al. .. Detection of subclinical keratoconus using an automated decision tree classification. Am J Ophthal. 2013; 156: 237–246.
    1. Ucakhan OO, Cetinkor V, Ozkan M, Kanpolat A. Evaluation of Scheimpflug imaging parameters in subclinical keratoconus, keratoconus, and normal eyes. J Cataract Refractive Surg. 2011; 37: 1116–1124.
    1. Ventura BV, Machado AP, Ambrosio R Jr., et al. .. Analysis of waveform-derived ORA parameters in early forms of keratoconus and normal corneas. J Refractive Surg (Thorofare, NJ: 1995). 2013; 29: 637–643.
    1. Mahmoud AM, Roberts CJ, Lembach RG, Twa MD, Herderick EE, McMahon TT. CLMI: the cone location and magnitude index. Cornea. 2008; 27: 480–487.
    1. Qin B, Chen S, Brass R, et al. .. Keratoconus diagnosis with optical coherence tomography-based pachymetric scoring system. J Cataract Refractive Surg. 2013; 39: 1864–1871.
    1. Hwang ES, Perez-Straziota CE, Kim SW, Santhiago MR, Randleman JBJO. Distinguishing highly asymmetric keratoconus eyes using combined Scheimpflug and spectral-domain OCT analysis. Ophthalmology. 2018; 125: 1862–71.
    1. Podgorelec V, Kokol P, Stiglic B, Rozman IJJoms. Decision trees: an overview and their use in medicine. 2002; 26: 445–463.
    1. Chastang PJ, Borderie VM, Carvajal-Gonzalez S, Rostene W, Laroche L. Automated keratoconus detection using the EyeSys videokeratoscope. J Cataract Refractive Surg. 2000; 26: 675–683.
    1. Twa MD, Parthasarathy S, Roberts C, Mahmoud AM, Raasch TW, Bullimore MA. Automated decision tree classification of corneal shape. Optom and Vision Sci. 2005; 82: 1038–1046.
    1. Ambrosio R Jr. Lopes BT, Faria-Correia F, et al. .. Integration of Scheimpflug-Based Corneal Tomography and Biomechanical Assessments for Enhancing Ectasia Detection. Journal of refractive surgery (Thorofare, NJ: 1995). 2017; 33: 434–443.
    1. Breiman L. Random forests. Machine Learn. 2001; 45: 5–32.
    1. Saad A, Guilbert E, Gatinel D. Corneal enantiomorphism in normal and keratoconic eyes. J Refractive Surg (Thorofare, NJ: 1995). 2014; 30: 542–547.
    1. Silva APD, Stam A. Discriminant analysis.Reading and understanding multivariate statistics; Washington, DC, US: APA; 1995: 277–318.
    1. Hearst MA. Support vector machines. J IEEE Intelligent Sys. 1998; 13: 18–28.
    1. Ramchoun H, Idrissi MAJ, Ghanou Y, Ettaouil M. Multilayer perceptron: architecture optimization and training with mixed activation functions. Proceedings of the 2nd international Conference on Big Data, Cloud and Applications; Tetouan, Morocco: 3090427: ACM; 2017. p. 1–6.
    1. Carvalho LA. Preliminary results of neural networks and zernike polynomials for classification of videokeratography maps. Optom and Vision Sci. 2005; 82: 151–158.
    1. Maeda N, Klyce SD, Smolek MK. Neural network classification of corneal topography. Preliminary demonstration. Invest Ophthal Visual Sci. 1995; 36: 1327–1335.
    1. Smolek MK, Klyce SD. Current keratoconus detection methods compared with a neural network approach. Invest Ophthal Visual Sci. 1997; 38: 2290–2299.
    1. Souza MB, Medeiros FW, Souza DB, Garcia R, Alves MR. Evaluation of machine learning classifiers in keratoconus detection from orbscan II examinations. Clinics (Sao Paulo). 2010; 65: 1223–1228.
    1. Rafati S, Hashemi H, Nabovati P, et al. .. Demographic profile, clinical, and topographic characteristics of keratoconus patients attending at a tertiary eye center. J Curr Ophthalmol. 2019; 31: 268–74.
    1. Rodriguez LA, Villegas AE, Porras D, Benavides MA, Molina J. Treatment of six cases of advanced ectasia after LASIK with 6-mm Intacs SK. J Refractive Surg (Thorofare, NJ: 1995). 2009; 25: 1116–1119.
    1. Ernst BJ, Hsu HYJE, Lens c. Keratoconus association with axial myopia: a prospective biometric study. Eye & Contact Lens. 2011; 37: 2–5.
    1. Sahebjada S, Xie J, Chan E, Snibson G, Daniel M, Baird PN. Assessment of anterior segment parameters of keratoconus eyes in an Australian population. Optom and Vision Sci. 2014; 91: 803–809.
    1. Abolbashari F, Mohidin N, Ahmadi Hosseini SM, Mohd Ali B, Retnasabapathy S. Anterior segment characteristics of keratoconus eyes in a sample of Asian population. Cont Lens Anterior Eye. 2013; 36: 191–195.
    1. Emre S, Doganay S, Yologlu S. Evaluation of anterior segment parameters in keratoconic eyes measured with the Pentacam system. J Cataract Refract Surg. 2007; 33: 1708–1712.
    1. Piñero DP, Nieto JC, Lopez-Miguel A. Characterization of corneal structure in keratoconus. J Cataract Refractive Surg. 2012; 38: 2167–2183.
    1. Baird PN, Schäche M, Dirani M. The GEnes in Myopia (GEM) study in understanding the aetiology of refractive errors. Progress in Retinal and eye Res. 2010; 29: 520–42.
    1. Podgorelec V, Kokol P, Stiglic B, Rozman I. Decision trees: an overview and their use in medicine. Med Syst. 2002; 26: 445–63.
    1. Cover T, Hart P. Nearest neighbor pattern classification. J IEEE Trans Inf Theor. 2006; 13: 21–27.
    1. Tibshirani R. Regression shrinkage and selection via the lasso. 1996; 58: 267–288.
    1. Jiang F, Jiang Y, Zhi H, et al. .. Artificial intelligence in healthcare: past, present and future. Stroke and Vascu Neurology. 2017; 2: 230–43.
    1. Kim SM, Kim Y, Jeong K, Jeong H, Kim JJU. Logistic LASSO regression for the diagnosis of breast cancer using clinical demographic data and the BI-RADS lexicon for ultrasonography. Ultrasonography. 2018; 37: 36.
    1. Kim SJ, Cho KJ, Oh S. Development of machine learning models for diagnosis of glaucoma. PLoS One. 2017; 12: e0177726.
    1. Tsao HY, Chan PY, Su EC. Predicting diabetic retinopathy and identifying interpretable biomedical features using machine learning algorithms. BMC Bioinformatics. 2018; 19: 283.
    1. Yang JJ, Li J, Shen R, et al. .. Exploiting ensemble learning for automatic cataract detection and grading. Comput Methods Programs Biomed. 2016; 124: 45–57.
    1. Cover T, Hart P. Nearest neighbor pattern classification. IEEE Transactions on Information Theory. 1967; 13: 21–7.
    1. Ramchoun H, Idrissi MA, Ghanou Y, Ettaouil M. Multilayer perceptron: architecture optimization and training. International Journal of Interactive Multimedia and Artificial Intelligence. 2016; 4: 26–30.
    1. Hearst MA, Dumais ST, Osuna E, Platt J, Scholkopf B. Support vector machines. IEEE Intell Sys Their Applications. 1998; 13: 18–28.
    1. Kuhn M. Building predictive models in R using the caret package. J Stat Software. 2008; 28: 1–26.
    1. Raschka S. Model evaluation, model selection, and algorithm selection in machine learning. arXiv preprint arXiv:1811.12808. 2018.
    1. Zhu W, Zeng N, Wang N. life sciences B, Maryland. Sensitivity, specificity, accuracy, associated confidence interval and ROC analysis with practical SAS implementations. NESUG proceedings: Health Care and Life Sciences, Baltimore, Maryland. 2010; 19: 67.
    1. Sonego P, Kocsor A, Pongor S. ROC analysis: applications to the classification of biological sequences and 3D structures. Briefings Bioinform. 2008; 9: 198–209.
    1. Hossin M, Sulaiman M.. A review on evaluation metrics for data classification evaluations. Int J Data Mining Knowledge Manage Proc. 2015; 5: 1.
    1. Deo RCJC. Machine learning in medicine. Deo RC. Circulation. 2015; 132: 1920–30.
    1. Hall M. PhD Thesis, Correlation-based feature selection for machine learning. 1999. New Zealand Department of Computer Science, Waikato University.
    1. Chan E, Baird PN, Vogrin S, Sundararajan V, Daniell MD, Sahebjada S. Economic impact of keratoconus using a health expenditure questionnaire: a patient perspective. Clin Exp Ophthalmol. 2019; 48: 287–300.
    1. Dienes L, Kránitz K, Juhász É, Gyenes A, Takács Á, Miháltz K, et al. .. Evaluation of intereye corneal asymmetry in patients with keratoconus. A Scheimpflug Imaging Study. 2014; 9: e108882.

Source: PubMed

Sponsorer og samarbeidspartnere

Medisinsk tilstand

Legemiddelintervensjoner

3
Abonnere