Algorithm Variability in Quantification of Epithelial Defect Size in Microbial Keratitis Images

Abstract

Purpose: To investigate the sources of measurement variability when quantifying the morphology of microbial keratitis (MK) from slit-lamp photography (SLP) images using a semiautomated, image-analysis algorithm.

Methods: Prospectively enrolled patients with MK underwent SLP to obtain images of their epithelial defects (ED). Eyes were stained with fluorescein and imaged multiple times under blue light, at low and high magnifications. A masked research assistant chose the 3 best images and annotated each 3 times to provide seed regions corresponding to ED and healthy cornea. The algorithm returned the ED area for each seeded image. Eyes without EDs and algorithm failures were excluded. Variance components were estimated with a random effects model and intraclass correlation coefficients estimated with intragrader reliability.

Results: A total of 42 eyes from 42 MK participants were photographed. After excluding poor quality images, eyes with no EDs, and algorithm failures, 34 patients with 92 images and 274 seeds were analyzed. No significant differences in the average ED area were found between seedings or high- versus low-SLP magnifications (all P > 0.5, paired t tests). Minimal measurement variability was because of image (0.9%), magnification (0.2%), or seed (0.1%). Most variability was attributable to differences in ED sizes between patients (85.2%). 13.7% of variability was unexplained. Multiple iterations of the algorithm on the same image showed good consistency (intraclass correlation coefficient = 0.98, 95% confidence interval, 0.97-0.99).

Conclusions: Image-analysis algorithms showed good reliability for measuring the ED area from SLP images. Most measurement variability was because of between-patient differences, not imaging settings or application of the algorithm by the user.

Conflict of interest statement

Conflict of interest: The authors have no proprietary or commercial interest in any of the materials discussed in this article.

Figures

Figure 1.

Flow chart for final sample of patients, images, and seeds for analysis of epithelial defect (ED) area measurement. *Two seedings were excluded due to large deviations in the algorithm identifying the true ED boundaries.

Figure 2.

Boxplots displaying the distribution of measured epithelial defect (ED) area (mm2) for A) the 22 microbial keratitis infections each with 3 images seeded 3 times, and B) the 22 infections imaged under both low and high magnification. Boxplots display the interquartile range (IQR), including the lower quartile (bottom of the box) and the upper quartile (top of the box), the median (line within the box), the mean (black dot within the box), the lower fence/whisker (smallest observation within 1.5*IQR), the upper fence/whisker (largest observation within 1.5*IQR), and outliers (asterisk; observations located outside 1.5*IQR).

Figure 3.

Scatterplots comparing measure epithelial defect area (mm2) between repeat seedings and images.