Novel Image-Based Analysis for Reduction of Clinician-Dependent Variability in Measurement of the Corneal Ulcer Size

Abstract

Purpose: To assess variability in corneal ulcer measurements between ophthalmologists and reduce clinician-dependent variability using semiautomated segmentation of the ulcer from photographs.

Methods: Three ophthalmologists measured 50 patients' eyes for epithelial defects (EDs) and the stromal infiltrate (SI) size using slit-lamp (SL) calipers. SL photographs were obtained. An algorithm was developed for semiautomatic segmenting of the ED and SI in the photographs. Semiautomatic segmentation was repeated 3 times by different users (2 ophthalmologists and 1 trainee). Clinically significant variability was assessed with intraclass correlation coefficients (ICCs) and the percentage of pairwise measurements differing by ≥0.5 mm. Semiautomatic segmentation measurements were compared with manual delineation of the image by a corneal specialist (gold standard) using Dice similarity coefficients.

Results: Ophthalmologists' reliability in measurements by SL calipers had an ICC from 0.84 to 0.88 between examiners. Measurements by semiautomatic segmentation had an ICC from 0.96 to 0.98. SL measures of ulcers by clinical versus semiautomatic segmentation measures differed by ≥0.5 mm in 24% to 38% versus 8% to 28% (ED height); 30% to 52% versus 12% to 34% (ED width); 26% to 38% versus 10% to 32% (SI height); and 38% to 58% versus 14% to 34% (SI width), respectively. Average Dice similarity coefficients between manual and repeated semiautomatic segmentation ranged from 0.83 to 0.86 for the ED and 0.78 to 0.83 for the SI.

Conclusions: Variability exists when measuring corneal ulcers, even among ophthalmologists. Photography and computerized methods for quantifying the ulcer size could reduce variability while remaining accurate and impact quantitative measurement endpoints.

Conflict of interest statement

Conflict of interest: The authors have no proprietary or commercial interest in any of the materials discussed in this article.

Figures

Figure 1

Study design: 50 patients with corneal ulcers were recruited. Three ophthalmologists measured the size of ED and SI at the slit-lamp. A cornea fellow took slit-lamp photographs of each ulcer. The image was then analyzed by quantitative corneal monitoring (QCM) software for the size of the ED and SI by manual and automated segmentation techniques. Measurements were analyzed for variability.

Figure 2

QCM analysis pipeline: given a digital photograph of the corneal ulcer (A), the user draws a line to measure the horizontal white to white distance (WTW), and seed regions to denote the foreground (stromal infiltrate or epithelial defect, depicted in blue) and the background (clear cornea, depicted in red). A random forest tissue classifier generates a probability map of the foreground and background image (C). The probability map is used as a speed image for active contour evolution (D – E) and segmentation (F). The maximum vertical and horizontal distance is measured and recorded as the height and width, respectively.

Figure 3

Dice overlap coefficient between manual and semi-automated segmentations. Each corneal ulcer photograph (A) was both manually segmented (B, yellow) and semi-automatically segmented (C, green). The fractional spatial overlap between the manual and computerized segmentations (D) is the Dice overlap coefficient (equals 0.78 in this example).

Figure 4

Scatterplots displaying agreement in measurement of ED and SI dimensions, height and width, between a) pairs of examiners, b) manual and semi-automated segmentation methods, and c) repeated semi-automated segmentation. Note, all comparisons between pairs of examiners (E1 vs E2, E1 vs E3, E2 vs E3), manual and semi-automated segmentation (M vs A1, M vs A2, M vs A3), and repeated semi-automated methods (A1 vs A2, A1 vs A3, A2 vs A3) are represented with the same symbol on each corresponding scatterplot for ease of viewing. Reference lines for no difference (black dashed line) between two examiners and ± 0.5 mm (dark gray, solid lines) and ± 1.0 mm (light gray, solid lines) differences are displayed.

Figure 5

Histograms displaying differences in measurement of ED and SI dimensions, height and width, between a) pairs of examiners, b) manual and semi-automated segmentation methods, and c) repeated semi-automated segmentation. E1-E3=Examiner1-Examiner3; M=Manual Segmentation; A1-A3=Semi-Automated Segmentation1-Semi-Automated Segmentation3; v=versus.

Figure 6

Forest plot displaying intraclass correlation coefficients (ICC) for reliability of measurements between examiners and between repeated semi-automated segmentations from photos. ED=epithelial defect; SI=stromal infiltrate.