Open-Source Automatic Segmentation of Ocular Structures and Biomarkers of Microbial Keratitis on Slit-Lamp Photography Images Using Deep Learning

Abstract

We propose a fully-automatic deep learning-based algorithm for segmentation of ocular structures and microbial keratitis (MK) biomarkers on slit-lamp photography (SLP) images. The dataset consisted of SLP images from 133 eyes with manual annotations by a physician, P1. A modified region-based convolutional neural network, SLIT-Net, was developed and trained using P1's annotations to identify and segment four pathological regions of interest (ROIs) on diffuse white light images (stromal infiltrate (SI), hypopyon, white blood cell (WBC) border, corneal edema border), one pathological ROI on diffuse blue light images (epithelial defect (ED)), and two non-pathological ROIs on all images (corneal limbus, light reflexes). To assess inter-reader variability, 75 eyes were manually annotated for pathological ROIs by a second physician, P2. Performance was evaluated using the Dice similarity coefficient (DSC) and Hausdorff distance (HD). Using seven-fold cross-validation, the DSC of the algorithm (as compared to P1) for all ROIs was good (range: 0.62-0.95) on all 133 eyes. For the subset of 75 eyes with manual annotations by P2, the DSC for pathological ROIs ranged from 0.69-0.85 (SLIT-Net) vs. 0.37-0.92 (P2). DSCs for SLIT-Net were not significantly different than P2 for segmenting hypopyons (p > 0.05) and higher than P2 for WBCs (p < 0.001) and edema (p < 0.001). DSCs were higher for P2 for segmenting SIs (p < 0.001) and EDs (p < 0.001). HDs were lower for P2 for segmenting SIs (p = 0.005) and EDs (p < 0.001) and not significantly different for hypopyons (p > 0.05), WBCs (p > 0.05), and edema (p > 0.05). This prototype fully-automatic algorithm to segment MK biomarkers on SLP images performed to expectations on an exploratory dataset and holds promise for quantification of corneal physiology and pathology.

Figures

Figure 1:

Examples of SLP images taken with diffuse white light illumination and diffuse blue light illumination after topical fluorescein staining. Pathological and non-pathological ROIs were manually annotated by a physician (P1).

Figure 2:

SLIT-Net architecture.

Figure 3:

Examples of manual annotations by P1 and fully-automatic segmentations by the baseline U-Net, nnU-Net, Mask R-CNN, and the proposed SLIT-Net. SLIT-Net’s segmentations were closest to that of P1 and outperformed the alternative methods.

Figure 4:

Examples of manual annotations by P1 and P2 and fully-automatic segmentations by SLIT-Net on diffuse white light and diffuse blue light images. There was good agreement among the three methods.

Figure 5:

Examples of the difficulty with identification and segmentation of WBCs and edema on diffuse white light images. Top: P1 identified the region surrounding the SI as WBCs, while P2 identified the same region as edema instead. SLIT-Net’s segmentations were closer to that of P1. Middle: P1 identified the region surrounding the SI as only WBCs, while P2 identified an additional region of edema. P1 also identified a smaller SI compared to P2. SLIT-Net identified a SI that was closer to P2 but did not identify any edema. Bottom: P1 identified both WBCs and edema surrounding the SI, while P2 and SLIT-Net identified only WBCs.

Figure 6:

Examples of the difficulty with segmentation of EDs on diffuse blue light images. SLIT-Net’s segmentations were closer to that of P2 in these cases.

Figure 7:

Examples of poor segmentation by SLIT-Net due to the heterogeneity of MK phenotype or poor image quality. In some cases, there was also low agreement between P1 and P2.