AI Gum Health Evaluation With Smartphone

Automatic Multiple Level Gum Disease Detection Based on Deep Neural Network: Algorithm and System

Background The most common dental diseases are tooth decay (caries) and gum disease (gingivitis and periodontitis). Obviously, these diseases are caused by dental plaque (bacterial biofilm). Although most patients brush their teeth every day, they cannot keep all their teeth clean. Areas in the mouth that are difficult to access, such as crowded areas, posterior teeth or interdental areas, are usually affected (site-specific). After a thorough professional tooth cleaning, dental plaque will begin to accumulate on the tooth surface near the gum edge within a few days. Clinical studies indicating that regular disruption to the plaque is needed and can prevent and arrest gum disease. However, dental diseases may take years to develop, the patient usually does not have any pain symptoms unless the disease has progressed to the advanced stage. A significant amount of resources and clinical time have been used to motivate and instruct patients to keep their mouth clean and yet the results are not satisfactory. It is desirable to adopt an automated technique for monitoring oral health daily so we can seek treatment when it is needed.

Patients' response to plaque accumulated at the gum margin is by inflammation which brings more blood cells to the site to fight against the bacterial invasion. Inflammation of gum is manifested as an increase in redness (color), an increase in volume (oedema), and loss of surface characteristics (stippling; gum fibre attachment). These affected areas can be identified by visual inspection with the dentist during the consultation or using intraoral photography. The objective of this research is to apply deep neural network technology to detect gum inflammation from intraoral photos. As the target inflammation site is at gum margin with varied shape and size, semantic segmentation at pixel level is needed.

In this research, we are planning to have an extensive study of deep neural network (DNN) approach for the automatic multiple level gum disease detection. Standardized intraoral photography will be collected for 1200 cases and will be labelled by several dentists as "diseased" (inflammation), "healthy" or "questionable". Only gum area in which the dentists have same rating will be used to train/validate the system. Using the successfully developed system, one can use his/her mobile device to monitor their gum health when needed. They may be able to prevent the two main oral diseases (tooth decay and gum diseases) with minimal additional cost. It will be an important contribution to the promotion of public dental care.

Aim of study This study aims to train and validate the computer to automatically monitor gum inflammation using standardized intraoral photos and selfie by smartphone.

1. to collect 1200 standard intraoral photographs and randomly cropped into training and validation sets.
2. to develop ground truth gingivitis label images into four health status levels (healthy, questionable healthy, questionable diseased and diseased) and verified by dental specialists.
3. to develop intelligent system for automatically detect inflamed disease sites with four health status levels.
4. to develop and standardize the image acquisition protocol for the detection with mobile devices.

Hypothesis A diagnostic tool should be able to diagnose true disease and true health which described as sensitivity (positive when true disease) and specificity (negative when true health). The primary outcome will be the area under the receiver operating characteristic (ROC) curve (AUC). The hypothesis of this study is the trained gingival detection system is able to detect the changes of gum inflammation with high sensitivity and specificity.

Study Overview

• Status: Recruiting
• Conditions: Gingivitis
• Intervention / Treatment: Diagnostic test: Standardized intraoral photography

Detailed Description

I. Introduction The most prevalent dental diseases are tooth decay (caries) and gum diseases (gingivitis and periodontitis). It is evidence that these diseases are caused by dental plaque (bacterial biofilm) [1]-[3]. Although most patients brush their teeth every day, they cannot keep all their teeth clean. Areas in the mouth that are difficult to access, such as crowded areas, posterior teeth or interdental areas, are usually affected (site-specific) [4]. After thorough professional tooth cleaning, plaque will begin to accumulate on the surface of the teeth near the edge of the gum within few days. Clinical studies indicating that regular disruption to plaque are needed and can prevent and arrest gum disease [5]. However, dental diseases may take years to develop, the patient usually does not have any pain symptoms [6] unless the disease has progressed to the advanced stage. Significant amount of resources and clinical time have been used to motivate and instruct patients to keep their mouth clean and yet the results are not satisfactory. It is desirable to adopt an automated technique for monitoring oral health daily so we can seek for treatment when it is needed.

Patients' response to plaque accumulated at the gum margin is by inflammation which brings more blood cells to the site to fight against the bacterial invasion [7]. Inflammation of gum is manifested as increased in redness (colour), increase in volume (oedema), and loss of surface characteristics (stippling; gum fibre attachment) [8]. These diseased sites can be identified by visual examination of dentists. Moreover, these inflammatory changes of gum can also be recognized by intraoral photography. The objective of this research is to apply artificial intelligent (AI) techniques to detect gum inflammation from intraoral photographs. As the target inflammation site is at gum margin with varied shape and size, semantic segmentation at pixel level is needed. For this research, we have done some preliminary research works. DeepLabv3+ [9] encoder-decoder network with a lightweight backbone MobileNetV2 [10] was adopted to perform pixelwise semantic segmentation of the gingival inflammation from the intraoral photographs. The photographs are indexed by a dental specialist with more than 15 years clinical experience to obtain the index category images for the network training.

II. Work done by others The first attempt to automated segmentation of gingival diseases from intraoral images with deep learning approach is proposed in [11]. It adopts an autoencoder network architecture with deep convolution neural network. The dataset used comprises 405 color-augmented intraoral biomarker images from 150 individuals. Areas of gingival inflammation were labelled by dental professional and the trained network can predict the inflammation with area under the receiver operating characteristic curve (AUC) 0.746. The precision and recall values are 0.347 and 0.621, respectively. The network was trained with the labeling on diseased gum. Some calculus on teeth was also predicted as diseased gum as its yellowish color is technically close to that of diseased gum. Moreover, some parts of uninterested gingival area were also predicted as diseased gum. Overall segmentation is not satisfactory.

III. Our preliminary works The intraoral photographs of patients from the Faculty of Dentistry, The University of Hong Kong (HKU), which underwent periodontic treatment, were collected for the preliminary study. The study was approved by the Institutional review board of HKU (UW20-230). Total 110 standard intraoral photographs with different resolutions were collected. They are manually cropped into different smaller images, and the target labels occupy the largest possible image, which is very beneficial for training. The size of the cropped image is unified to 512×512. The completed dataset is divided into two sets, respectively 337 images for training, and 110 images for validation. Considering that there are multiple images corresponding to one patient, so when dividing the dataset, the image of the same patient will not appear in the two divided datasets. They are labeled into four health status levels (healthy, questionable healthy, questionable diseased and diseased) and verified by a dental specialist with more than 15 years clinical experience. The proposed semantic segmentation architecture is based on the DeepLabv3+ network with Xception and MobileNetV2 as the backbone. Experimental results show the effectiveness of the proposed system, which shows possible application on dental self check-up using mobile app particularly during the disease pandemic where visit to dentists are difficult or even impossible. The proposed network model can predict the contour of the interested gingival area. Experiment results show that the proposed segmentation model can accurately divide most of the gum inflammation area into four categories. The mIoU is 0.3514. It is believed that by expanding the dataset and optimizing the network structure, the performance can be further improved. The research work about the preliminary was published in [16].

IV. Key issues and Research gap

A. There is no multiple level site-specific semantic segmentation neural network model for automatic gingival inflammation detection from intraoral photograph.

B. There is no well-labeled training dataset for the application of automatic gingival inflammation detection from intraoral photograph.

Reference:

1. J.K. Clarke, "On the bacterial factor in the aetiology of dental caries," British journal of experimental pathology, vol. 5, no. 3, pp. 141-147, 1924.
2. S.S. Socransky, A.D. Haffajee, "The bacterial etiology of destructive periodontal disease: current concepts," Journal of periodontology, vol. 63, pp. 322-331, 1992.
3. W.F. Liljemark, C. Bloomquist, "Human oral microbial ecology and dental caries and periodontal diseases," Critical Reviews in Oral Biology & Medicine, vol. 7, no. 2, pp. 180-198, 1996.
4. H.J. Breen, N.W. Johnson, P.A. Rogers, "Site-specific attachment level change detected by physical probing in untreated chronic adult periodontitis: review of studies 1982超1997," Journal of periodontology, vol. 70, no. 3, pp. 312-328, 1999.
5. H. Löe, "Oral hygiene in the prevention of caries and periodontal disease," International dental journal, vol. 50, no. 3, pp. 129-139, 2000.
6. L. Croxson, "Periodontal awareness: the key to periodontal health," International dental journal, vol. 43, (2 Suppl 1) pp. 167-177, 1993.
7. R. Genco, J. Slots, "Host Responses Host Responses in Periodontal Diseases," Journal of dental research, vol. 63, no. 3, pp. 441-451, 1984.
8. G.C. Armitage, "Clinical evaluation of periodontal diseases," Periodontology 2000, vol. 7, no. 1, pp. 39-53, 1995.
9. L.C. Chen, Y. Zhu, G. Papandreou, F. Schroff and H. Adam, "Encoder-decoder with atrous separable convolution for semantic image segmentation," in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 801-818.
10. M. Sandler, A. Howard, M. Zhu, A. Zhmoginov and L. Chen, "MobileNetV2: Inverted Residuals and Linear Bottlenecks," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 4510-4520.
11. A. Rana, G. Yauney, L.C. Wong, O. Gupta, A. Muftu and P. Shah, "Automated segmentation of gingival diseases from oral images," in 2017 IEEE Healthcare Innovations and Point of Care Technologies (HI-POCT), Bethesda, MD, USA, 2017, pp. 144-147.
12. O. Ronneberger, P. Fischer, T. Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation," in 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2015, pp. 234-241.
13. B. Zoph, V. Vasudevan, J. Shlens and Q.V. Le, "Learning Transferable Architectures for Scalable Image Recognition," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 8697-8710.
14. F. Chollet, "Xception: Deep Learning with Depthwise Separable Convolutions," in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 1800-1807.
15. M.X. Tan and Q.V. Le, "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks," in Proceedings of the 36th International Conference on Machine Learning, PMLR, 2019, vol. 97, pp. 6105-6114.
16. G.-H. Li, T.-C. Hsung, B. W.-K. Ling, W. Y.-H. Lam, G. Pelekos, and C. McGrath, "Automatic Site-Specific multiple level gum disease detection based on deep neural network," in 2021 15th International Symposium on Medical Information and Communication Technology (ISMICT) (ISMICT2021), Xiamen, China, Apr. 2021.

• Study Type: Observational
• Enrollment (Anticipated): 1200

Contacts and Locations

• Study Contact: Name: Tai Chiu Hsung, PhD; Phone Number: (852)28590270; Email: tchsung@hku.hk
• Study Contact Backup: Name: Yu Hang Lam, MDS; Phone Number: (852)28590306; Email: retlaw@hku.hk

Study Locations

• Hong Kong
  • Hong Kong, Hong Kong
    • Recruiting
    • Faculty of Dentistry, The University of Hong Kong
    • Contact: Yu Hang Lam, MDS; Phone Number: (852) 28590306; Email: retlaw@hku.hk

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

• Sampling Method: Non-Probability Sample

Study Population

The target study population is the adult subjects attending The Prince Philip Dental Hospital who are able to give informed consent.

Description

Inclusion Criteria:

1. Adult subjects attending The Prince Philip Dental Hospital (PPDH) whoare able to give informed consent.
2. Subjects who are diagnosed to have gingivitis only and have 24 or more teeth.
3. Subjects who are otherwise medically healthy.
4. Subjects who can attend multiple dental visits.

Exclusion Criteria:

1. Subjects who are in acute dental infection or in pain.
2. Subjects who have oral mucosal diseases that preclude retraction of soft tissues for photos.
3. Subjects who are in a fixed appliance for orthodontic treatment.
4. Subjects who are pregnant, or medically unfit for periodontal charting or require antibiotic coverage (e.g. risk of infective endocarditis)

Study Plan

How is the study designed?

Design Details

• Observational Models: Case-Only
• Time Perspectives: Prospective

Number of groups / cohorts

1

Cohorts and Interventions

Group / Cohort

Intervention / Treatment

Gingivitis samples; Inclusion criteria: Adult subjects attending PPDH can give informed consent.; Subjects who are diagnosed to have gingivitis only and have 24 or more teeth.; Subjects who are otherwise medically healthy.; Subjects who can attend multiple dental visits. Exclusion criteria; Subjects who are in acute dental infection or in pain.; Subjects who have oral mucosal diseases that preclude retraction of soft tissues for photos.; Subjects who are in the fixed appliance for orthodontic treatment.; Subjects who are pregnant, or medically unfit for periodontal charting or require antibiotic coverage (e.g. risk of infective endocarditis)

Diagnostic test: Standardized intraoral photography; Mouth photos will be taken with a plastic retractor to retract the subject's cheek and lips. This is a standard clinical procedure that is non-invasive and does not cause any harm or adverse effect on subjects.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Database of standard intraoral photographs with ground truth gingivitis label	A database of 1200 standard intraoral photographs with ground truth gingivitis label images into four health status levels (healthy, questionable healthy, questionable diseased and diseased) which are developed and verified by dental specialists.; An intelligent system for automatically detect inflamed disease sites with four health status levels.	1/1/2023-31/12/2025

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Patient reported outcome on the use of smartphone selfies	Patient-reported outcome on the use of smartphone selfies would be recorded in visual analogue scale (VAS)	1/1/2023-31/12/2025

Collaborators and Investigators

• Sponsor: The University of Hong Kong
• Collaborators: Chu Hai College of Higher Education

Study record dates

Study Major Dates

• Study Start (Actual): January 1, 2022
• Primary Completion (Anticipated): December 31, 2025
• Study Completion (Anticipated): December 31, 2025

Study Registration Dates

• First Submitted: January 5, 2023
• First Submitted That Met QC Criteria: January 5, 2023
• First Posted (Estimate): January 16, 2023

Study Record Updates

• Last Update Posted (Estimate): January 16, 2023
• Last Update Submitted That Met QC Criteria: January 5, 2023
• Last Verified: January 1, 2023

More Information

Terms related to this study

• Keywords: machine learning; artificial intelligence; inflammation; mobile device; photography; Gingivitis detection; Semantic segmentation; deep neural network
• Additional Relevant MeSH Terms: Infections; Stomatognathic Diseases; Periodontal Diseases; Mouth Diseases; Gingival Diseases; Gingivitis
• Other Study ID Numbers: UW21-447

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: Undecided
• IPD Plan Description: Individual participant data will not be shared until the project is complete. We will review the plan when complete.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No