Deep convolutional neural network-based automated segmentation and classification of teeth with orthodontic brackets on cone-beam computed-tomographic images: a validation study

Summary

Objective

Tooth segmentation and classification from cone-beam computed tomography (CBCT) is a prerequisite for diagnosis and treatment planning in the majority of digital dental workflows. However, an accurate and efficient segmentation of teeth in the presence of metal artefacts still remains a challenge. Therefore, the following study aimed to validate an automated deep convolutional neural network (CNN)-based tool for the segmentation and classification of teeth with orthodontic brackets on CBCT images.

Methods

A total of 215 CBCT scans (1780 teeth) were retrospectively collected, consisting of pre- and post-operative images of the patients who underwent combined orthodontic and orthognathic surgical treatment. All the scans were acquired with NewTom CBCT device. A complete dentition with orthodontic brackets and high-quality images were included. The dataset were randomly divided into three subsets with random allocation of all 32 tooth classes: training set (140 CBCT scans-400 teeth), validation set (35 CBCT scans-100 teeth), and test set (pre-operative: 25, post-operative: 15 = 40 CBCT scans-1280 teeth). A multiclass CNN-based tool was developed and its performance was assessed for automated segmentation and classification of teeth with brackets by comparison with a ground truth.

Results

The CNN model took 13.7 ± 1.2 s for the segmentation and classification of all the teeth on a single CBCT image. Overall, the segmentation performance was excellent with a high intersection over union (IoU) of 0.99. Anterior teeth showed a significantly lower IoU (P < 0.05) compared to premolar and molar teeth. The dice similarity coefficient score of anterior (0.99 ± 0.02) and premolar teeth (0.99 ± 0.10) in the pre-operative group was comparable to the post-operative group. The classification of teeth to the correct 32 classes had a high recall rate (99.9%) and precision (99%).

Conclusions

The proposed CNN model outperformed other state-of-the-art algorithms in terms of accuracy and efficiency. It could act as a viable alternative for automatic segmentation and classification of teeth with brackets.

Clinical Significance

The proposed method could simplify the existing digital workflows of orthodontics, orthognathic surgery, restorative dentistry, and dental implantology by offering an accurate and efficient automated segmentation approach to clinicians, hence further enhancing the treatment predictability and outcomes.

Introduction

Tooth segmentation on cone-beam computed tomography (CBCT) images is a fundamental task in the majority of computer-aided dental workflows. It provides a high-resolution three-dimensional (3D) volumetric data of a tooth and are most commonly employed for guiding diagnosis, treatment planning phase and/or follow-up evaluation of orthodontic therapy, orthognathic surgery, dental implantology, guided-endodontics, restorative dentistry, and tooth autotransplantation (1).

Currently, manual segmentation acts as the gold standard for segmenting teeth, which is a time-consuming and tedious task as the operator has to manually delineate the boundaries of a tooth and check for any deformity in all the slices of the CBCT image. To overcome these limitations, alternative solutions have been deployed mainly in the form of threshold-based semi-automatic commercial or open-source software programs (2). Although these tools offer a faster approach compared to their manual counterpart, the development and optimization of such software have been primarily based on medical CT images, which are superior in segmentation accuracy as compared to CBCT (3). Furthermore, segmentation on CBCT images is below par owing to the presence of beam-hardening artefacts, heterogeneous intensity distribution, lacking Hounsfield units, low-contrast resolution, and unclear boundaries between inter-tooth proximity and the tooth root and alveolar bone (4,5). The error introduced by an inaccurate segmentation could negatively influence the later steps of the digital workflows and the final expected outcome.

Recently, the application of artificial intelligence (AI) in the form of deep convolutional neural networks (CNN) have been extensively employed for developing automated tools to achieve an accurate and efficient tooth segmentation and classification (6,7). These AI approaches have the ability to learn nonlinear spatial characteristics in a scan and have overcome the limitations associated with both manual and semi-automatic approaches (8–11).

Various studies have assessed different CNN models and found their performance to be higher compared to other conventional and state-of-the-art approaches for classifying and segmenting pristine teeth and those with high-density restorative materials (12–22). However, the main challenge that still persists with both conventional and automated CNN-based segmentation tools is their inability to segment teeth with metal artefacts originating from orthodontic brackets. The integration of automated tools allowing accurate segmentation and isolation of teeth from brackets could further optimize the efficacy of current digital dental workflows and decrease a clinician's workload with the possibility of improving patient care.

To the best of our knowledge, no study has previously investigated the application of CNN models for the segmentation of teeth with brackets on CBCT images. Therefore, this study aimed to validate an automated multiclass deep CNN-based tool for an accurate and efficient segmentation and classification of teeth with brackets on CBCT images.

Materials and methods

This study was conducted following the Helsinki World Medical Association Declaration on Medical Research. Ethical approval was obtained from the Ethical Review Board (reference number: S57587). Informed consent was not required as patient-specific information was anonymized.

Dataset

A total of 215 CBCT scans (1780 teeth: anterior = 646, premolars = 486, molars = 648) were retrospectively collected from LORTHOG database of the University Hospital, which consisted of pre- and post-operative images of the patients who underwent combined orthodontic and orthognathic surgical treatment for the correction of dentoskeletal deformities. All scans were acquired with NewTom VGi evo (NewTom, Verona, Italy) CBCT device with the following acquisition parameters: 110 kV, voxel size: 0.2 × 0.2 × 0.2 mm³, FOV: 122.8 × 122.8 × 80.2 mm³/103.2 × 103.2 × 100.8 mm³/244.8 × 244.8 × 188.7 mm³. The inclusion criteria were the presence of both maxillary and mandibular complete dentition (anterior, premolars, and molars) with orthodontic brackets and high-quality images. Patients with partial edentulous jaws, dental implants, and motion artefacts were excluded.

The complete dataset was randomly divided into three subsets with random allocation of all 32 tooth classes as follows:

• - Training set (140 CBCT scans—400 teeth), to train the CNN model

• - Validation set (35 CBCT scans—100 teeth), to assess the model performance based on trained set and hyperparameter optimization.

• - Test set (40 CBCT scans—1280 teeth), to assess the performance of CNN-based automated segmentation compared to the ground truth. This set was further divided into two subgroups, pre-operative (25 CBCT scans) and post-operative groups (15 CBCT scans), both of which consisted of all tooth groups. The difference between pre- and post-operative images was the inclusion of artefacts generated from osteotomy lines, fixation plates, and screws in the post-operative images as such to assess the robustness of the algorithm.

Both training and test sets were prepared by two experts, where one expert segmented and labelled all teeth, followed by verification by another expert to ensure quality control. The segmentation procedure has been adopted from similar work (6), where the training set was developed by a previously validated method (23). The operator manually trimmed the CBCT image around each tooth, followed by automated 3D contouring and segmentation of the individual teeth in axial, sagittal, and coronal views while carefully excluding the brackets. Manual refinement of the contours was performed when needed, however, the contouring protocol described by EzEldeen et al. (23) overcame the inaccurate estimation of the tooth contour around bracket-tooth contact region. Furthermore, a second expert validated and corrected the segmentation.

The test set ground truth was prepared with a hybrid approach using an online cloud-based AI system, known as ‘Virtual Patient Creator’ (Relu BV, Leuven, Belgium) (24). First, the CBCT images were imported in Digital Imaging and Communication in Medicine (DICOM) format, and the platform automatically generated the initial segmentation of individual maxillary and mandibular teeth. Thereafter, the discrepancies in the segmentation were refined by an expert for generating a corrected AI driven (C-AI) segmentation.

CNN framework

The CNN framework for the automated segmentation and classification of pristine teeth without brackets or any type of artefacts has been previously described and validated (6). The same pipeline was applied which is configured based on multiple U-Net models (25). All of these function at different spatial resolutions and each model focuses on a different subproblem for creating a high-resolution multiclass tooth segmentation and classification to 32 tooth classes. These models were implemented in Pytorch and optimized using Adam optimization (26) to decrease the learning rate. The loss function in the training procedure was a binary cross-entropy loss for the first and third models and cross-entropy loss for the second model (6). Additionally, an early stoppage to the validation set was applied to prevent overfitting. During the training phase, random spatial augmentations were performed which included rotation, scaling, and elastic deformation.

Evaluation metrics

The evaluation metrics for comparing automated and C-AI ground truth segmentation consisted of intersection over union (IoU), dice similarity coefficient (DSC), precision, recall, accuracy, 95% Hausdorff Distance (HD), and segmentation time. For automated segmentation, the time was recorded starting from the DICOM upload till the generation of multiclass segmentation and classification of all the teeth on a CBCT image. Additionally, the time for C-AI segmentation was assessed by summing up the automated segmentation time and correction time required after the expert carefully examined and identified the slices that required corrections per tooth group (anterior, premolar, and molar). However, the time required for import, export, and inspection of the data prior to corrections was not included. The classification of teeth was evaluated based on accuracy, precision, and recall rate. The specifications of the computing device used for assessing the segmentation time have been listed in Table 4.

Data were analyzed using MedCalc statistical software (version 16.2.0, Ostend, Belgium). Mean and standard deviation values were calculated for assessing the network’s performance for complete dataset segmentation, individual segmentation of tooth subgroups (anterior, premolar, and molar), and classification. The performance of tooth segmentation in pre- and post-operative subgroups for each tooth group was calculated using the Kruskal–Wallis test with Bonferroni correction. A P-value of <0.05 was considered statistically significant.

Results

The mean segmentation and classification time of all the teeth on a single CBCT image with the CNN model was 13.7 ± 1.2, which was three times faster than the hybrid C-AI approach (43.56 ± 20.31 s).

Table 1 demonstrates the overall performance metrics for segmentation which were calculated by comparing CNN model with the C-AI ground truth. The CNN model showed a high IoU, DSC, precision, and recall score of 0.99, indicating towards a near-to-perfect segmentation. In addition, the overlap between the automated segmentation and ground truth was excellent, which was confirmed by the 95% HD value of 0.12 ± 0.15 mm. Figure 1 illustrates an example of a case with an almost perfect overlap between automated segmentation and ground truth.

Table 2 describes the segmentation metrics of pre- and post-operative subgroups based on different tooth types. All the performance metrics in both subgroups showed a high score ranging between 0.97 and 0.99. Figure 2 illustrates some cases requiring minor correction. In addition, Figure 3 displays the difference and the impact of AI training on teeth with brackets segmentation.

According to Table 3, anterior teeth showed a significantly lower IoU (P < 0.05) compared to premolar and molar teeth in both pre- and post-operative groups. The classification of teeth to the correct 32 classes showed an almost perfect performance with a high accuracy (100%), recall rate (99.9%), and precision (99%).

Discussion

The present study was conducted to validate an innovative CNN-based tool for multiclass segmentation of teeth with brackets and classification on CBCT images. Our findings showed that the tool was time-efficient and highly accurate in the presence of brackets.

Previous studies have shown that manual segmentation is a time-consuming process, as it might take approximately 3.5–7 h for segmenting all the teeth in a single scan depending on the observer’s experience (6,7). Similarly, semi-automatic approaches also suffer from the limitations of time consumption, where a single or double-rooted individual tooth could take up to 6.6 min to correctly segment (7). Another study reported a time of 29.8 s with a CNN model for the segmentation of teeth with dental fillings (27). In contrast, the presented CNN-based model took 13.7 ± 1.2 s for simultaneous segmentation and classification of all the teeth in a scan independent of the number of roots. Thereby, implying that it could act as a more efficient alternative in dental workflows where either manual or semi-automatic segmentation approaches still remains a clinical standard. Furthermore, the ability of the model to segment teeth with brackets magnifies its clinical potential for dental applications such as orthodontic analysis, surgical guide and/or wafer designing in orthognathic surgery, dental implantology and tooth auto-transplantation, and follow-up assessment of tooth eruption and root resorption. As segmentation of teeth with brackets is time-consuming with manual approaches and thresholding-based semi-automatic techniques fail to optimally separate the brackets from teeth due to the presence of the same range of thresholding value to that of teeth, thereby, requiring a laborious post-processing phase for manual correction. Hence, the integration of this automated tool in the digital dental workflow could lower the possibility of error associated with the nonautomated steps of the workflows which could further improve the standard of patient care.

Based on the accurate preparation of the training dataset, the CNN model was able to segment teeth with brackets with higher performance (DSC: 0.99 ± 0.06) compared to other previously reported state-of-the-art algorithms which focussed on tooth segmentation without brackets (6,28). Lee et al. applied a multiphase strategy to train a U-Net-based architecture with a DSC score ranging between 0.91 and 0.92 (20). Cui et al. presented a two-stage network consisting of a tooth edge map extraction network and a region proposal network with a DSC of 0.93 (19). Shaheen et al. assessed the performance of a CNN-based model for tooth segmentation with a DSC score of 0.90 (6). In addition, Wang et al. used a mixed-scale dense CNN model and found a DSC of 0.95 (28). The lower performance of the previous studies could be associated with a smaller training set which was not the case in the present study. It is generally a common knowledge that a large labelled training dataset is essential to avoid overfitting of a model, enhance its learning and optimization, and to effectively capture the inherent data distributions.

However, further studies are required to confirm the cause of this minimal error to avoid the chances of a higher accumulative error at the later steps of image processing in digital workflows.

The findings also showed that the surface deviation between the automated segmentation and C-AI ground truth was 0.12 ± 0.15 mm. In comparison, Wang et al. reported a slightly higher value of 0.20 ± 0.06 mm for segmenting teeth without brackets (28). Similarly, Shaheen et al. also observed a surface deviation of 0.56 ± 0.38 mm for segmenting teeth without the inclusion of artefacts from implants or brackets (6).

It should be noted that in the majority of digital dental workflows (implantology, orthodontics, and orthognathic surgery) three main individual steps exist, which include segmentation of CBCT dataset, segmentation of intraoral scanned dataset, and registration (fusion) of both datasets. The first and the most vital step is the segmentation of teeth from CBCT datasets which is then used for registration or fusion with the intraoral scanned datasets (mainly using surface-based fusion or iterative closest point algorithm) (29). If the segmentation of CBCT data are suboptimal then it would impact the accuracy of fusion step. Therefore, in the present study we proposed an accurate, reliable, and time-efficient automated segmentation approach of the CBCT data which could replace the conventional semi-automatic methods. Moreover, having the possibility of lowering the accumulative error of the digital workflows. The next step for future research would be to propose and investigate the accuracy of automated intraoral scanned data segmentation and fusion/registration between CBCT and intraoral scanned data which at the moment is outside the scope of the current study. Furthermore, from a technical point of view, automated fusion is only possible after achieving individual automated segmentation of the CBCT (present aim) and intraoral scanned data (30–32). The strength of the following study was the inclusion of teeth with brackets, which enhances its clinical applicability. However, the training of the CNN model was limited to CBCT dataset acquired from a single device with different acquisition settings. Hence, its generalizability and performance with other devices are still questionable. Future studies are planned to improve its performance and robustness by training with data from multiple devices and cases with dental implants, high-density restorative materials, and partial edentulism. Furthermore, the accuracy of automated versus C-AI segmentation needs to be qualitatively and quantitatively investigated to appropriately define the regions where maximal corrections are required, for facilitating improvements in the performance of the network.

Conclusions

The proposed multiclass CNN model showed excellent performance with high accuracy and efficiency for the segmentation and classification of teeth with brackets. It could act as a viable alternative to existing segmentation approaches. Its integration into various digital workflows might increase the efficacy of patient-specific treatment planning, while ensuring predictable outcomes.

Conflict of Interest

We declare that the authors quoted here have no conflicts of interest.

Author Contributions

KA: Conceptualization, Methodology, Data Analysis, Writing original draft, review & editing. RJ: Conceptualization, Methodology, Writing review & editing. AS: Conceptualization, Methodology, Writing review & editing. AV: Conceptualization, Methodology, Writing review & editing. HW: Conceptualization, Methodology, Writing review & editing. SS: Conceptualization, Writing review & editing. ES: Conceptualization, Data extraction, Writing original draft, review & editing.

Data Availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References