Evaluation of the performance of two software artificial intelligence-based by means of the measurements according to Mcnamara’s Analysis in lateral cephalometric radiographs

Authors

DOI:

https://doi.org/10.33448/rsd-v11i14.35820

Keywords:

Artificial intelligence; Orthodontics; Machine learning; Radiology; Diagnosis.

Abstract

The aim of this study was to compare the performance of two software programs with AI in lateral cephalometric teleradiography by assessing the reproducibility and reliability of the linear and angular measurements of McNamara's analysis. Thirty cephalometric teleradiographs were marked using the digital method by the examiner in Radiocef (RadioMemory). Subsequently, the sample was marked using the CEFBOT (RadioMemory) and WebCephTM (AssembleCircle) software AI to evaluate the reproducibility and reliability of the examiner and the software. To calibrate the examiner and evaluate the reliability of the examiner, CEFBOT, and WebCephTM markings, the Intraclass Correlation Coefficient (ICC) was used, as well as the ANOVA test and Tukey's post-test evaluated the reproducibility of the software, using the cephalometric landmarks that comprise McNamara's analysis. The mean ICC of the examiner, CEFBOT and WebCeph were 0.960, 0.940 and 0.954, respectively, indicating almost perfect agreement. When comparing CEFBOT with examiner, statistical difference (p<0.01) was observed only in the perpendicular A-N measurement. As for WebCephTM, when comparing with the examiner there was a significant difference between factors two to six and ten. And compared to CEFBOT, there was divergence in the same factors plus factor eleven. In addition, WebCephTM did not identify the measurements Nfa-Nfp and Bfa-Bfp. CEFBOT showed reproducibility and reliability in identifying the cephalometric landmarks determined by McNamara's analysis but required human supervision. WebCeph showed almost perfect agreement in the markings, but six measurements were different from the examiner and two were not performed by the application.

References

Albarakati, S., Kula, K., & Ghoneima (2012). A. The reliability and reproducibility of cephalometric measurements: a comparison of conventional and digital methods. Dentomaxillofacial Radiology, 41 (1), 11–17.

Bissoli, C. F., Takeshita, W.M., Castilho, J.C.M., Médici-Filho, E.M (2007). Digitalização de imagens em radiologia: uma nova visão de futuro. Revista Odonto, 30 (15), 34-39.

Borba, A. M.; Haupt, D.; Almeida Romualdo, L. T. De; Silva, A. L. F. Da; Graça Naclério-Homem, M. Da; Miloro, M (2016). How Many Oral and Maxillofacial Surgeons Does It Take to Perform Virtual Orthognathic Surgical Planning? Journal of Oral and Maxillofacial Surgery 74 (9), 1807–1826.

Chen, S.-K., Chen, Y.-J., Yao, C.-C. J., Chang, H.-F (2004). Enhanced Speed and Precision of Measurement in a Computer-Assisted Digital Cephalometric Analysis System. Angle Orthodontist,74 (4), 1-11.

Chen, Y., Stanley, K., & Att, W (2020). Artificial intelligence in dentistry: current applications and future perspectives. Quintessence International, 51 (3), 248–257.

Chien, P., Parks, E., Eraso, F., Hartsfield, J., Roberts, W., et al. (2009). Comparison of reliability in anatomical landmark identification using two-dimensional digital cephalometrics and three-dimensional cone beam computed tomography in vivo. Dentomaxillofacial Radiology, 38 (5), 262–273.

Debelmas, A., Ketoff, S., Lanciaux, S., Corre, P., Friess, M., K, et al. (2019). Reproducibility assessment of Delaire cephalometric analysis using reconstructions from computed tomography. Journal of Stomatology, Oral and Maxillofacial Surgery, 121 (1), 35–39.

Dreyer, K. J., & Raymond Geis, J (2017). When machines think: Radiology’s next frontier. Radiology, 285 (3), 713–718.

Durão, A. P. R., Morosolli, A., Pittayapat, P., Bolstad, N., Ferreira, A. P., et al. (2015). Cephalometric landmark variability among orthodontists and dentomaxillofacial radiologists: a comparative study. Imaging Science in Dentistry, 45 (4), 213–20.

Farooq, M. U., Khan, Mohd. A., Imran, S., Sameera, A., Qureshi, A., et al. (2016). Assessing the Reliability of Digitalized Cephalometric Analysis in Comparison with Manual Cephalometric Analysis. Journal of Clinical and Diagnostic Research, 10 (10), 20–23.

Forsting, M (2017). Machine Learning Will Change Medicine. Journal of Nuclear Medicine, 58 (3), 357–358.

Hung, K., Montalvao, C., Tanaka, R., Kawai, T., Bornstein, M. M (2019). The use and performance of artificial intelligence applications in dental and maxillofacial radiology: A systematic review. Dentomaxillofacial Radiology, 48 (20190107), 1-22, 2019.

Hwang, H.-W., Park, J.-H., Moon, J.-H., Yu, Y., Kim, H., H. et al. (2020). Automated identification of cephalometric landmarks: Part 2- Might it be better than human? The Angle Orthodontist, 90 (1), 69–76.

Khan, A., Javed, M. Q., Bilal, R., Gaikwad, R. N (2020). Retrospective quality assurance audit of Lateral Cephalometric Radiographs at postgraduate teaching hospital. Pakistan Journal of Medical Sciences, 36 (7), 1601-1606.

Khanagar, S. B., Al-Ehaideb, A., Maganur, P. C., Vishwanathaiah, S., Patil, S., et al. (2021). Developments, application, and performance of artificial intelligence in dentistry – A systematic review. Journal of Dental Sciences, 16 (1), 508–522.

Kunz, F., Stellzig-Eisenhauer, A., Zeman, F., Boldt, J (2019). Artificial intelligence in orthodontics: Evaluation of a fully automated cephalometric analysis using a customized convolutional neural network. Journal of Orofacial Orthopedics / Fortschritte der Kieferorthopädie, 81 (1), 52–68.

Landis, J. R., Koch, G. G (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–74.

Leonardi, R., Giordano, D., & Maiorana, F (2009). An evaluation of cellular neural networks for the automatic identification of cephalometric landmarks on digital images. Journal of Biomedicine and Biotechnology, (2009), 1-11.

Livas, C., Delli, K., Spijkervet, F. K. L., Vissink, A., Dijkstra, P. U (2019). Concurrent validity and reliability of cephalometric analysis using smartphone apps and computer software. The Angle Orthodontist, 89 (6), 889–896.

Mahto, R. K., Kafle, D., Giri, A., Luintel, S., Karki, A (2022). Evaluation of fully automated cephalometric measurements obtained from web-based artificial intelligence driven platform. BMC Oral Health, 22 (1), 1-8.

Masse, J.-F (2019). Will the orthodontic profession disappear? Journal of Dental Sleep Medicine, 6 (2), 1-2.

Mcnamara, A (1984). A method of cephalomettic evaluation. American Journal of Orthodontics, 6, 449-469.

Meriç, P., & Naoumova, J (2020). Web-based Fully Automated Cephalometric Analysis: Comparisons between App-aided, Computerized, and Manual Tracings. Turkish Journal of Orthodontics, 33 (3), 142–149.

Moon, J. H., Hwang, H. W., Yu, Y., Kim, M. G., Donatelli, R. E., L. …, S. J (2020). How much deep learning is enough for automatic identification to be reliable? A cephalometric example. Angle Orthodontist, 90 (6), 823–830.

Obermeyer, Z., & Emanuel, E. J (2016). Predicting the Future — Big Data, Machine Learning, and Clinical Medicine. New England Journal of Medicine, 375 (13), 1216–1219.

Olmez, H., Gorgulu, S., Akin, E., Bengi, A. O., Tekdemir, İ., Ors, F (2011). Measurement accuracy of a computer-assisted three-dimensional analysis and a conventional two-dimensional method. The Angle Orthodontist, 81 (3), 375–382.

Ongkosuwito, E. M., Katsaros, C., Van’t Hof, M. A., Bodegom, J. C., Kuijpers-Jagtman, A. M (2002). The reproducibility of cephalometric measurements: a comparison of analogue and digital methods. European Journal of Orthodontics, 24, 655–665.

Park, J.-H., Hwang, H.-W., Moon, J.-H., Yu, Y., Kim, H., H. et al. (2019). Automated identification of cephalometric landmarks: Part 1—Comparisons between the latest deep-learning methods YOLOV3 and SSD. The Angle Orthodontist, 89 (6), 903–909.

Ravikumar, D., N., S., Ramakrishna, M., Sharna, N., Robindro, W (2019). Evaluation of McNamara’s analysis in South Indian (Tamil Nadu) children between 8–12 years of age using lateral cephalograms. Journal of Oral Biology and Craniofacial Research, 9 (2), 193–197.

Santoro, M., Jarjoura, K., & Cangialosi, T. J (2006). Accuracy of digital and analogue cephalometric measurements assessed with the sandwich technique. American Journal of Orthodontics and Dentofacial Orthopedics, 129 (3), 345–351.

Shahidi, S., Oshagh, M., Gozin, F., Salehi, P., Danaei, S. M (2013). Accuracy of computerized automatic identification of cephalometric landmarks by a designed software. Dentomaxillofacial Radiology, 42, (1), p. 1-8.

Silva, T. P., Hughes, M. M., Menezes, L. Dos S., Melo, M. De F. B. De, Takeshita, W. M., Freitas, P. H. L. De (2021). Artificial Intelligence-Based Cephalometric Landmark Annotation and Measurements According to Arnett’s Analysis: Can we trust a bot to do that? Dentomaxillofacial Radiology, 50, (20200548), 1-6.

Subramanian, A. K., Chen, Y., Almalki, A., Sivamurthy, G., Kafle, D (2022). Cephalometric Analysis in Orthodontics Using Artificial Intelligence—A Comprehensive Review. BioMed Research International, 2022, 1–9.

Yu, H. J., Cho, S. R., Kim, M. J., Kim, W. H., Kim, J. W., Choi, J (2020). Automated Skeletal Classification with Lateral Cephalometry Based on Artificial Intelligence. Journal of Dental Research, 99 (3), 249–256.

Zamrik, O. M., & Iseri, H (2021). The reliability and reproducibility of an Android cephalometric smartphone application in comparison with the conventional method. Angle Orthodontist, 91 (2), 236–242.

Downloads

Published

19/10/2022

How to Cite

SOUZA, L. L. T. de; SILVA, T. P. .; SILVA FILHO, W. J. e; LIMA, B. N. S.; MEIRELES, A. C. N.; OLIVEIRA, I. T. de S. .; TAKESHITA, W. M. . Evaluation of the performance of two software artificial intelligence-based by means of the measurements according to Mcnamara’s Analysis in lateral cephalometric radiographs. Research, Society and Development, [S. l.], v. 11, n. 14, p. e73111435820, 2022. DOI: 10.33448/rsd-v11i14.35820. Disponível em: https://rsdjournal.org/index.php/rsd/article/view/35820. Acesso em: 17 apr. 2024.

Issue

Vol. 11 No. 14

Section

Health Sciences

License

Copyright (c) 2022 Laura Luiza Trindade de Souza; Thaisa Pinheiro Silva; William José e Silva Filho; Bruno Natan Santana Lima; Amanda Caroline Nascimento Meireles; Iris Tamara de Santana Oliveira; Wilton Mitsunari Takeshita

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors who publish with this journal agree to the following terms:

1) Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

2) Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

3) Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.