Student Theses, Dissertations and Capstones

Document Type

Thesis - NSU Access Only

Degree Name

Master of Science (M.S.) in Dentistry

Copyright Statement

All rights reserved. This publication is intended for use solely by faculty, students, and staff of Nova Southeastern University. No part of this publication may be reproduced, distributed, or transmitted in any form or by any means, now known or later developed, including but not limited to photocopying, recording, or other electronic or mechanical methods, without the prior written permission of the author or the publisher.

Department

College of Dental Medicine

First Advisor

Cristina Godoy

Date of original Performance / Presentation

January 2025

Publication Date / Copyright Date

1-1-2025

Abstract

Objectives: The objective of this study was to analyze ChatGPT 4o Mini’s answers to the most common questions about orthognathic surgery and evaluate the differences in answers provided based on the language used (English and Spanish). Four orthodontists, two from academic settings and two from private practice settings will evaluate the answers. Each sector will have one English speaker and one Spanish speaker to account for any bias. Background: Artificial intelligence (AI) is being used to aid in answering everyday questions, including questions about orthodontic treatment. ChatGPT has become one of the most commonly used artificial intelligence chatbots with a worldwide consumer base with the capability to provide 80 different languages. There is little literature to evaluate the differences that language has on the answers provided by ChatGPT, especially with respect to orthognathic surgery. Methods: ChatGPT 4o Mini was asked to generate sixty of the most commonly asked questions about orthognathic surgery. Sixty questions were chosen because the desired power of hypothesis is 0.8 with four evaluators; the sample size will be a minimum of thirty-five. Those sixty questions were evaluated by the four graders to remove any repeated or irrelevant questions and reached a total of forty-five questions. The forty-five questions were translated into Spanish and evaluated by the Spanish speaking orthodontists to ensure accuracy. Forty-five commonly asked questions regarding orthognathic surgery were asked to ChatGPT 4o Mini in both English and Spanish. The forty-five questions in English were evaluated by one English speaking orthodontist from an academic background and one English speaking orthodontist from a private practice background. The forty-five questions in Spanish were evaluated by one Spanish speaking orthodontist from an academic background and one Spanish speaking orthodontist from a private practice background. The evaluations were scored independently by each orthodontist. The evaluations were scored using a modified four-point scale: 1: Objectively True, 2: Selected Facts, 3: Minimal Facts, 4: False. The accuracy of the responses were then compared. A joint meeting was held with all evaluators before scoring to establish a common understanding of the grading system. The meeting consisted of an explanation of each scoring scale and the criteria for each score. A pilot was done to make sure the scores were accurately provided. Each evaluator was given an answer to a question regarding orthognathic surgery and scored it using the four-point scale. This pilot was done to ensure the standardization of grading. Results: Forty-five questions were evaluated by the four orthodontists. For the orthodontist assigned to English 1, the grader had a mean of 1.02 ( ± 0.149) with amedian score of 1. For orthodontist assigned to English 2, the grader had a mean of 1.22 ( ± 0.420) with a median score of 1. For orthodontist assigned to Spanish 1, the grader had a mean of 1.22 ( ± 0.420) with a median score of 1. For orthodontist assigned to Spanish 2, the grader had a mean of 1.16 ( ± 0.520) with a median score of 1. The Wilcoxon test found to have statistically significant differences in score between the two English graders with a p-value being a 0.004. The Wilcoxon test found that the answers provided by the two Spanish graders were not statistically significant (p-value > 0.05). When the answers provided by ChatGPT 4o Mini in English were compared to the answers provided by ChatGPT 4o Mini in Spanish, it was found that the answers were not statistically significant (p-value > 0.05). The average grade for the English questions asked to ChatGPT 4o Mini was 1.12 while the average grade for the Spanish questions asked to ChatGPT 4o Mini was 1.19. Conclusions: 1) The answers provided by ChatGPT 4o Mini regarding orthognathic surgery are accurate. 2) There is no statistical difference between the answers provided by Chat GPT 4o Mini for the most common orthognathic surgery questions based on language (English vs Spanish). This suggests that the information provided by ChatGPT 4o Mini regarding orthognathic surgery does not have a major disparity between English and Spanish. 3) There is no statistical difference between the grades provided by Spanish speaking academic orthodontist vs the Spanish speaking private practice orthodontist. 4)There was a statistical difference between the grades provided by the English speaking academic orthodontist vs the English speaking private practice orthodontist.

Disciplines

Dentistry

Keywords

Artificial Intelligence; ChatGPT; Orthognathic Surgery; Patient Questions

Download

Download Full Text (1.8 MB)

Download
To access this thesis/dissertation you must have a valid nova.edu OR mynsu.nova.edu email address and create an account for NSUWorks.

Free My Thesis

If you are the author of this work and would like to grant permission to make it openly accessible to all, please click the Free My Thesis button.

  Contact Author

Share

 
COinS