Accuracy of Generative Artificial Intelligence in providing oral health information

Objectives: Generative artificial intelligence (AI), including Natural Language Processing (NLP) and its specific application Large Language Models (LLM), has recently experienced remarkable development and popularity. Patients may seek advice by inputting their conditions into LLM. The accuracy of LLM in interpreting written input and providing accurate answers is crucial in providing personalized oral health information to individual patients.
Methods: A total of 1461 multiple-choice questions from the dental licensing examinations including all dental subjects were input into ChatGPT 4.0. The performance of AI in examinations and individual dental subjects was analyzed and compared to that of ChatGPT 3.5.
Results: ChatGPT 4.0 correctly answered 70.5% (n=1030/1461) of questions which was higher than the passing marks of the included dental examinations. ChatGPT performed the best in Periodontics (94.4% correct answers, n=102/108) while the worst was in Orthodontics and Pediatric Dentistry (51.9% correct, n=80/154) and Restorative Dentistry and Prosthodontics (58.6% correct, n=112/191). While ChatGPT 4.0 answered 327 more questions correctly, it answered 102 incorrectly compared to ChatGPT 3.5.
Conclusions: ChatGPT 4.0 passed the written dental licensing examinations and may be used for providing patient education, especially in periodontics. This may have implications for the role of dentists in patient management as well as in related dental education and training. While ChatGPT 4.0 performed better than ChatGPT 3.5, it may not perform universally better in all scenarios, and further improvement is needed.