IADR Abstract Archives
Confirmation Bias of Dentists Using Artificial Intelligence for Caries Detection
Objectives: Artificial intelligence (AI) is increasingly used to support caries detection, for example, on radiographs. We aimed to assess if dentists who are exposed to a series of correct AI detections started to rely on the AI even if it subsequently showed erroneous findings, i.e., showed confirmation bias.
Methods: Twenty dentists were introduced to an AI software, informing them of its diagnostic accuracy for caries detection. Forty bitewing radiographs, each of them showing at least one caries lesion (confirmed by 4+1 experienced dentists as the reference test) were employed. In the first 20 images (Set 1), the caries detection by the AI matched that of the reference test. In the next 20 images (Set 2), 35% contained misdiagnoses by the AI, e.g., superficial caries being missed, or non-carious areas being diagnosed as carious. Supported by the AI, dentists examined each image for the presence of caries. The time spent per image was recorded. Dentists’ diagnostic accuracy against the reference test was assessed.
Results: Dentists had lower sensitivity on Set 2 versus Set 1 (mean: 0.77 [95% CI: 0.71, 0.82] vs. 0.87 [0.82, 0.92]), and also specificity (0.96 [0.95, 0.97] vs. 0.99 [0.98, 0.99]), and negative predictive values (0.93 [0.91, 0.94] vs. 0.99 [0.98, 0.99]). Inter-rater agreement between the dentists and AI was limited but not significantly different between sets (Set 1: 0.42 [0.06, 0.77], Set 2: 0.63 [0.36, 0.90]). The time spent per image did not differ significantly between sets.
Conclusions: After being exposed to a series of correct AI assessments, dentists continued to agree with the AI, even if its detections were incorrect. There is a need to safeguard dentists against confirmation bias.
Methods: Twenty dentists were introduced to an AI software, informing them of its diagnostic accuracy for caries detection. Forty bitewing radiographs, each of them showing at least one caries lesion (confirmed by 4+1 experienced dentists as the reference test) were employed. In the first 20 images (Set 1), the caries detection by the AI matched that of the reference test. In the next 20 images (Set 2), 35% contained misdiagnoses by the AI, e.g., superficial caries being missed, or non-carious areas being diagnosed as carious. Supported by the AI, dentists examined each image for the presence of caries. The time spent per image was recorded. Dentists’ diagnostic accuracy against the reference test was assessed.
Results: Dentists had lower sensitivity on Set 2 versus Set 1 (mean: 0.77 [95% CI: 0.71, 0.82] vs. 0.87 [0.82, 0.92]), and also specificity (0.96 [0.95, 0.97] vs. 0.99 [0.98, 0.99]), and negative predictive values (0.93 [0.91, 0.94] vs. 0.99 [0.98, 0.99]). Inter-rater agreement between the dentists and AI was limited but not significantly different between sets (Set 1: 0.42 [0.06, 0.77], Set 2: 0.63 [0.36, 0.90]). The time spent per image did not differ significantly between sets.
Conclusions: After being exposed to a series of correct AI assessments, dentists continued to agree with the AI, even if its detections were incorrect. There is a need to safeguard dentists against confirmation bias.