Classifying Features for an Oral Lesion Image Learning Tool

Objectives: A web-based image classification tool (DiLearn) was developed to facilitate active learning in the oral health profession. Students engage with oral lesion images using swipe gestures to classify each image into pre-determined categories (e.g., Left – refer; Right – no intervention). To assemble the training modules and to provide feedback to students, DiLearn requires each oral lesion image to be classified with various features displayed in the image. Collection of accurate meta-information is a crucial step for enabling the self-directed active learning approach taken in DiLearn. The purpose of this study is to evaluate the classification consistency of features in oral lesion images by experts and students for use of the learning tool.
Methods: Twenty oral lesion images from DiLearn’s image bank were classified by three oral lesion experts and two senior dental hygiene students using the same rubric containing eight features. Fleiss’ Kappa was used to evaluate the classification agreement among oral lesion experts and to evaluate the overall agreement. Cohen’s Kappa was used to compare agreement between student raters.
Results: Classification agreement between the three experts ranged from identical (Fleiss’ Kappa = 1) for Clinical Action to slight agreement for Border: Regularity (Fleiss’ Kappa = 0.136) with the majority of categories having fair to moderate agreement (Fleiss’ Kappa = 0.332-545). With the exception of Morphology, inclusion of the two student raters with the experts yielded fair to moderate overall classification agreement (Fleiss’ Kappa = 0.224-0.554). In sum, the feature Clinical Action can be accurately classified while other anatomical features indirectly related to diagnosis have lower classification consistency.
Conclusions: Findings suggest one oral lesion expert or two student raters can provide fairly consistent meta-information for select category of feature implicated in the creation of image classification tasks in DiLearn.