Investigation of Transfer Learning Strategies for Dental Radiographic Imagery

Objectives: Deep learning models are often initialized with weights of pre-trained ImageNet models, which usually boosts performance compared with random initialization. Notably, features learned on ImageNet may differ from features on radiographs, with initialization on medical images being potentially more suitable. We aimed to compare transfer learning on ImageNet versus CheXpert, an open repository for chest radiographs, versus random initialization for the segmentation of anatomic tooth structures (enamel, dentin, pulpal cavity) on bitewing radiographs.
Methods: We built 12 segmentation models by combining U-Net with different backbones (ResNet, VGG, DenseNet) and applied three initialization strategies (ImageNet, CheXpert, random). The resulting 36 models were trained up to 200 epochs with the Adam optimizer (lr=0.0001) and a batch size of 16. Our dataset consisted of 1721 bitewing human-annotated bitewings. We utilized a train/validation/test split of 80%/10%/10%. Model performances were primarily quantified by the Dice score.
Results: Random initialization led to a mean (SD) Dice score of 0.843 (0.007), while ImageNet and CheXpert reached 0.856 (0.014) and 0.856 (0.01), respectively. The lower performance of a random initialization was statistically significant compared to training based on ImageNet (p=0.013/t-test) or CheXpert (p=0.001). No significant difference was observed between ImageNet and CheXpert (p=0.944/t-test).
Conclusions: Transfer learning boosts model performances. The origin of transferred knowledge seems less relevant. Dental segmentation models benefitted similarly from pre-training on RGB photographs or monochromatic chest radiographs.