IADR Abstract Archives
Applying Natural Language Processing to Classify Restorative Materials in Abstracts
Objectives: The aim of this study was to use natural language processing (NLP) to classify the restorative material class used in studies based on the abstracts.
Methods: Abstracts containing CAD/CAM restorative materials published until July 2021 were collected from PubMed. Two independent evaluators manually classified the studies according to whether resin, ceramic, composite, and polymer were used as the material classification. NLP methods (both supervised and unsupervised) were implemented to classify the abstracts into four material classes (resin, ceramic, composite, and polymer) using one-hot encoding. The supervised methods involved removing expressions (i.e., epoxy resin) from the abstracts that could result in a false positive, whereas the unsupervised methods did not remove any expressions. The precision, recall, F1, and accuracy scores were calculated for the NLP results using the independent evaluators' results as a reference.
Results: From 526 found, the independent evaluators classified these studies as follows: 238 resin, 272 ceramic, 207 composite, and 134 polymer. Amongst the NLP methods, the highest accuracy scores for each material were: 1 - Resin: 0.87 (Keybert supervised, Rake supervised, Yake supervised, and supervised regular expression matching); 2- Ceramic: 0.83 (regular expression matching both supervised and unsupervised); 3 - Composite: 0.87 (Keybert supervised, Rake both supervised and unsupervised); 4 - Polymer: 0.97 (Biosample Annotator both supervised and unsupervised). For the other scores (precision, recall, F1), Keybert and Rake (either unsupervised or supervised) generally performed best for classifying resin, ceramic, and composite studies. The classification of polymer studies was an exception, with Biosample Annotator performing best.
Conclusions: NLP can assist with the classification of restorative materials avoiding misuse and wrong clinical indication of these materials. However, no model outperforms all materials classes, and NLP classification tasks should have model and hyperparameters tuning for each type of restorative material class.
Methods: Abstracts containing CAD/CAM restorative materials published until July 2021 were collected from PubMed. Two independent evaluators manually classified the studies according to whether resin, ceramic, composite, and polymer were used as the material classification. NLP methods (both supervised and unsupervised) were implemented to classify the abstracts into four material classes (resin, ceramic, composite, and polymer) using one-hot encoding. The supervised methods involved removing expressions (i.e., epoxy resin) from the abstracts that could result in a false positive, whereas the unsupervised methods did not remove any expressions. The precision, recall, F1, and accuracy scores were calculated for the NLP results using the independent evaluators' results as a reference.
Results: From 526 found, the independent evaluators classified these studies as follows: 238 resin, 272 ceramic, 207 composite, and 134 polymer. Amongst the NLP methods, the highest accuracy scores for each material were: 1 - Resin: 0.87 (Keybert supervised, Rake supervised, Yake supervised, and supervised regular expression matching); 2- Ceramic: 0.83 (regular expression matching both supervised and unsupervised); 3 - Composite: 0.87 (Keybert supervised, Rake both supervised and unsupervised); 4 - Polymer: 0.97 (Biosample Annotator both supervised and unsupervised). For the other scores (precision, recall, F1), Keybert and Rake (either unsupervised or supervised) generally performed best for classifying resin, ceramic, and composite studies. The classification of polymer studies was an exception, with Biosample Annotator performing best.
Conclusions: NLP can assist with the classification of restorative materials avoiding misuse and wrong clinical indication of these materials. However, no model outperforms all materials classes, and NLP classification tasks should have model and hyperparameters tuning for each type of restorative material class.