IADR Abstract Archives
Artificial Intelligence Classifier and Search Strategy Performance for Retrieving Studies on TMD
Objectives: The Temporomandibular Disorders (TMD) field has accumulated a large body of evidence. Given this scenario, developing optimal search strategies and using Artificial Intelligence (AI) for evidence synthesis may be crucial to manage the exponential increase of articles and the complexity of the skills required to identify the best available research evidence. The TMD L●OVE Platform Project, supported by Epistemonikos Foundation, intends to provide a one-stop-shop repository for TMD evidence. Considering that systematic reviews conduct high-quality searches, this study assesses the performance of the search strategy and the AI classifier against the total number of studies in a representative sample of TMD systematic reviews (SR).
Methods: Our reference standard was based on the relative recall method. All articles included in 60 random TMD SRs were used to create our sample. The search strategy sensitivity was calculated by counting how many studies were acquired. How many articles were TMD-relevant determined the AI classifier sensitivity.
Results: An initial search strategy yielded 172/203 studies in 20 SR with a sensitivity of 84.8%. The TMD AI classifier detected 136/203 studies (67.2%). Subsequently, additional terms were incorporated from the studies not initially identified and then re-tested this upgraded version. The TMD AI classifier reached a search strategy sensitivity of 93.3% and 83.3% against the studies included in a new set of SRs (235 and 210/out of 252 studies included in 20 SR). A third search strategy and the TMD Artificial intelligence classifier versions test reached sensitivity of 93.9% and 84.6%, respectively (263 and 237/280 studies in 20 SR).
Conclusions: The proposed search strategy retrieved TMD-related articles effectively. However, artificial intelligence mislabelled a percentage of items. Optimising TMD evidence synthesis is promising, but further research is still required.
Methods: Our reference standard was based on the relative recall method. All articles included in 60 random TMD SRs were used to create our sample. The search strategy sensitivity was calculated by counting how many studies were acquired. How many articles were TMD-relevant determined the AI classifier sensitivity.
Results: An initial search strategy yielded 172/203 studies in 20 SR with a sensitivity of 84.8%. The TMD AI classifier detected 136/203 studies (67.2%). Subsequently, additional terms were incorporated from the studies not initially identified and then re-tested this upgraded version. The TMD AI classifier reached a search strategy sensitivity of 93.3% and 83.3% against the studies included in a new set of SRs (235 and 210/out of 252 studies included in 20 SR). A third search strategy and the TMD Artificial intelligence classifier versions test reached sensitivity of 93.9% and 84.6%, respectively (263 and 237/280 studies in 20 SR).
Conclusions: The proposed search strategy retrieved TMD-related articles effectively. However, artificial intelligence mislabelled a percentage of items. Optimising TMD evidence synthesis is promising, but further research is still required.