Ovid and PubMed Underestimate Orthodontic RCT Numbers Compared to Hand-Searching

To examine the results of searching MEDLINE, via Ovid or PubMed, for randomised controlled trials (RCTs), published in four key orthodontic journals between 01/01/2001 and 31/12/2010 compared with hand-searching these journals using Cochrane Collaboration criteria.

Methods:

RF satisfactorily completed the Cochrane Oral Health Group’s (OHG) hand-searching test search.

The American Journal of Orthodontics and Dentofacial Orthopaedics, Angle Orthodontist, European Journal of Orthodontics and Journal of Orthodontics were hand-searched to identify all RCTs published in these journals between 01/01/2001 and 31/12/2010.

Ovid and PubMed were used to search MEDLINE, using the term publication type RCT for “orthodontic”, for RCTs published in these journals between 01/01/2001 and 31/12/2010. A free text PubMed search using the terms “orthodontic” AND “random*” was also performed. The numbers of RCTs identified using each method were compared.

Results:

Hand-searching identified 218 RCTs, fulfilling Cochrane criteria, published in these journals between 01/01/2001 and 31/12/2010. 56 RCTs (25.7%) were found by all 3 electronic searches, however 37 RCTs (17%) were not identified by any of these strategies.

Ovid was significantly less sensitive than PubMed (OR 8.43, 95%CI 5.48, 12.97) missing 157 RCTs (72.0%), while PubMed missed 51 (23.4%). The free text PubMed search, using the terms orthodontic AND random*, was the most sensitive missing 45 RCTs (20.6%) but this was not statistically significant (OR 0.85, 95%CI 0.54, 1.34)

Conclusions:

Handsearching was much more sensitive than electronic searching at identifying RCTs published in key orthodontic journals. There was a significant underestimation of RCTs using Ovid and PubMed. This may bias systematic reviews relying on electronic searching to identify eligible papers.