Inter-Rater Variability in Orthodontic Treatment Needs Assessment

Objectives: To describe the inter-rater variability in the assessment of the need for orthodontic treatment among 5th grade children in 3 Danish municipalities. Methods: In each municipality a random sample of 100 5th graders (age 10-11 years) were screened to eliminate those children who had already begun orthodontic treatment, those who had an unquestionable treatment need, and those who definitely required no treatment. The remaining children were invited to participate in the study, and among the 22, 23 and 20 candidates, respectively, a total of 18, 20 and 19 children accepted. In connection with the annual DSOS meeting an inter-rater variability assessment exercise was organised comprising 99 orthodontic specialists. These were allocated in blocks of 33 to each municipality dental clinic where they performed independent evaluations of the orthodontic treatment need (yes/no) of each child as judged by a clinical examination. Later, the orthodontic specialists were rotated among the municipalities and this time asked to assess the orthodontic treatment need of the children based on study casts and a standard set of extraoral photographs. Results: A notable variation in the specialists' propensity to recommend orthodontic treatment for these borderline cases. While the propensity to recommend treatment based on a clinical evaluation and on a model evaluation was positively correlated it was also clear that some specialists obtained pronouncedly different results depending on the evaluation mode. Conclusion: The results suggest a need for further clarification of the criteria used to identify orthodontic treatment need in order to reduce the inter-rater variation in the assessment of borderline cases.