Comparison of Statistical Methods in Randomized Controlled Trials

Objectives: Observations of repeated measurements on the same outcome variables are the most popular study design in dental research. For instance, to compare treatment efficacy of a new treatment for periodontal regeneration with conventional surgical treatment, clinical attachment level are measured before the surgery and one year after the surgery on the same teeth. Then statistical methods are used to test whether there is significant difference in the changes of attachment level between two treatments. The aims of this study are to investigate whether there are substantial differences in statistical power between different statistical methods in analyzing data from randomized controlled trials (RCT). Methods: Four univariate statistical methods were explored: testing the means of post-treatment score; testing the means of change scores; testing the means of percentage change scores; and using the Analysis of Covariance (ANCOVA). In addition, two multivariate methods were explored: random effect models and the Multivariate Analysis of Variance (MANOVA). Hypothetical data were generated by computer simulation of clinical trials that compare two treatments in periodontal research. The power was calculated by counting the success rate of these methods to detect a difference in the treatment effects between both groups. Results: ANCOVA was found to be the most powerful method in detecting differences in the treatment effects in most simulated scenarios. The commonly used methods such as change scores and percentage change scores had less power than ANCOVA, and their power was related to the correlation between pre- and post-treatment scores. Multivariate methods did not show greater power than ANCOVA. Conclusions: In RCTs with pre-test/ post-test design, ANCOVA is recommended for comparing the treatment effects between groups as it has the greatest power. Using ANCOVA can therefore reduce the risk of type II errors and thus reduce the sample sizes needed for trials.