IADR Abstract Archives
Comparison of the AMSTAR-2 and ROBIS Tools for Quality Appraisal of Systematic Reviews
Objectives: The assessment of the quality of the studies is an essential aspect of evidence-based medicine. The two popular tool for quality analysis of systematic reviews are AMSTAR-2 and ROBIS. Since these methods might have subjective variations, this study aimed to identify the statistical correlations between these methods. It also attempted to explore the sources of concordance which existed between the indices.
Methods: A total of 100 systematic reviews were randomly selected from a pool of 144 studies through simple randomization. The AMSTAR-2 and ROBIS were used for quality analysis by one trained and calibrated reviewer. The intra-rater reliability was assessed and descriptive analyses were performed to assess the correlation between the two methods. The homogeneity was evaluated by using Stuart Maxwell test. A subgroup analysis was also performed.
Results: Among the 100 SRs included in this study, 63 were classified as critically low and 25 as low in confidence as per AMSTAR-2 and high risk of bias was observed in 78 of them as per ROBIS. The high confidence and low ROB were observed in 11 and 22 systematic reviews respectively. The quality of 96.4% (n=54) of the studies without registration of their protocols in any of the registries was found to be critically low and the risk of bias was high in 55 of them. Among the 78 studies with high risk of bias, critically low quality was seen in 61 while among the SRs with critically low quality 61 had high risk of bias. The symmetry and homogeneity between the observations of AMSTAR-2 and ROBIS were found to be absent.
Conclusions: Within the limitations of present study, it can be concluded that the two methods of critical appraisal of SRs namely AMSTAR-2 and ROBIS lacked statistical symmetry and homogeneity. A trend could be discerned related to higher chances of having high quality and low ROB with an a priori registration of protocols. The subjectivity related to performing any of these quality appraisals and lack of symmetry warrants the use of both the tools when performing overview of systematic reviews/umbrella reviews, evidence mapping and quality analysis and meta-synthesis.
Methods: A total of 100 systematic reviews were randomly selected from a pool of 144 studies through simple randomization. The AMSTAR-2 and ROBIS were used for quality analysis by one trained and calibrated reviewer. The intra-rater reliability was assessed and descriptive analyses were performed to assess the correlation between the two methods. The homogeneity was evaluated by using Stuart Maxwell test. A subgroup analysis was also performed.
Results: Among the 100 SRs included in this study, 63 were classified as critically low and 25 as low in confidence as per AMSTAR-2 and high risk of bias was observed in 78 of them as per ROBIS. The high confidence and low ROB were observed in 11 and 22 systematic reviews respectively. The quality of 96.4% (n=54) of the studies without registration of their protocols in any of the registries was found to be critically low and the risk of bias was high in 55 of them. Among the 78 studies with high risk of bias, critically low quality was seen in 61 while among the SRs with critically low quality 61 had high risk of bias. The symmetry and homogeneity between the observations of AMSTAR-2 and ROBIS were found to be absent.
Conclusions: Within the limitations of present study, it can be concluded that the two methods of critical appraisal of SRs namely AMSTAR-2 and ROBIS lacked statistical symmetry and homogeneity. A trend could be discerned related to higher chances of having high quality and low ROB with an a priori registration of protocols. The subjectivity related to performing any of these quality appraisals and lack of symmetry warrants the use of both the tools when performing overview of systematic reviews/umbrella reviews, evidence mapping and quality analysis and meta-synthesis.
IMAGES
4010782_File000000.jpg
4010782_File000001.jpg
4010782_File000000.jpg
4010782_File000001.jpg