IADR Abstract Archives

A Comparison of Different Standard Setting Methods in Final Year Dental Examination

Objectives: This study aims to compare between the conventional, norm reference and modified Angoff standard-setting methods for final year dental examination and to estimate the test-retest and inter-rater reliability of the modified Angoff method.
Methods: The norm reference method of standard setting (mean minus 1 SD) was applied to the real scores of 40 final year (5th year) dental students on a multiple-choice examination (MCQ), short answer question (SAQ) and objective structured clinical examination (OSCE). A panel of 10 judges set the standard using the modified Angoff method for the same paper in one sitting and two judges set the passing mark of 10 OSCE questions after 2 weeks. Comparison of the grades and pass/fail rates derived from the absolute standard, norm reference and the Angoff methods were made. The test-retest and inter-rater reliability of the modified Angoff method were also assessed.
Results: The pass rate with the absolute standard was 100% (40/40), norm-reference method was 62.5% (25/40) and that by the Angoff method was 80% (32 out of 40). The percentage agreement between modified Angoff method and convkentional method was 59% and with norm reference was 55%, between conventional standard and norm-reference was 30%. The modified Angoff method had a good inter-rater reliability of 0.876 and excellent test-retest reliability of 0.941.
Conclusions: There were significant differences in the outcomes of these three standard setting methods, as shown by the difference in the proportion of candidates that passed and failed the assessment. The modified Angoff method was found to have an acceptable percentage agreement with other methods, a good inter-rater reliability and an excellent intra-rater reliability to be applied in final year dental assessment with a future direction for the professional qualifying examinations to determine the passing candidates.

2020 South East Asia Division Meeting (Virtual)

2020
P043
Education Research
  • Abd Rahman, Aida  ( Universiti Teknologi MARA , Sungai Buloh , Selangor , Malaysia ;  University of South Wales , Cardiff , Wales , United Kingdom )
  • Baharuddin, Izyan  ( Universiti Teknologi MARA , Sungai Buloh , Selangor , Malaysia )
  • Abu-hassan, Mohamed-ibrahim  ( Universiti Teknologi MARA , Sungai Buloh , Selangor , Malaysia )
  • Davies, Sally  ( University of South Wales , Cardiff , Wales , United Kingdom )
  • NONE
    Poster Session
    Poster 3 (Senior Hatton)
    Thursday, 11/26/2020 , 02:30PM - 03:00PM