Assessing Patient Safety Training in Detecting Dental Adverse Events

Objectives: Our objective was to develop and test a training course for chart reviewers to consistently identify dental adverse events (AEs) in electronic health records (EHR) and compare AEs collected by expert and trained new chart reviewers from the EHR.
Methods: We developed a web-based ‘dental patient safety’ training course for new chart reviewers that included 1) Education 2) Skill training using videos and guided walkthrough 3) Quizzes with feedback and 4) Hands on learning exercises. The course included introductory material on patient safety and AEs and a methodology of identifying, documenting, categorizing and assigning severity to AEs. After taking the course, AEs identified by new chart reviewers were compared against AEs identified by experts in charts indicating post-surgical treatment complications. Interrater reliability was calculated for identifying charts with AEs.
Results: Expert and new chart reviewers reviewed 169 charts triggered for possible post-surgical treatment complications between Jan 01-Dec 31, 2015. Experts identified 50 AEs whereas new chart reviewers identified 56 AEs. Both expert and new chart reviewers identified 32 common charts containing AEs. Experts identified additional 13 charts and new chart reviewers identified 21 charts with AEs. Experts reviewed those 21 charts and did not find AEs. Experts summarized AEs in their own words and added pertinent patient’s notes as supporting evidence whereas new chart reviewers often used patient’s notes in their AE summary. Experts classified AEs into one or two categories while new chart reviewers had more than two categories. Experts assigned severity only to AEs whereas new chart reviewers assigned severity to AEs and their outcomes. There was moderate agreement (PABAK = 0.60) in assessing the charts that contained the AEs between expert and new chart reviewers.
Conclusions: Our results suggest that our training was an effective tool in measuring the level of calibration of new chart reviewers.