Usability Heuristic Evaluation of a Web-based Case Simulator for Dentists

Objectives: Dental Decision Simulation (DDSim) is a web-based case simulator for dentists to practice evidence-based care. The purpose of this evaluation was to identify issues according to established heuristics and improve the application.
Methods: Three patient cases of varying complexity were reviewed by four evaluators. Applying Nielsen’s heuristics, the evaluators worked through the cases and documented violations. The evaluators met to review, collate, and document issues and brainstorm recommendations. Usability issues were labeled with the heuristic involved, usability severity, developer, and adjusted scores. Usability severity was scored, 1 to 4, where 1 being a cosmetic problem and 4 being a catastrophic problem. The developer score, 4 to 1, where 4 represented an easy fix and 1 represented a fix not to be attempted. These two scores were multiplied to get the adjusted score. Issues were sorted by highest adjusted score. Mock-up design solutions were provided to the developers.
Results: Seventy-four usability issues were identified. Severe issues with their associated heuristic and highest adjusted score are as follows: Consistency and efficiency of use violations when using procedure/diagnosis functionalities, 16; error prevention violation by being able to skip questions when completing a risk assessment, 15; consistency and recognition over recall violations when using hyperlinks, 13; error recovery violation when an incorrect diagnosis was selected and an error message was displayed, 13. On multiple screens, procedure/diagnosis, tooth number, and surface(s) were needed to enter information. On one screen, they appeared left-to-right, ending in an “Add” button. On another screen, they appeared “stacked,” with the procedure/diagnoses appearing on top of the tooth number and surface(s), and ending in a “Record” button. We recommended a consistent left-to-right layout and wording.
Conclusions: This heuristic evaluation was a time-saving and cost-effective resource to identify the “low-hanging fruit” of DDSim. The developers worked on addressing the usability issues identified in this evaluation.