Validating Caries Indices From An Electronic Health Record

Objectives: To design, program, test, and validate automated electronic health record (EHR) procedures to calculate traditional caries indices compared to a previously-validated clinical electronic data capture (EDC) system and SAS program.
Methods: The research team provided specifications to EHR (axiUm, Exan Group, Coquitlam, British Columbia, Canada) programmers who developed database procedures to calculate traditional caries indices (DMFS, dmfs, DMFT, dmft, etc.) based on data collected in the EHR. Case scenarios of primary, mixed, and permanent dentition were developed and entered into the EHR. Caries indices were calculated for validation using developed procedures. Calculations included decayed, missing, and filled teeth and surfaces. Four university clinician-researchers abstracted the EHR caries exam data into the caries research instrument (CARIN) EDC and post-processed with SAS to compute caries indices. Discrepancies among the 4 abstractors were resolved by consensus. Agreement statistics were computed among the 4 abstractors (surface level kappas) and between the automated EHR calculation and the abstractor consensus (person level Lin’s concordance correlations (LCCs)).
Results: The team developed 45 test cases covering a wide variety of common and uncommon scenarios; 64% of scenarios had decay or filled surfaces; mean dmfs+DMFS was 31 and mean ds+DS was 5. Researchers were in extremely high agreement with the senior clinical abstractor: kappa values 0.95-0.97. Abstractor consensus had extremely high correlation with EHR caries indices: LCCs>0.99 for dmfs, dmft, DS, ds, DT, dt, ms, mt, FS, fs, FT, and ft; LCCs>0.95 for DMFS and DMFT; and LCCs of 0.92-0.93 for MS and MT. Permanent missing surfaces and teeth had slightly lower agreement.
Conclusions: The abstractors were in extremely high agreement and the automated procedures to calculate caries indices in the EHR were validated. Caries indices, important primary outcome measures for research and patient care purposes, can be reliably derived from an electronic health record.