Developmental Evaluation and Research: An Example in Dental Case-Management

Objectives: Developmental evaluation is part of a continuum of evaluation for emerging evidence-bases or best practices for applications as guidelines for program implementation or situationally specific questions. Its evolution has taken decades and the concept has been viewed in tension with research for social innovation and complex interventions. Nonetheless, dental literature appears essentially void of its consideration. The evaluation of a dental case-management program associated with a public school-based sealant program demonstrates need for duality of evaluation and research.
Methods: A program evaluator and an epidemiologist were contracted in 2019 to ecologically evaluate a dental case-management program in its 5th year as an intervention (CMI) for improving outcomes related to a department of public health (DPH) urban school-based sealant program (SBSP). The DPH and CMI prioritized zip codes and schools with particularly negative SBSP outcomes.
Results: The complexities of the ecologically available social (e.g., socio-economic status, demographics and resources) and political situations (e.g., state budget impasse) dampened the ability to access typical evaluation measures for CMI. Zip code level assessments failed to capture improvements with CMI versus comparisons. Shifting focus to students who had participated in the SBSP across years of CMI in their schools demonstrated improvement (reduction) among the proportions of participants remaining in “Urgent Care” on SBSP screening. Change of "Urgent Care" percentages in year 4 versus year 1 was -0.8% and -1.1% (mean=-0.95%) for comparison zip codes, +1.5%, -2.1%, and -0.9% (mean=-0.50%) for intervention zip codes, and -8.1%, -13.2%, and -18.5% (mean=-13.3%) for CMI intensely focused schools.
Conclusions: Evaluating the CMI program adjunct to an urban SBSP under traditional assessments failed on demonstrating program value. The inability to capture ecological contrasts at the zip code level reflects situational complexities of the diversity of the schools and their neighborhood. Refined focus informed by specific programatic structures and developmental evaluation for the urban area allowed for value to be detected. Community-based interventions can benefit by involving partners with research skills across program developement through evaluation.