Measurement in Evaluation
Evaluations can be divided into two major types: formative and summative. The Research Evaluation and Systems unit has experience conducting both types for a variety of programs.
Formative evaluations determine if a program has been implemented as intended. This is also described as a program's fidelity to the program model. Formative evaluations can be particularly important for programs with a more formal and rigorous model. Measures that take prominence in this type of evaluation refer to the model standards for the program as well as the procedures used to implement the program. The measures developed must refer to model standards. An example of a model standard is requiring that a particular procedure be conducted, such as client intake. Measures that refer to client intake might document that the intake occurred, the extent to which the intake procedure was complete, and the timing of the intake in relation to the initial contact with the client and subsequent services. A measure of client satisfaction with the intake process might also be appropriate.
Summative evaluations determine if a program has achieved the desired effects. In summative evaluations, outcomes are the key measures. These measures indicate if program goals were achieved. They can refer to the attainment of knowledge, perceptions or attitudes, actual behavior, or the attainment of a status. The measures can be short-term and correspond with the goal achievement at the time the service was completed or long-term and correspond with a time period that extends beyond the end of the service delivery. For programs serving children in after school programs, outcome measures might be academic achievement in school or GPA, attendance record in school, or perception of their relationships with a parent. The validity and reliability of the measurement tools selected are also important and the ability to use them in comparisons across subgroups and programs is often essential.
For more information on these services, please contact Terry J. Rhodes, Director of Research, Evaluation and Systems. You may call 850-921-4494 x 216 or email trhodes@ounce.org.
