Step 6: Evaluation

Evaluation is based on the CDC’s recommended broad outline of how to evaluate a community-based injury prevention program with the public health model in mind. Evaluation starts on day 1! Perhaps most critical to the evaluation process is the notion that there is more to evaluation than just capturing recidivism. Intermediate and surrogate measures (such as finding employment) along with qualitative outcomes are essential to collect in order to demonstrate the true value of these programs.

Evaluation Standards

Reach: Are the case managers going to the bedside of patients, prior to hospital discharge, to begin the process?
Feasibility: Is the target population being enrolled and staying enrolled?
Functionality: Are case managers “connecting” with the target population? Are case managers conducting a needs assessment? Are the case managers finding appropriate risk reduction resources in the community?
Process outcomes: Are clients sticking with resources (such as school, jobs, and staying connected to their case managers)? If not, why?
Long-term outcomes: Injury and criminal recidivism capture the qualitative value of the program.
Qualitative analysis is critical in understanding more of the inherent value of the programs not captured in the typical measures.
- Semi-structured interviews and evaluating for common themes can reveal value not captured by other measures.
Cost effectiveness analysis
Make everything count

Case Study

Robert joined the violence intervention program after being hospitalized for his second violent injury. When he recovered from his injury, his case manager, who had assessed his needs, accompanied him to mental health services for three months. Robert’s anxiety was improving and he felt ready for work. The case manager was able to help place him in a program in which Robert would learn how to be an arborist. This program paid a stipend and had the potential of landing him a permanent job. Robert stuck with the program and felt empowered by the skills he was learning. He was put in charge of teaching middle schoolers how to trim trees during a summer seminar. He was building confidence and feeling good about life. Six months after the beginning the program, Robert was on his front steps arriving home from work, when he was shot in the leg. Robert was treated for his injury and recovered enough to go back to the arborist program several months later.

Potential Pitfalls

Evaluation as an afterthought could lead to poor programmatic outcomes. For example, if the feasibility of the program is not assessed early on, the appropriate target population may not be adequately served. Poor enrollment of a target population occurs if programs do not regularly reassess registry data to determine if at-risk groups are exposed to HVIP. Programs need to be nimble enough to adapt to unforeseen population changes and changes in resources. This can only occur through data collection and evaluation.
Singular evaluation of long-term outcomes, such as recidivism, misses the nuanced value of these programs.
- The Step 6 Case Study brings up the painful fact that “secure” housing may not be safe housing. Is this recidivist event considered a failure of his violence prevention program? Not necessarily. While we may be unable to modify all risk factors in a client’s life, such as the surrounding community, the services provided to a client, such as mental health services and vocational training, should be taken into consideration when evaluating the outcomes of these public health programs. If only recidivism is tracked, the other components of value go unrecognized.