Employee Toolbox : Evaluating Workplace Programs

Evaluation Plan

Adapted from Take Action! California (www.takeactionca.com)

Registration

At registration, you should collect basic information about the participants. This information identifies the participant’s level of interest in being physically active, and these answers serve two purposes. First, they can give you baseline information so you can tailor programs to their level of interest. Second, by asking the stages of change question on the Registration Form at the beginning of the program as well as on the evaluation form at the end of the program, the Program Coordinator will be able to determine if participants moved to a higher stage over the time period that the program was implemented. An analysis of these answers will help to determine the effectiveness of the program. You can also ask participants to set individual goals.

During the Program

During the program, you should collect information about the workshops and activities you offer. You should track basic participation information such as how many people attended sessions, joined a challenge program or walked to or during work. If your program includes classes or activities, you should also ask participants to evaluate the sessions by giving them short surveys to fill out. Ask them how useful the program was, as well as questions about whether the time of day and the length of the program were convenient for them. This information can help you plan future programs.

Final Evaluation

At the end of the program, ask participants the staging question again. Their answers will allow you to see if people moved from one stage of change to the next. Movement to higher stages indicates that the program was successful. You should also determine if people reached their final goals. Other questions to ask include:

• How satisfied were they with the program?
• What did they like about the program?
• What did they dislike about the program?
• What helped them to reach their goals?
• What got in the way of reaching their goals?
• How can the program be improved?

Data Collection

The Planning Committee should decide early on how it will collect data from the participants in hard copy or on-line. If the company is not very big, participants can submit their information directly to the Program Coordinator. In a large company, the participants can submit their information to Team Captains. The Team Captains can then provide a summary to the coordinator. Whatever system is developed, it should minimize the paperwork for participants, Team Captains, and committee members. Too many evaluation questions or forms can be a barrier to participation.

We Collected Data. Now What?

Evaluation has three important functions. You can use the results of the evaluation to:

• Make improvements to the program
• Promote future programs
• Increase management support (Identify early what they value.)

A Note about Forms

Take Action! California provides a kit on its website (www.takeactionca.com) with forms that can be used or adapted to evaluate programs. If you decide to develop your own forms, keep these points in mind:

• Keep your questions and the forms short and simple.
• Avoid asking questions that will not give you useful information.
• Make sure directions are clear and easy to understand.
• Keep responses confidential, and make sure employees know that the responses will be kept confidential.
• Do not ask participants about program features that you cannot deliver. For example, do not ask participants if they would be more active if the office had a swimming pool, unless it is feasible to offer one at your worksite or contract with a local recreation center.

As with any program that is costing company resources, measures of effectiveness are important to determining and shaping the future of the program. Conducting any evaluation of a program - judging the satisfaction of participants, the number of classes held, or the impact - is better than no evaluation. Program managers that do not assess their methods, direction or potential impact and outcomes may have a limited understanding of their program and may lack the data to justify future program funding. Most disturbing, they may not be making any impact on their community. This type of self-examination will help ensure that programs, new paths, trails and parks are both promoted and maintained.

In addition to measuring the output and impact evaluations following a program, it is important to conduct process evaluations throughout the project cycle. Process measures include evaluating the way that the program was organized, initial participation, and internal buy-in from management.

For workplace wellness programs, the following evaluations are recommended:

Metric	Means of Measurement
Participation/Attrition	Sign-in sheets for each session Enrollment against goals Completion rates Staff awareness rate Barriers to participation
Program Completion	Percent of program agenda completed Percent of participants who completed each session and total · List of issues that have arisen and resolution
Diet/activity changes, including: Body Mass Index (HP2010) Waist-to-hip ratio Resting heart rate/cardiorespiratory fitness Blood pressure or actions taken to combat high blood pressure Amount of activity per day, frequency per week Flexibility Muscular strength and endurance	Logs or self-report survey before and after intervention at 1 month, 3 months, and 6 months
To determine scientific significance of the program, compare outcomes of a group of participants to a control group of non-participants. Look for changes in: Activity Pattern Health of Employees Health Costs/Return on Investment Productivity	Formal Research Host focus groups
Budgetary issues	Track actual vs. expected expenses Overall Return on Investment