What Are the Basic Steps?

Conduct the Evaluation

The type of evaluation you plan to conduct will determine what you need to measure and subsequently the data you need to collect. Start your evaluation by—

• Reviewing your evaluation plan.
• Developing data collection protocols.
• Training program staff.

After you have developed your evaluation design and selected your method of data collection, the next step is to conduct the evaluation by collecting, analyzing, and interpreting the data.

• Collect Data
• Analyze and Interpret Data

Collect Data

To ensure that you collect the appropriate data, start with a data collection plan. This plan should outline your evaluation questions, the data you need to answer those questions, the data sources you expect to use (e.g., program staff, participants), your data collection methods (e.g., observation, interview) and instruments (e.g., surveys, interview guide), and your schedule (e.g., site visit schedule or survey administration dates). The plan also may include how you will enter, track, and store or secure data. The plan allows you to compare what you planned to do in your program with what you actually did and to monitor your overall progress.

Many types of data will be collected from various sources and will be driven by the key evaluation questions. For process evaluations, you may collect data on aspects of your program that relate to program activities such as—

• Program interventions.
• Characteristics of clients and program staff.
• Resources.
• Organizational structures.
• Standard operating procedures.
• Perceptions of program staff and participants about the program’s effectiveness and efficiency.
• Internal and external factors influencing the attainment of program goals.

For outcome/impact evaluations, you may collect data that measure intermediate outcomes, which may include the number of—

• Services provided to precertified victims.
• Law enforcement or social service professionals who received training.
• Collaborative partners within a region.

To get the best quality data, administer your instrument in the same way each time. When interviewing, for example, ask the questions in the same order and in the same manner. Other important suggestions for properly entering, tracking, and securing your data are listed below.

Data Entry. You will need to create a system for entering data. Your data entry system will allow you to know, at a glance, how many people have responded and when and any missing data not provided by the respondent. Before entering data from hardcopies (i.e., paper documents), be sure to make a copy of the form. You can make edits or comments on the copies. If the data are in electronic form, be sure to make a master backup copy on your hard drive or keep separate copies of your database on jumpdrives or CD–ROMs. Exhibit 10 shows a sample data entry sheet.

Exhibit 10 Sample Data Entry Sheet
Title of Instrument	Respondent Identification Number	Completion Date	Q. Have you assisted a trafficking victim in the past 6 months? (1 = Yes; 2 = No; 3 = Unsure)	Q. Have you received training on trafficking victims’ rights and laws? (1 = Yes; 2 = No; 3 = Unsure)
Law Enforcement Training Feedback Survey	061704	05/05/09	1	1

Data Tracking. It also is important to track the data you collect. There are several ways to do this. You can create a chart that includes the title of the instrument, the source of the data (e.g., document review, program participants), how the instrument was delivered, and the start and end dates of the data collection. It also might be a good idea to use computer software, such as Excel or Corel Quattro Pro. Exhibit 11 provides a sample data tracking sheet.

Exhibit 11 Sample Data Tracking Sheet
Title of Instrument	Source of Data	How Administered?	Data Collection Period	Number of Instruments Administered
Client Satisfaction Survey	Program participants	Self-administered	May 1–30, 2009	25

Data Storage and Security. You will also want to think about where and how you will store and secure the data you collect. Be sure to store hardcopy forms in a place safe from damage or loss. For electronic data, be sure to back up hard drives or keep separate copies of your database on an external drive or CD–ROM.

Analyze and Interpret Data

Analyzing the data you have collected should begin with a review of your research questions. This will help you organize your data and focus your analysis. Here are a few tips for analyzing and interpreting qualitative and quantitative data.

Qualitative Data. Qualitative data are typically obtained from open-ended questions, the answers to which are not limited by a set of choices or a scale. For example, qualitative data include answers to questions such as “What experiences have you had working with victims?” or “How can services to victims be improved in your area?”—but only if the study participant is not restricted by a preselected set of answers. You would typically ask these types of questions during interviews, focus groups, or as open-ended questions on a survey instrument. They yield responses that explain in detail the participant’s position, knowledge, or feelings about an issue. Analyze qualitative data to look for trends or patterns in the responses. These trends or patterns are the general statements that you can make about what you have learned about your community.

Below are some basic steps for analyzing qualitative data:

• Review all of the data.
• Organize and label responses into similar categories or themes. For example, all comments
  or concerns can be labeled “Suggestions,” and program activities can be labeled “Program
  inputs.”
• Try to identify patterns or associations among the data (e.g., all of the law enforcement officers who attended training have less than 1 year of experience).
• Look for similarities and differences among your respondents. This review will allow themes to emerge from the data and provide a basis for your coding scheme.
• Develop a coding scheme based on the data collected. For example, if you find that all of the law enforcement officers trained have less than 1 year of experience on the job, then you can code that response, “Less 1,” so that when you come to that response in subsequent reviews, you can easily categorize and code it.

Quantitative Data. These data are collected in surveys or through other means in the form of numbers and are usually presented as totals, percentages, and rates. For example, quantitative data include answers to questions such as “How many hours do you spend looking for service providers for your clients?” or “How many victims have you served this year?” You would typically ask these closed-ended questions in a survey, on which the participant circles a preset answer choice or provides a numeric response. Use quantitative data to generate averages or percentages across the responses. These averages or percentages tell you what proportion of your respondents feel a certain way or have a certain level of knowledge about an issue.

When embarking on the data analysis process, keep in mind the following questions:

• What do the raw data tell you?
• Are the results low, average, or high?
• Are there any red flags or extreme values?
• What can you infer from the data?

Depending on your skills as a qualitative or quantitative data analyst, you may want to hire a local evaluator or consultant. Some questions to ask when considering whether you need outside help include the following:

• Do you have enough experience analyzing qualitative and quantitative data to make sense of the data collected?
• Do you have sufficient time to thoroughly analyze the data?
• Do you have the funds to hire an evaluator?
• Are you able to use the data to answer the research questions in the most effective way?

The more questions that you, and those in your initiative, answer “no” to, the more advantageous it might be to hire a local evaluator or consultant, if you have the funds, to assist you with the data analysis. The Guide to Hiring a Local Evaluator, included within this series, can help with finding an evaluator.