Natalie Schock, PhD
Evaluation Services
most evaluations just tell you "what" ...
Most program evaluations focus on a few key numbers, such as satisfaction scores. These data are valuable, but tend to be one-dimensional. For instance, learning that 80% of participants strongly agreed or agreed that your session was valuable doesn't tell you why, or what could have gone better, or how to make sure you keep the good parts for next time.
For organizations who want to understand how and why –– who are using an evaluation to truly get better –– snapshot data aren't enough.
... mine also tell you how, why, and what to do about it
I specialize in mixed methods evaluations, meaning I use quantitative (numbers) and qualitative (transcripts, observations) data. A typical mixed methods evaluation might consist of a survey and interviews. The survey data captures closed questions, like if people would recommend a service on a scale of 1 to 10. The interview data captures why, the details of participants' experiences, the context, and—almost always—awesome quotes that you can use in grant proposals, annual reports, websites, newsletters, and more. Combining hard numbers with rich context makes for an evaluation that is more thorough, actionable, and clear. (Note: This method is best for clients seeking to truly measure impact and improve. It is not a good tool for those looking for a check-a-box, off-the-shelf evaluation.)
My Approach
-
Survey design: Respect the user by following Gricean maxims. Priorities are efficiency, clarity, and asking the right questions. I don't push surveys to do the work of interviews.
-
Interviews and focus groups: Interview protocols have core questions and comprehensive probes so you get the information (and quotes) you need. Respondents often say my interviews are respectful and spur high-value reflection, while clients get robust, powerful insights. If there were awards for listening, I would win.
-
Integration: Tailored to your needs. Sometimes qualitative and quantitative data work together to form one rich picture. Sometimes the qualitative data explain the quantitative. Sometimes the qualitative pick up something interesting and the quantitative measures it.
-
Work culture: I prioritize thorough planning, extensive communication, and extreme transparency. ​​