In this edition
- Selecting the Right Data Collection Methods
- Seeing Clearly: Qualitative and Quantitative Data as Different Lenses
- When Qualitative Data Adds Context to Quantitative Findings
About Evaluation Matters
Evaluation Matters is a monthly newsletter published by University of Nevada, Reno Extension. It is designed to support Extension personnel and community partners in building practical skills for evaluating programs, making sense of data, and improving outcomes. Each issue focuses on a key concept or method in evaluation and provides clear explanations, examples, and tools that can be applied to real-world programs.
This issue, published in April 2025, focuses on selecting appropriate data collection methods for evaluation. It outlines how to match different tools—such as surveys, interviews, and focus groups—to specific evaluation questions, and explains the benefits of combining qualitative and quantitative approaches. Whether you're designing a new evaluation or refining an existing one, this issue provides clear guidance to help you collect data that is relevant, reliable, and useful.
Selecting the Right Data Collection Methods
How to align evaluation questions with effective tools
Choosing the right data collection method for your evaluation is a lot like selecting the right tool from a toolbox. You wouldn’t use a hammer to tighten a screw or a paintbrush to cut a piece of wood—each job requires a specific tool designed for a specific task. In the same way, each evaluation question calls for the right method to gather the clearest, most useful information.
It’s tempting to reach for familiar tools first, especially when time or resources are limited. Surveys, for example, are often the “go-to” choice because they’re relatively quick and easy to administer. But just like grabbing the first tool you see without thinking, defaulting to a survey without considering your specific evaluation needs can lead to incomplete or misleading results. A good evaluation starts by asking: What exactly am I trying to learn, and what’s the best way to uncover that information?
Choosing the right tool depends on your evaluation question.
Think about your evaluation questions like tasks on a project list. If you need to measure change over time, (like whether participants improved their skills after a workshop), a structured survey with pre/post comparisons might be the right “wrench” for the job. If you’re trying to understand participant experiences or unexpected barriers to success, you might need the “screwdriver” of interviews or the “level” of focus groups to get a balanced, detailed view. Matching your method to your question helps ensure the data you collect is both relevant and useful, providing the clarity needed to draw accurate conclusions and make informed decisions.
Audience matters, too. Different groups are more responsive to different methods, just as some tools work better with different materials. Youth participants, for example, might engage more with informal interviews or interactive activities rather than formal written surveys. Busy professionals might prefer quick online questionnaires over in-depth interviews. Keeping your audience in mind helps you select a method that not only gathers good information but also respects people’s time, attention, and comfort.
Resource constraints, like time, staffing, and budget, also play a role. If you have limited time to collect data or a small team to analyze it, trying to conduct a large number of in-depth interviews might not be practical. In that case, a few carefully designed surveys or a small number of focus groups could be more realistic and still provide valuable insights. The goal isn’t to use the most complex method possible, but to choose an approach that fits your context and still produces meaningful information.
Finally, remember that you don’t have to stick with only one method. Just like complex projects often require multiple tools, strong evaluations often benefit from using both quantitative and qualitative approaches. For example, you might start with a survey to identify broad trends and then conduct a few interviews to explore surprising results in more depth. Using the right combination of tools leads to a sturdier, more complete understanding of your program’s impact.
At the end of the day, evaluation is all about craftsmanship. The more thoughtfully you choose your methods, the better your data will serve your program. With the right tools and a clear plan, you can build an evaluation that is not only effective but also meaningful and lasting.
Seeing Clearly: Qualitative and Quantitative Data as Different Lenses
How numbers and narratives work together
At the heart of any evaluation lies a basic question: what kind of information do we need to make sense of this program? In most cases, the answer involves looking through different lenses: numbers and words, quantitative and qualitative data. These two types of information serve different but complementary purposes, and understanding when and how to use each can dramatically improve both the depth and clarity of your findings.
Quantitative data (e.g. numbers, counts, percentages, or ratings) are often used to track change over time, compare groups, or demonstrate scale. For example, you might use a survey with Likert scales to measure how confident participants feel about a topic before and after a training. These numbers are easy to summarize and analyze statistically, making them useful for spotting trends or reporting to stakeholders who want to see progress in measurable terms. However, numbers alone don’t always explain the underlying reasons behind your trends.
That’s where qualitative data comes in. Open-ended responses, interviews, focus groups, and observations allow participants to express their thoughts in their own words. This type of data reveals meaning, context, and nuance. For example, while your quantitative data might show that participants rated a training session lower than expected, qualitative comments might reveal that the room was too cold, the presentation moved too quickly, or the material didn’t feel relevant to their experience. These insights help explain why people respond the way they do, and what changes might improve the program.
That’s where qualitative data comes in. Open-ended responses, interviews, focus groups, and observations allow participants to express their thoughts in their own words. This type of data reveals meaning, context, and nuance. For example, while your quantitative data might show that participants rated a training session lower than expected, qualitative comments might reveal that the room was too cold, the presentation moved too quickly, or the material didn’t feel relevant to their experience. These insights help explain why people respond the way they do, and what changes might improve the program.
There’s often a temptation to view qualitative data as “extra” or less rigorous, but that’s a misconception. When collected and analyzed carefully, qualitative data can be just as valuable, and in some cases more revealing, than numbers. It gives people a chance to describe their experiences in their own terms, which is especially important when working with diverse communities or sensitive topics that don’t lend themselves well to checkboxes and scales. Narratives can surface concerns, opportunities, and unintended consequences that might otherwise be overlooked if only numbers were considered.
In practice, many evaluations benefit from using both types of data. Quantitative methods can give you the big picture, showing you what’s happening across a large group. Qualitative methods can fill in the details, helping you understand the subtleties and stories behind those patterns. For example, a program might report that 90% of participants completed a course, but follow-up interviews could reveal that many struggled with the pacing or felt the content wasn’t relevant to them... feedback you wouldn’t capture from numbers alone.
When deciding which type of data to collect, think first about the questions you’re trying to answer. If you need to measure change, compare outcomes, or demonstrate scale, quantitative data will likely be essential. If you’re trying to understand how people feel, why they act a certain way, or how a program fits into their lives, qualitative data is more appropriate. And when your evaluation needs both breadth and depth, combining methods can give you the most complete view of your program’s impact.
Sharp, meaningful evaluation relies on gathering information that captures both the measurable outcomes and the experiences behind them. Quantitative data helps define patterns, offering clear evidence of reach, change, or difference across groups. Qualitative data brings those patterns to life, offering context, interpretation, and insight into how and why those changes occur. Together, they sharpen your understanding, bring focus to your findings, and help you make decisions that are grounded in a full, nuanced view of your program’s work.
When Qualitative Data Adds Context to Quantitative Findings
Clarifying unexpected results and filling in gaps
Quantitative data can show that something happened, but it doesn’t always reveal the full image. Like trying to assemble a puzzle with a few key pieces missing, you might be able to see the general shape of what’s going on, but the picture isn’t always clear. Whether it’s a sudden spike in responses, an unexpected pattern between groups, or surprising year-over-year changes, numbers often leave you wondering what’s behind them. This is where qualitative data helps complete the picture.
Let’s say a survey reveals that participants overwhelmingly selected the same response on a key item—far more than expected. The number stands out, but the ‘why’ is unclear. Were participants genuinely aligned in their opinion, or did something about the question itself influence their choice? By following up with interviews or focus groups, evaluators can uncover whether the wording was confusing, the context changed, or the topic triggered a shared concern. These qualitative insights act as clarifying pieces, helping evaluators understand what the data is really showing.
Qualitative data helps reveal unexpected trends.
Qualitative data also plays an important role when findings seem inconsistent. A program might report strong attendance and high satisfaction, but show little change in outcomes. Or the reverse might happen: the data shows impact, but satisfaction is low. These contradictions can feel like pieces from different puzzle boxes! When participants are given the opportunity to explain their experience in their own words, evaluators can begin to see how these conflicting results make sense in context. Maybe participants enjoyed the sessions but struggled to apply what they learned. Maybe outcomes improved despite a difficult delivery method. These details help bridge the gaps between what the numbers say and what they mean.
In some cases, qualitative data reveals missing categories or flawed assumptions in the survey itself. A well-intentioned question might offer answer options that don’t reflect how participants actually think or feel. A question might be misinterpreted, or a key issue might be left out entirely. When evaluators review open-ended responses or conduct interviews, they often find participants raising points the original survey didn’t account for. These discoveries don’t just add context... they can directly improve future evaluations by making instruments more accurate and inclusive.
Timing matters, too. Sometimes qualitative data is collected after the fact, to make sense of confusing or unexpected results. However, it can also be used in advance, to shape survey design and prevent common pitfalls. Talking with participants beforehand can help ensure that questions are framed clearly and that the issues being measured actually reflect what matters to the people involved. In this way, qualitative data doesn’t just clarify, it strengthens every stage of the evaluation process.
A good evaluation doesn’t stop with numbers. It takes the extra step of asking what those numbers mean, and whether they’re telling the whole story. When qualitative data is brought in to fill in the gaps, it brings shape and clarity to results that would otherwise remain incomplete. Just like the last few puzzle pieces that make the picture come into focus, context transforms isolated data points into a more accurate, coherent view of what’s really happening. Numbers help you see the structure, but stories and perspectives help you make sense of what you’re seeing. When qualitative data fills in the missing pieces, you can move beyond surface-level results and build a more accurate, more complete view of your program.
