Qualitative Data Analysis: How to Turn Open-Ended Answers Into Themes

Qualitative data analysis is the process of turning open-ended answers, such as interview transcripts or free-text survey responses, into a small set of themes that explain what people think and why. In practice that means reading the answers, coding each one by what it says, grouping the codes into themes, and reporting how often each theme appears with quotes that show what it means.

This guide walks through the steps we use, with examples from our Super Bowl ad testing study, and covers how to check AI-assisted analysis before you trust it.

What thematic analysis involves

Thematic analysis is the most common approach to qualitative data analysis in market research. You read through the data, label each meaningful idea with a short code, then group related codes into themes. A theme is a pattern that recurs across participants and answers a research question.

In deductive coding you start with a codebook built from your hypotheses. In inductive coding you let the codes come from the data. Most commercial projects use both, with a starter codebook plus room for new codes when participants say something unexpected.

In our brand tracking buyers study, the argument for coding after collection rather than before is that nothing is predefined, so the themes that matter surface on their own instead of being limited to the answer options someone thought to write.

How to analyze open-ended responses step by step

1. Read a sample before coding anything. Read a few dozen full answers end to end. You will see the main ideas and the language people use for them.

2. Draft a codebook. Write each code as a short label with a one-line definition and an example quote. Keep codes specific enough that two people would apply them the same way.

3. Code every answer. One answer can carry several codes. A viewer who says an ad was funny and had a surprise ending gets both.

4. Group codes into themes. Merge duplicates and split any catch-all code.

5. Count and compare. Report the share of participants who mentioned each theme, then compare across segments.

6. Pick quotes last. Choose quotes that represent the theme fairly.

Counting themes and why they add to more than 100%

Counting makes qualitative findings usable in a deck. A theme mentioned by most participants carries more weight than one mentioned by three people.

When the Super Bowl study asked what made an ad stick, 59% of viewers said it was funny, 41% said the brand was part of the idea or setting, and 36% said it had a twist or surprise. As the study notes, viewers could name more than one reason, so the shares add up to more than 100%. That is normal for coded open answers, and you should say so on the chart so nobody tries to read it as a pie.

Overlap is a finding too. In the same study, 78% said a well-cast star makes an ad more memorable and 63% said this year's game had too many celebrities. Because those two shares add up to well over 100%, at least 21 of the 51 viewers felt both ways at once. Reporting that tension is more useful than picking one side.

Two more counting rules. Always state the base, because not every participant answers every question; the Super Bowl study reports 56 answering recall questions and 51 answering the celebrity and AI questions. And count participants, not mentions, or one talkative person can inflate a theme.

Keep every quote linked to the participant

A quote nobody can trace loses stakeholder trust fast. Keep each coded excerpt tied to a participant ID, their segment and the question it answered, so you can filter by segment and check any quote in context.

Traceability also lets you connect themes within one person. In the Super Bowl study, the follow-ups are where viewers who loved a funny ad admitted they couldn't name the brand. Some could describe a funny spot in detail and still not say who it was for. You only see that pattern if each viewer's humor code and brand recall answer stay attached to the same participant.

Common pitfalls in qualitative data analysis

• Coding to confirm. If your codebook only contains the hypotheses you walked in with, you will find them. Leave room for an "other" pile and read it.

• Themes that are topics. "Price" is a topic. "Shoppers doubt quality when a brand leads with price" is a theme. Write themes as statements.

• Overweighting vivid quotes. Check how many participants said something similar to that one memorable line.

• Ignoring the negative. In the Super Bowl study, 51% left the game with a better view of at least one brand, but 32% left with a worse view of one. A summary that only coded positive reactions would miss the backlash.

Qualitative data analysis software and AI-assisted coding

Qualitative data analysis software ranges from tools that help a researcher tag transcripts by hand to AI systems that code thousands of answers automatically. Manual tagging keeps you close to the data but is slow; the brand tracker study describes a thousand free-text responses per wave as a coding project. AI coding is fast enough that open questions no longer have to be cut for cost.

Researchers are open to this but cautious. In our study of 76 market researchers, 91% were open to AI in their workflow, but only 29% trust AI-generated insights as much as traditional research. Among those who described what AI would have to do to earn their trust, 65% said it must prove it's accurate and 52% want a human checking the output. Data cleaning and organization was the single step most often named as most manual (26%).

How to check AI-assisted analysis before you rely on it:

• Run it on a past study. Compare the AI's themes with the ones your team produced by hand. The market researchers study recommends this as a test before rolling out any AI research tool.

• Spot-check the codes. Read a random sample of answers under each theme and see whether they belong there.

• Demand a source trail. Every theme should link to the quotes behind it so a person can verify it.

• Keep interpretation with the researcher. Let AI code and count. Decide what it means yourself.

On Listen, the AI moderator follows up on each answer during the interview and then codes the open-ended responses into themes, with every theme linked back to the participant's clip so a researcher can review it. For how to design the questions that produce codable answers, see our qualitative interview guide, and for applying this to ad research, our article on creative testing.

FAQ

What is qualitative data analysis?

Qualitative data analysis is the process of turning open-ended data, such as interview transcripts or free-text answers, into themes that explain what people think and why. It usually involves coding answers, grouping codes into themes, counting how often each appears, and supporting each theme with quotes.

What is the difference between coding and thematic analysis?

Coding is labeling individual answers with short descriptive tags. Thematic analysis is the broader method that groups those codes into recurring themes that answer the research question.

How do you analyze open-ended survey responses?

Read a sample first, build a codebook with clear definitions, code every response (allowing more than one code per answer), group codes into themes, and report the share of respondents per theme with a stated base. Keep each quote linked to the respondent so findings can be checked.

Why do theme percentages add up to more than 100%?

A single participant can mention several ideas in one answer, so they count under several themes. Each percentage is the share of participants who mentioned that theme, and charts should say so.

Can AI do qualitative data analysis accurately?

AI can code and count open-ended answers quickly, but accuracy needs checking. Test it against human-coded results from a past study, spot-check samples under each theme, and use tools that link every theme back to the original quotes so a researcher can verify it.