How Do Companies Test Advertisements? Methods and When to Use Each

Pre-testing evaluates an advertisement before it launches, while post-testing measures its effect afterwards. The two use different methods because they answer different questions.

The split also follows the difference between qualitative and quantitative research. Before launch you are usually asking why something works or confuses, so qualitative methods lead. After launch you are measuring, so quantitative methods lead. Specialist methods are mostly used before launch, for narrow questions.

In our view, pre-testing is where changes are easiest to make, because the creative is not yet running. Copy testing is the trade term for evaluating an ad’s wording and execution with a sample of its audience, usually before it runs.

Qualitative ad testing shows a rough or finished ad to a small number of the intended viewers and asks them to explain, in their own words, what they understood, felt and would do. Three formats are used: in-depth interviews (IDIs), focus group discussions (FGDs) and online bulletin boards.

In-depth interviews put one person with a moderator. They suit questions of comprehension and belief, because nobody else in the room influences the answer. We run interviews of 30 to 120 minutes, online or in person. Our in-depth interview method guide covers how to plan and run one.

Focus group discussions put a moderated group together, and our sessions run 1 to 3 hours, online or in person. They suit tone, humour and social acceptability, because you see how reactions build or clash between people. The risk is that talkative participants can dominate unless the moderator manages the room, as the Nielsen Norman Group's guide to focus groups notes. Our focus group discussion guide shows how to run one.

Online bulletin boards are asynchronous: participants respond in their own time over several days. They suit reaction testing where people need to revisit the ad, or where participants are spread across cities. Our online bulletin board guide explains how to run one.

Testing the underlying idea before any ad exists is a separate stage, covered in concept testing. For how an ad-reaction interview is set up in India, see our India method for advertising pre-testing.

A small group is not a representative sample, so the findings tell you which reasons exist and how they sound, not how many people share them. Liking an ad is also not the same as the ad working: people can enjoy a film and forget the brand, or dislike one that persuades. Use qualitative reactions as grounds to change a line or a scene, not as a forecast of results.

Pre-test surveys show an ad to a larger audience and score it on four things: recall, message clarity, brand linkage and purchase intent. The result is a comparable set of scores across creative routes, which is what you need when choosing between near-final options.

• Recall: whether people remember seeing the ad and the brand afterwards.
• Message clarity: whether they take away the point you intended.
• Brand linkage: whether they connect the ad to the right brand, rather than a competitor or none.
• Purchase intent: how likely they say they are to buy.

Treat purchase intent with care. It records what people report, and what people say and what they do often differ, as the Nielsen Norman Group's note on attitudinal and behavioural research explains. It is useful for comparing routes with each other, but it should not be read as a sales forecast. For the method steps, see our guide to advertising pre-testing in India.

Specialist methods measure responses viewers cannot easily put into words, and they suit narrow questions about specific moments in a film. Two kinds are common: theatre tests and physiological measures.

A theatre test records an audience's reactions as the ad plays. Physiological measures, such as eye tracking, galvanic skin response (GSR) and electroencephalography (EEG), record the body's response. They show that something happened at a given second, not why. Pair them with a qualitative method when the why matters.

After launch, companies compare live versions with A/B tests and measure the effect on awareness and recall with brand-lift studies and recall tracking. Each tells you what happened to a live ad, not why it happened.

• A/B test: two versions of an ad, differing in one thing, are shown to comparable audiences and compared on a response such as clicks or sign-ups.
• Brand lift: people who saw the ad are compared with a similar group who did not, using a short survey. The method rests on randomised experiments with exposed and control groups, as a Google Research paper on brand-lift measurement describes. Google Ads Help defines Brand Lift as measuring the effect of campaigns on perception, including ad recall, brand awareness and purchase intent.
• Recall tracking: the same recall questions asked repeatedly over time, compared with a baseline before launch.

When a live ad underperforms, the numbers show that it did, not why. A qualitative round on the live ad, with the people it was meant for, can show the reason.

Five things get tested in an ad: the message, the visuals, the tone, the language and the cultural fit. Each is a separate question, and different methods suit each.

• Message: what the viewer takes away, and whether it is the intended point.
• Visuals: what they notice first, whether the product and brand are visible, and which scene confuses.
• Tone: whether humour, urgency or seriousness suits the brand and the product, and whether it feels believable.
• Language: whether the words, register and accent sound like the audience's own speech.
• Cultural fit: whether references, family roles, festivals or gestures feel familiar, foreign or offensive.

Score message and brand linkage in a survey. Explore tone, language and cultural fit in conversation, because they depend on how people react, not only how many do.

Ask "why" or "what did they understand" and use a qualitative method. Ask "which" or "how many" and use a quantitative one. The table maps six common decisions to the method that informs them.

For how to choose between the two methods, see our guide on qualitative vs quantitative research.

Four recruitment choices decide whether an ad test in India means anything: the language of the session, the cities, the household and the socio-economic class of participants. Specify each in the brief, because a group that does not resemble the people who will see the ad cannot tell you how they will react.

Language. Test each language version of an ad with people who speak that language, rather than running one English session about a Hindi or Tamil ad. Humour, idiom and tone of voice rarely carry over word for word. Decide the language of the ad and the language of the session separately, and write both down.

City. Reactions depend on reference points and media habits that differ between metros and smaller towns. Choose cities by where the ad will run, and do not assume a result from one city holds in another.

Household. Ads are watched and discussed in households, so decide who in the household you need: the buyer, the user, or the person who influences the decision. Exclude people who work in advertising, media or research, or whose household members do, since they react as professionals.

Class. Two classification schemes are in circulation, so a brief that says only “SEC” can mean either. The Market Research Society of India (MRSI) announced the Indian Socio-Economic Classification (ISEC) in February 2024 to replace the older New Consumer Classification System (NCCS), as Research Live and Press Insider report.

ISEC uses the occupation of the chief wage earner and the education of the highest-educated male and female adults in the household. Name the scheme in your brief, and see our guide to NCCS classification and recruiting by class for how recruiters grade a household. The class belongs to the household, not the person interviewed.

Prepare five things before a qualitative ad test: audience, city, language, stimuli (the material shown to participants, such as a storyboard or finished film) and objectives. Each should be specific enough to check, as in the pass and fail tests below.

• Audience. Pass: you can write who is in and who is out in two lines, including category use. Fail: "young urban adults".
• City. Pass: each city is named, with a reason tied to where the ad will run. Fail: "metros".
• Language. Pass: the session language and the ad language are both written for each group. Fail: "Hindi and English" with no pairing.
• Stimuli. Pass: you state whether you will show a rough storyboard or a finished cut, the file is ready, and confidentiality terms are written. Fail: "we will send something". A rough storyboard tests the idea and message; a finished cut also tests pacing, music and casting.
• Objectives. Pass: three to five questions, each tied to a decision the result will change. Fail: "get feedback".

Our one-page market research brief template shows what to put in a brief. If you already have these five answers, share the brief with us and we will review it.

When we receive a brief, we review the objective, method, audience, geography and recruitment criteria, assess feasibility and send a tailored proposal. Recruitment and fieldwork begin once scope and timelines are agreed. Our work is qualitative research; surveys and physiological testing are outside the formats we offer.

Companies test advertisements at two moments, and the method should follow the decision; the result will change. Before launch, interviews, groups and bulletin boards explain why an ad works or confuses, and pre-test surveys score near-final routes. After launch, A/B tests, brand lift and recall tracking show what the live ad achieved. Specialist methods answer narrow questions about single moments in a film.

In India, settle the language, city, household, and class of participants before recruitment starts, and have the five checklist items ready. A clear brief also makes it easier for any agency to scope the work. For the method steps behind an ad-reaction interview, see our guide to advertising pre-testing in India.

If you have chosen a qualitative method, share your research brief with Qualfacto at support@qualfacto.io.