Concept Testing: How Many People You Need and How to Decide

Concept testing asks whether a product idea deserves to exist, using surveys and sometimes interviews or groups before launch. Usability testing asks whether people can use something that already exists. A concept is the idea in words, a sketch or a rough render, without brand polish.

Advertising, brand and packaging tests are different jobs: a concept test judges the idea, not its execution. Test marketing, which sells the finished product in a limited area, comes after. For screens and prototypes, the concept stage sits before prototype user testing.

Match the method to the question. Qualitative methods answer "why" and "how do we fix it". Quantitative methods answer "how many" and "how much", as the Nielsen Norman Group's guide to research methods sets out.

Our recommendation is to run a small qualitative round first. It removes confusing wording and weak ideas, so the larger survey tests concepts worth measuring.

For a single score you need about 150 people per concept. To compare two concepts you need more: around a 40% baseline score, 173 per concept detects a 15-point gap and 388 detects a 10-point gap. The figures below are our model, not an official standard.

Margin of error for one score. Formula: 1.96 × √(0.5 × 0.5 ÷ n), at 95% confidence, with the worst case of a 50% score.

Respondents per concept to tell two concepts apart (approximate, rounded up). Inputs: 95% confidence, 80% power (an 80% chance of detecting a real gap of this size), two-sided test, scores measured as top-2-box (the share choosing the top two answers, such as "definitely" or "probably would buy").

Worked example: four concepts for a new beverage flavour, each expected near 40%, and you want to detect a 10-point lead. That is 4 × 388 = 1,552 respondents. Testing two concepts needs 2 × 388 = 776. Using 200 per concept gives 800 in total, but it only detects gaps of about 14 points or more at that baseline.

Treat these as minimums. Four concepts mean six pairwise comparisons (4 × 3 ÷ 2), and every extra comparison raises the chance that one gap looks real by luck. Real panels also add weighting (adjusting results so the sample matches the target population) and drop-outs.

Treat a gap as a tie unless it exceeds the margin of error of the difference, which is about 1.4 times the single-score margin: about ±7 points at 388 per concept. The formula is 1.96 × √(p1(1−p1)/n + p2(1−p2)/n), where p1 and p2 are the two scores. Where the gap is smaller, decide on the qualitative reasons.

For qualitative work, count audience segments, not scores. The Nielsen Norman Group describes focus groups as typically 3 to 12 participants. Our recommendation is 6 to 8 per group, and at least two groups per audience segment, so one unusual group cannot mislead you.

For IDIs, interview 6 to 8 people per segment and stop when new interviews repeat what you have already heard. Never report percentages from groups this size; report patterns and reasons.

Segments multiply quickly. Two audiences (current buyers and non-buyers), two city types (metros, and Tier-2 cities, India's mid-sized urban markets) and two groups each is 2 × 2 × 2 = 8 groups. Our guide to running focus groups in India covers moderator, language and venue choices.

Cost scales with the number of cells (each concept, audience and city combination you test), not with the number of questions you ask. We do not work from a fixed rate card; we quote each project against the brief. The factors are the method, target audience, participant profile, sample size, geography, recruitment difficulty and overall scope.

Two practical rules follow:

• Cut concepts before cutting respondents. Halving four concepts to two halves the sample in the example above (1,552 to 776).
• Budget each group or interview as a line. Recruitment, incentive, venue or platform, moderator, translation and analysis. Ask any supplier to quote those lines separately so you can compare like with like.

No. Stated interest is not a purchase. Attitudinal research captures what people believe and report, which is limited by what they are aware of and willing to say, while behavioural methods observe what they do.

Our recommendation is to use scores to rank concepts, not to forecast sales. For products people use, follow the survey with an in-home use test, where a small group tries the real product at home and reports back over days. Include a comprehension check in every test: a person cannot want what they have misunderstood.

Write the rules before fieldwork, using this sheet. Fill the benchmark column from your own past launches.

Concept card checks (each is pass or fail)

1. One idea per card. Fail if the card needs "and" to describe what it does.
2. Price shown. Fail if a respondent could not tell what it costs.
3. No brand imagery or slogans. Fail if the card reads as an advert.
4. Restatement test. Pass if at least 8 of 10 pilot participants can say the idea back in one sentence without looking (our recommended threshold).
5. Language test. Pass if two native speakers who have not seen the English card translate the regional-language card back into English (back-translation) and the meaning matches the original on price, benefit and who it is for. Fail if either reading differs.

Our recommended decision rules:

• Go: comprehension, need and purchase interest all meet the benchmark.
• Revise: comprehension fails, purchase interest is within the margin of error of the benchmark, purchase interest passes in one segment only, or need is below in some segments. A concept people misunderstand tells you about the card, not the idea.
• Kill: comprehension passes and need is below in all segments, or purchase interest is clearly below the benchmark and the qualitative reasons are fundamental objections.

We are a qualitative research participant panel in India, so we help with the recruitment and fieldwork behind your concept test. You share your brief, and we review the objective, method, audience, geography, sample, timeline and recruitment criteria. We check feasibility and send a tailored proposal and quote.

Once the project is agreed, we recruit and screen participants, schedule interviews, group discussions or product tests, and handle participant incentives, which we pay study by study after validation. We recruit across metros, Tier 1 and Tier 2 cities and urban, semi-urban and rural areas, with multilingual fieldwork assessed for each project. For first projects we are open to a small pilot or introductory arrangement, set by the scope of your study.

To see how we screen people, read our participant panel quality checklist, or see how to recruit participants for a study in India. Share your brief with our team and we will tell you which method and sample fit your concepts.