“Anything that won’t sell, I don’t want to invent.’’ – Thomas Edison

That idea of Thomas Edison’s is shared by most companies today. It’s the principle behind new product concept testing – trying to evaluate the potential for a new product before it is fully developed, contracted for, stocked or has any ad funds invested in.

Unfortunately, creating new ideas and turning them into successful new products is a difficult, high-risk process. Tens of thousands of new products are introduced each year. They represent the best of the ideas generated by consumer product companies for their offline and online markets.

Yet most of those tens of thousands products fail and are gone from the retail or virtual shelves within a few months.

Most performance marketers are familiar with wet testing, dry testing and the lean startup method when it comes to testing new product ideas. But…

… This is Not Another Lean Startup or “Create a Landing Page With a Dummy Product” Guide

While dry/wet testing or lean startup methods let you test your ideas for your online ventures, they won’t help you much if your idea bank consists of hundreds or thousands of new product concepts.

It’s usually not viable to dry or wet test hundreds or thousands of new product ideas, especially if you’re a huge e-commerce venture or affiliate business which require you to come up with new products on a regular basis.

Even if you’re into drop-shipping (and have no product development or stocking cost) you’d still have to pay for the traffic to have a multitude of wet or dry tests done.

That’s where new product concept testing methodology comes in: to sift through all the new product ideas you have, identify the likely losers and stick with the likely winners to properly test them afterwards with dry or wet testing or the lean methods.

This is a “Long Lost” Secret of the Direct Response Marketers Behind the 20th Century Success of Reader’s Digest

Reader’s Digest people called it “the Q test” and in their golden years it was responsible for selecting thousands upon thousands of winning new products for their sweepstakes and catalog direct mail campaigns.

As a matter of fact, a very similar methodology was employed by consumer packaged goods giants like Unilever or Procter&Gamble to successfully introduce new products to the retail market.

And now it’s been updated for the 2020 and beyond.

We all know that people usually are attracted by the idea behind a product before they become interested in the real product. They want a more convenient dessert, more beautiful hair, or cleaner floors – products are merely “vehicles” for delivering those benefits (or jobs-to-be-done).

Because the idea is the most critical element in a product, consumers can react to the new product idea without seeing the real product – they can react before the real product even exists.

If the idea of the product stimulates little interest among potential customers, it’s unlikely the real product will do much better (but still possible, especially if the concept is complex and requires the prospect to be educated about the value proposition, more on that later).

On the other hand, if the idea does have appeal, the product has potential if it can deliver the benefits promised by the concept.

The Purpose of New Product Concept Testing

When concept testing is done at the very early stages of the new product development process, it can have several important benefits:

• It can identify “winners” and “losers” at an early stage of development, while the concepts are no more than beginning ideas.
• This allows your company to set priorities and focus its development efforts (including all the lean startup, prototyping and dry/wet testing tactics) on the concepts that have the highest chances of becoming successful.
• If screening is done at an early stage, concepts can be evaluated before significant resources are committed to their development. In this way, it can help make optimum use of your R&D funds, marketing budgets, and management time.
• Finally, developing a low-risk, low-cost screening system encourages the generation of concept ideas as this methodology gives pretty much instant feedback to you and your employees on the quality of the product ideas. Since any new product effort is dependent on the input of many new ideas, this stimulation can be very healthy for the development of new products.

Concept testing has been proven to work well for screening most kinds of products. The exception is products that are radically innovative and would require people to significantly alter their behavior.

For example, it probably would have been difficult to adequately measure, before introduction, the true potential impact of iPad, Kindle, Facebook, or now obvious TV and instant coffee. These dramatic new products diffused steadily through the population, and people are poor predictors of their long-term behavior with regard to products that require substantial commitment and behavior change.

Contrary to the popular belief, these dramatic, innovative new products are actually extremely rare. The kinds of ideas which most companies spend most of their time screening, and you should too, are very testable with concept-screening techniques.

Since a primary purpose of new product concept testing is to sort beginning product ideas into “winners” and “losers,” it’s important that you test early, before substantial money is spent on development.

If concept testing is used only as a “disaster check” before large-scale quantitative research or test marketing, a great portion of the potential savings from concept screening has already been lost.

In addition, there is a higher emotional commitment to ideas that are further along in their development than to beginning ideas. It is easier to kill a weak concept while it’s still at the idea stage than when it’s about to go into test market.

Typical Results of New Product Concept Testing

Here’s a real-life illustration of why screening many concepts early in the new-product-development process is critical. To put it simply, most new product ideas are bad ideas. They generate very little interest from consumers. Fortunately, that’s easy to tell from the first test of a concept, which is why concept testing works.

The table below shows the range of buying-intent scores on a typical batch of 20 new product ideas I recently tested. As can be seen from the figure, only 12 percent of the new product concepts tested stimulated one-fourth or more of the consumers surveyed to say they’d “definitely buy” the product described.

Percentage of consumers saying “Definitely Buy”	Percentage of consumers
Over 30%	2%
25-29%	10%
20-24%	28%
15-19%	31%
10-14%	22%
Under 10%	7%

As a rule of thumb, a your product should receive a “definitely buy” score of at least 20 percent to 25 percent at the concept stage. These all too typical results show that a large proportion of all concepts tested don’t even generate enough initial interest to justify taking them to the next step of development.

(While rules of thumb may be great, you should establish your own “definitely buy” benchmarks by screening your proven, already-selling products as concepts.)

Step 1: Showing the New Product Concept

Fortunately, product descriptions used in testing need not be elaborate and expensive. In fact, simpler concept boards seem to work better in the long run.

A fancy ad may inflate reactions to the product idea in the test. And if elaborate concept boards are used (including video, for example), production of the concept boards can become so expensive that it’s impractical to test large numbers of new product ideas.

Only two elements are critical for inclusion in the concept statement:

1. Copy describing the product, how it works, and its benefits. The copy should not be full of exaggeration, and hard sell, but neither should it be too bland. A good guideline for preparing the copy is to try to write it as all your other promotional copy – use the style that is already proven to work with your prospects. Make it interesting and loaded with benefits, but stick to the facts. Make sure you present the value proposition properly.
2. Some type of illustration, if only a simple photo or infographic. This seems to help consumers in the test grasp the new product concept more quickly, and it makes the copy seem more interesting. For most products, simple sketches seem to be quite adequate. Video can be very expensive and does not seem necessary for most categories. The exception may be certain fashion or food products for which appearance, or “eye appeal,” is expected to be a major factor in the product’s success. Most concept boards look something like this:

Is it necessary to include price in the concept? Probably not, unless the price of the product will be much higher or lower than consumers would expect to pay for other products of the same type.

If price is different from what consumers would expect, of course, it becomes an element of the concept that could increase or decrease appeal, and therefore should be mentioned.

If price isn’t given, it’s usually referred to by a phrase such as: “assuming this product were available at a price you considered reasonable.”

Should the name of the brand be included? Yes, but only if it adds real value to the concept. If the brand is pretty much unknown, there’s no reason to include it.

Step 2: Preparing the New Product Concept Survey

While there are a number of different ways to conduct concept screening (intercept interviews at malls, phone interviews, direct mail surveys, interviews after the sales call etc.) I’d recommend just the one: online surveys targeted at your current customers and qualified prospects.

It’s all about a few steps:

1. Conduct the email, SMS or Facebook (targeted) campaign and direct all users to a landing page with your survey.
2. Divide your current customer and prospect database into a several segments. This provides segment representation where variations in concept reactions would be expected in different parts of your audience.
3. Show each respondent no more than 10 concepts – or perhaps 20 if the questioning about each is very short. This is important. My experience indicates that when consumers are given more concepts to evaluate, wear-out sets in and the quality of information deteriorates (that’s just a rule of thumb, make sure you test the number of concepts variable on your own).
4. Rotate the order of concept presentation, of course, so that each concept appears an equal number of times in each position during the survey. Your web developer should help, just tell him or her to have the order of concepts randomized each time someone responds. Don’t ignore this step as the order of concepts may influence the responses.
5. Avoid showing the same consumers extremely similar concepts. It’s usually better to mix up the types of concepts so that respondents are exposed to a variety of product ideas.
6. Include your currently selling product you know should do well in the survey. This will be your benchmark and a control to make sure the concept screening works properly.

Step 3: Crafting Questions to Ask During New Product Concept Screening Survey

Without doubt, the most critical question in any concept test is the one that measures buying intent. What you really want to know is: how many people will buy this product?

The most widely used question for evaluating buying interest is this one: “Which of these statements best describes how you feel about buying this product?”

• I definitely would buy it.
• I probably would buy it,
• I might or might not buy it.
• I probably would not buy it.
• I definitely would not buy it.

All the other questions that may be asked about a new product idea are for diagnostic purposes – to help explain why a concept has strong or weak appeal or to create a more complex decision models described later.

These additional questions include:

1. A follow-up, open-ended question about why the respondent is or is not interested in buying the new product. Typical wording: “Why do you say that?” Alternatively, focus the answers more by asking a two-part question:
   (a) “What’s your main reason for saying that?”
   (b) “What other reasons do you have for saying that?”
   This helps uncover the importance of different concept appeals and benefits.
2. Questions about the expected frequency of purchase – to help screen out novelty or low-frequency products. Typical wording: “Which statement best describes how often you think you would buy this product?”
   Once a week or more often
   Once every two or three weeks
   Once a month
   Once every two or three months
   Once every four or six months
   Once or twice a year
   Less often than once a year
   Never

Consumers are notoriously inaccurate in estimating how often they will use a product or even how often they currently buy products they already use.

But the frequency question can help screen out novelty or what might be called “birthday cake ideas” or products that are interesting to consumers, but which they wouldn’t buy often enough to make the product successful.

3. Questions about the uniqueness of the concept – to help identify “me-too” or generic ideas. Typical wording: “How different do you think this product is from other products now on the market?”
   Extremely different
   Somewhat different
   Slightly different
   Not at all different
4. Questions relating to price/value reactions – if price is included in the description and is felt to be potentially either a positive or negative factor. Typical wording: “Which statement best describes how you feel about the value of this product?”
   Very good value
   Fairly good value
   Average value
   Somewhat poor value
   Very poor value
5. Questions aimed at identifying other products the new product would replace – to help identify the expected degree of cut-in or cannibalization on the company’s existing products. Typical wording: “People often substitute new products for products they currently use. A new product may partly or totally replace something the family has been using. Do you feel that this product might partly or totally replace a product you are now using? If yes, what product would it be?”
6. Questions about whether the product solves a problem that isn’t being satisfied by products now on the market – to better understand the product’s appeal and uniqueness. Typical wording: “Do you feel this product might solve a problem or need you or other members of your family now have that isn’t being satisfied by products now on the market?”
7. If you’re into B2B, then the above-mentioned question this would be the main question you’d ask. Due to long sales cycles, usually complex products and numerous decision makers in the B2B realm, it’s best to ask the solving-a-problem question rather than the buying-intent one.
8. Questions to check whether there is anything confusing or difficult to understand about the concept – to check on the clarity of the product description. Typical wording: “Was there anything in the description of this product that was confusing or difficult for you to understand?”
   Yes. (If so: “What?”)
   No.
9. Questions aimed at determining whether the item is seen as more suitable for gift or for self-purchase – on housewares, clothing, and other products where gift purchase is significant. Typical wording: “Do you think you would be more likely to buy this product for yourself, as a gift for someone else, or both?”
   Self
   Gift
   Both

Step 4: Analyzing the New Product Concept Screening Results

The buying-intent question is the key measure of concept interest. Standard ways of analyzing the buying-intent question are to look at either the “top box” (“definitely buy”) score or the “top two boxes” (“definitely buy” plus “probably buy”).

Both of these measures identify the proportion of potential buyers who have a strong interest in the product, and they usually produce nearly the same ranking of concepts.

At the same time, some companies have moved beyond this simple “top box” system to a procedure which weights the buying-intent result to produce a number that seems to approximate penetration potential for a product.

See a weighting system that is often used:

Buying-intent scale	Test responses	Weights	Weighted score
Definitely will buy	21%	0.75	15.8%
Probably will buy	29%	0.25	7.3%
Might or might not buy	30%	0.10	3.0%
Probably will not buy	15%	0.03	0.5%
Definitely will not buy	5%	0.02	0.1%
Total:	100%		26.7%

The weights in this case assume that only about three-fourths (0.75) of the people who say they’d “definitely buy” the product will actually buy, about one-fourth (0.25) of those who say they’d “probably buy” will really buy, and so forth.

In this example, the estimated potential trial purchase level for the hypothetical test product is 26.7 percent – which is then discounted for expected awareness and distribution.

Another way to analyze concept test results is to plot the buying-intent and one other score e.g. uniqueness, problem-solving, or value (all the described above) for for each concept.

Where to Start If This New Product Concept Methodology Sounds Good For You

Companies that have not done any concept screening often wonder how to get started. A good way to begin is to conduct a “development project” to validate the applicability of concept testing for the company’s products and to begin developing guidelines for spotting “winners” and “losers.”

This type of project involves testing, in concept form, products already marketed by the company or its competitors – both successful and unsuccessful products.

Testing these products as concepts validates the ability of the test to predict or “backcast” – proven successes and failures. This is a low-risk, economical way to demonstrate the accuracy and value of concept testing.

A Word of Caution on Promoting Your Surveys

You may want to include some promotional tactics while conducting your new product concept surveys to increase survey completion rates. That’s OK as long as your promotional tactics won’t skew the survey results.

And if they’re aggressive, e.g. a free gift or a sample of the surveyed product for completing the survey, this is very likely.

One of my clients found that 10 percentage points at least are pushed into the “probably buy” box and 5 to 8 percentage points are added to the “definitely buy” box by adding free gifts.

Unfortunately (or fortunately) this bias does not always occur. Nevertheless, the possibility of such data distortion is serious enough to either not include heavy promo tactics or at least A/B test them first.

Summary

1. The most critical questions for evaluating interest in new product concepts relate to the following points:
   • Buying intent (the real key).
   • Uniqueness/difference of the product.
   • Expected frequency of use.
   • The consumer’s opinion on whether or not the product solves a problem.
2. Other questions that can be included as diagnostic measures include:
   • Open-ended questions: “Why do you say that?”
   • Questions on attractiveness of price/value.
   • Questions about products replaced.
   • Questions aimed at identifying confusing things about the concept,
   • Questions to determine whether the product is more likely to be bought as a gift or self-purchase.
3. Concept testing works. The huge payoff from identifying the one idea in a thousand that could be a runaway success has made new-product- concept testing one of the biggest “secrets” of true performance marketers and innovation-driven companies.