If you like this blog post and want more expert advice on performance marketing, direct marketing, direct response advertising, please share it. Thank you!

Share on linkedin
Share on twitter
Share on reddit
Share on facebook
Share on email
Share on whatsapp

This Blog Post's Table of Contents

Research for Performance Marketers: How to Increase the Chance Your Next A/B Test Wins.

If you like this blog post and want more expert advice on performance marketing, direct marketing, direct response advertising, please share it. Thank you!

Share on linkedin
Share on twitter
Share on reddit
Share on facebook
Share on email
Share on whatsapp

This Blog Post's Table of Contents

This article attempts to answer an age-old question among performance marketers and direct marketing experts – “Why conduct market or marketing research when we can test the actual idea out in the real world?” – and its corollary – “Why spend money on research? If the test is not successful, the costs are offset by selling at least some product, whereas there is nothing to offset the cost of research.”

Why Performance Marketing Needs Research

The problem with this line of reasoning is that once a successful test has been conducted, the elements which made it a success are still unknown. Therefore, it is difficult to determine what parts of the test should be repeated.

Further, if it is a success at a 1% response rate, perhaps by knowing which elements made it successful, you could increase the response rate to 2%. If the test is less than successful, the elements that you should delete or change also are unknown.

In several areas of performance marketing research about why something works or does not work is important. The five fundamental areas are:

  1. Research among current customers
  2. Research into customer inactivity
  3. Merchandise selection research
  4. Research among prospects
  5. Research on advertising development

Research Among Current Customers

Customer Attitudes and Demographics

Basic demographic information should be gathered about current customers. Such information includes age, income, type of employment, education, marital status, and presence of children in the household.

Also, “demographic-like” information (termed classification data) relevant to your individual product category or business should be collected. For example, if you sell crafts, your company should gather information on the type of crafts engaged in, hours per week spent on the crafts, number of projects started and completed per year, other sources for buying craft material, etc.

In addition to basic demographic and classification data, you should collect information on customers’ attitudes about merchandise selection and quality, your site’s layout and description, customer service, delivery time, shipping and handling charges, billing procedures, etc. Customers should be asked to rate competitive businesses by the same criteria, to see how your business measures up to competition.

Focus group discussions are often conducted as an initial step to determine which characteristics or attitudes are relevant to a specific business and, therefore, should be measured. The actual interviewing of customers is usually best conducted via an email questionnaire – an economical way of collecting a wide range of information.

This basic demographic and attitude information is a valuable tool for your merchandise buyers and management alike. Through knowledge of the customer, you can make more informed decisions about merchandise mix, new or improved customer services, pricing policies, etc. Additionally, this information can be helpful in evaluating new traffic sources to attract new customers.

You should conduct an attitude study at least annually so that you can closely monitor positive or negative shifts in customer attitudes.

Customer Psychographics

A somewhat more sophisticated approach to understanding the customer can be made by determining the customers’ “psychographics.” This refers to the characteristics that distinguish a person’s lifestyle, such as the person’s leisure activities, interests, beliefs, and opinions.

Psychographics are a more meaningful way to view customers’ lifestyles than are simple demographics. For example, within a group of people with the same age, income, and marital status one person might drive an SUV and go camping with the family, while another drives a sportscar and takes premium vacations at all-inclusive resorts. Obviously, these people have different attitudes toward life and buy diverse types of merchandise, although their demographics are similar.

A psychographic profile of customers is useful in selecting and depicting merchandise for your ecommerce site, so it fits in with customers’ lifestyles. Psychographics also can be extremely useful in developing new, spin-off business ventures.

Customers will probably fall into several different psychographic segments, and research can determine which segments are the largest and have the greatest sales potential.

Group discussions among customers are an essential first step in determining what questions to ask in psychographic surveys. Again, such surveys can most efficiently be conducted via an email questionnaire.

Customer Segments to Study

All performance marketers will have different segments of customers in which they are particularly interested. Customer studies are most valuable when different segments are analyzed, such as frequent vs. infrequent customers, new vs. long-time customers, different traffic sources from which customers were generated, high-order quantity vs. low-order quantity, cash-on-delivery vs. credit card buyers, telemarketing vs. online buyers, etc.

Once you identify segments of prime importance, you can make decisions on which to survey, and what are appropriate sample sizes. A sample size of 150 per segment of interest is recommended.

Specific Issues Among Customers

Periodically specific issues may come up which also can benefit from market research.

For example, you may wish to determine:

  • which product box design to produce,
  • what product return instruction is best,
  • which additional payment providers would be of value to the customers,
  • whether introducing alternative payment schemes (cash-on-delivery, bill-me-later, installments) would increase the likelihood of more frequent purchases,
  • whether a change in product quantity (higher or lower) will be noticed by customers,
  • whether a new customer service will be accepted or whether a new layout design or logo will improve customer perceptions.

These and many other issues may be addressed using proven marketing research techniques.

Research into Customer Inactivity

During the life of your business there will be times when customers become inactive in varying stages. That is, frequent customers become infrequent and infrequent customers drop out altogether.

Additionally, there are instances when consumers who have requested the “free $50.00 off coupon” with your newsletter subscription, thus showing an interest in your products, do not purchase. Also, some consumers may buy from your site once and never again.

Increases in such occurrences may indicate that you should undertake research. You may use group discussions to determine hypotheses for these various stages of inactivity. Group discussion decisions should always be followed by quantitative studies to determine the magnitude of the distinct reasons for inactivity.

When conducting research on inactive customers, it is important that you study a sample of active customers to determine how they may differ. This comparison provides a clearer understanding of the key issues in motivating an inactive customer to become more like an active customer. Phone interviews are most productive for interviewing inactive customers as they are less likely to answer an email survey.

Again, a minimum sample size of 150 per segment under study is recommended.

Merchandise Selection Research

Research also can be useful in selecting merchandise for inclusion in your offers. The goal of such studies is to develop a research model which will help identify potentially strong and weak sellers. This research also can strengthen your inventory control efforts by helping you to predict which items will be the strongest and should, therefore, be well stocked.

Two keys to successful studies in this area are presenting merchandise the way it will appear in reality, and understanding which questions are predictive of purchase behavior.

You can use various methodologies ranging from personal interviews to phone and email surveys. In many methodologies consumers sort through pictures of the merchandise and are asked how interested they are in purchasing the items, how unique they consider the items to be, and what they consider the price/value relationship of the items to be.

In some cases, respondents actually are asked if they wish to purchase merchandise.

Merchandise selection research is tricky. What will work for one performance marketer may not work for another. The key is to develop a predictive model based on actual sales compared to answers generated from consumers. This procedure will usually involve experimental research dollars and at least one year of study.

See more on product ideas screening and concept testing:

Research Among Prospects

Prospect Attitudes and Demographics

To generate a more complete picture of the marketplace and your competitive position, it is important you understand who the prospects are, why they are not buying from you and how to convert them into buyers. By comparing the profile of customers to that of prospects, you can target promotions, merchandising, ads, and other efforts to expand the customer base.

You should explore issues such as awareness, overall attitudes, positive and negative associations, reasons for shopping other ecommerce sites or retail stores. Research can also pinpoint “brand” imagery for your business and its strongest competitors, so your marketplace position can be assessed. By having customers and prospects relate their perceptions of your business’ quality, price/value, reliability, etc., you can identify current strengths, weaknesses, and areas of opportunity.

Your prospective customers might include retail buyers in the category, competitive ecommerce customers and potential customers available in targetable audiences. Your ecommerce newsletter requestor non-buyers and former (inactive) customers could also fall into the prospect category.

Attitudinal research is often best conducted by phone. This method can result in an accurate and projectable measurement of awareness levels within selected markets. Sometimes, though, it is necessary to have prospects view your offer, so that more in-depth impressions can be gathered. In such cases, personal interviews are in order.

Prospect Psychographics

Collecting prospects’ psychographic lifestyle data helps reveal the extent to which they represent a strong potential for your business. This is done by comparing the prospect profile to the customer profile. The analysis allows your marketing efforts to be directed to segments of greatest opportunity in terms of size and sales potential.

Psychographic segmentation studies generally are used to expand upon the information obtained from demographic and attitudinal research. Again, psychographic studies, even with prospects, are best conducted by email since the questionnaire is completely structured and consists of a lengthy series of rating scales on lifestyle, attitudes toward shopping, etc.

In many cases, respondents can be recruited on a selective basis from an initial phone study. They are then emailed the psychographic questionnaire, along with an incentive, and asked to complete and submit it.

Research on Advertising Development

Strategy/Concept Research

The main objective of performance-based advertising is to prompt prospective customers to respond. Therefore, the first step in advertising research is to find the message that will motivate the largest number of prospects.

While desk research, conducted by a skilled direct response copywriter, is often sufficient and the preferred way, there are situations where either no adequate input is available (e.g., truly innovative products, complex B2B solutions), or the communications are severely limited (e.g., due to company policy, legal or regulatory constraints).

Initially, in such cases, focus group discussions may be used to determine how prospects currently view the product or business, and to explore different promotional methods.

After alternative positioning strategies have been developed, you should determine which is the strongest and most unique. This is often accomplished by showing prospective customers different, one-paragraph concepts or “promises” that position the product or business in diverse ways. Sometimes more elaborate strategy statements are used in this type of research.

Personal interviews usually are conducted during which prospects are asked to rate their reactions to the concepts, i.e., likes and dislikes, possible confusion and believability. In addition, prospects are asked what image they have of the product or business as a result of reading each concept.

Respondents usually will view only one strategy statement, and results will be compared from one strategy to another to determine the strongest.

Rough Execution Research

When you have determined the strongest strategy, it should be executed in several ways. Further research will then determine which is the strongest and most motivating execution of the strategy. Again, since respondents must view these advertisement mock-ups, these interviews are usually conducted in a one-on-one situation.

Execution tests measure the communication, recall and persuasion of the executions by exploring the respondents’ likes and dislikes, and their subjective determination of uniqueness, confusion and believability of the product or business as depicted in the execution. The overall rating and the composite image characteristics derived from each execution also are measured and compared to determine a winner.


In the case of new ecommerce and specialty off-shoots of general product or service-selling sites, you can create simulated mock-ups for testing among potential buyers. Then you evaluate various elements, such as layout, merchandise, artwork, and descriptions in rough form.

In this way, overall interest in the subject matter, theme or products can be determined prior to being promoted. Thus, elements of the site or campaign can be tailored to the needs and desires of a specific target group, and costly revisions in the final merchandise, layouts and description and services often can be avoided.


  1. An attitude research program among customers and prospects is vital to determining why elements work or do not work.
  2. A well-designed marketing research program should be an on-going dynamic procedure, as is testing.
  3. When properly applied, such research is a vital part of a decision-making process that can lead to more informed, educated, and creative solutions to difficult performance marketing management questions.

If you like this blog post and want more expert advice on performance marketing, direct marketing, direct response advertising, please share it. Thank you!

Share on linkedin
Share on twitter
Share on reddit
Share on facebook
Share on email
Share on whatsapp

Leave a Reply to the Article

The expert's thoughts on direct response - growth hacking - performance-based marketing activities - DIRECT MARKETING

About Me, Rafal Lipnicki.

the direct / performance marketing consultant with a strange sounding name


Not your usual "guru" but a real-world performance marketing & innovation consultant based in Europe and an experienced senior executive at leading multinational companies.

What and Where.

I am a consultant for hire, working remotely and on-site all over the world (but Europe is always preferred). See my consulting services page for details.


Contrarian advice most of the time. Document-based audits, workshops, one-off projects, mentoring programs, and more.