If you like this blog post and want more expert advice on performance marketing, direct marketing, direct response advertising, please share it. Thank you!

Share on linkedin
Share on twitter
Share on reddit
Share on facebook
Share on email
Share on whatsapp

This Blog Post's Table of Contents

Performance Marketing Testing Framework: The Most & Least Important Elements to A/B Test.

If you like this blog post and want more expert advice on performance marketing, direct marketing, direct response advertising, please share it. Thank you!

Share on linkedin
Share on twitter
Share on reddit
Share on facebook
Share on email
Share on whatsapp

This Blog Post's Table of Contents

Even after years of experience, sometimes selling the same item or kind of item for many years, I still find myself surprised by the results of my campaigns. Often I discover that what worked very well last year, or even last month, may not work now. The one predictable event is change, and what the performance marketer needs most to know is what the changes will be.

A large percentage of organizations that make an effort to expand their marketing by turning to performance-based activities never test at all. They develop their landing pages, online or offline ads based on intuition and roll out expensive campaigns that rarely achieve more than a tiny fraction of the results they should.

There are exceptions. Occasionally such a hunch-based program works. But the true expert, the professional performance marketer, soon learns to turn to testing as the only reliable guide in performance marketing.

What to Test?

Even those pundits who recognize the need for and urge testing when preparing a performance marketing campaign appear sometimes to be confused about what testing really is.

Testing is not a campaign to discover the prospects’ preferences for color of landing page background and buttons, or the prospects’ willingness to go to the trouble of finding an envelope and stamp with which to mail their orders.

In fact, the main purpose of testing is simply to ask the prospect what he or she wants! If we accept the premise that you cannot sell a prospect anything that he or she does not truly want – or cannot be induced to want – marketing becomes a proposition of finding out what the prospect does want or can be persuaded to want.

Obviously, no one but the prospect can really tell you what he or she wants. Everyone else is simply guessing. (Actually, prospects often can’t tell what they would want.)

What is a “Want”?

True want is an emotional need, not the physical item you offer. The prospect never really wants a home exerciser, per se. What he or she really wants is to be slim and trim, feel good, look good, be healthy, add years to life expectancy, wear clothes well, fit into a smaller size, be sexually attractive, be envied or admired by others, and/or otherwise avoid disasters and gain satisfactions.

The real question is this: What kind of promise (and proof) will persuade the prospect to give up some of his or her cash? For most people who might be persuaded to invest in your home exerciser, the promise of satisfying one or more of those wants is most important and most likely to induce an order.

Which promise should you feature most prominently? Not all equally, for that is a shotgun approach that dilutes the entire campaign, and generates doubt as to your sincerity.

The performance marketing promotion demands that you focus on some single, powerful motivator to rivet the prospect’s attention. Other motivators must be subordinated to the primary one to be properly effective.

There is no truly reliable way to actually ask the prospects to choose the most motivating promise other than dry or wet tests.

In fact, the prospects should not be aware that they are the subjects of a test.

Testing When Budget or Time is Limited

If your budget or time constrains are tight you could, instead of producing one or more completely new test ads (e.g. landing pages, mailings etc.), modify your existing winner so that its different test versions exactly the same as the winner, except for the headline, the lead and possibly a few sentences of the body copy.

In each of the test versions, a different value proposition – promise – is featured, and the results of each are recorded most carefully.

The value proposition is the first thing to test, and it is what you promise to do for the prospect, the specific benefit you pledge, not the product you wish to sell and the terms of your sale – cash, time payments, bonuses, discount, free trial, and so forth.

In short, if I promise that I will help you lose 15 pounds in 30 days, that is my value proposition. If I ask you to send me $39.95 for my patent exerciser, agree to a 30-day free trial with a money-back guarantee, that is my offer.

None of this should be interpreted to mean that other factors and elements such as bonus add-ons and special inducements should not be tested. Such elements can help fence-sitting prospects decide to buy, help you close more than a few doubtfuls.

And even where the differences in response are fractions of a percentage point, in large performance marketing campaigns or sales of “big ticket” (costly) items, that fraction of a point can mean many thousands of dollars added income.

Which means those matters are not unimportant, but can become important only when enough prospects respond with orders to make the entire testing program viable.

Finding Winning Formulas

Because your prospect is an almost infinitely variable sample, it is impossible to establish absolute rules about needs or motivators on either an absolute scale or on any scale of needs versus demographics or needs versus any other parameter.

We do know that certain words are strong motivators. “Free,” “new,” and “sale,” for example, are appeals that appear to be resistant to wearout factor. But we can only generalize about needs and drives, and we must rely on tests, in addition to instincts, to set guidelines for specific cases and applications.

Thus the belief held by most performance marketing experts that testing is a must for avoiding disaster, as well as for finding winning formulas.

Bitter experience has taught many of us to trust nothing else as completely as timely and specific testing, tests that meet three general criteria or constraints, all of which appear to be absolutely essential:

  1. The tests, certainly at least the initial or main tests, should be of important factors such as the basic appeal of the offer (i.e. the value proposition), rather than of some relatively trivial point such as the color of buttons or the size of the headlines. (There is little point in testing anything else if the value proposition itself fails to capture the interest of your prospects.)
  2. Test only one thing at a time and make all other factors as nearly identical as possible, or you will be unable to gauge accurately what makes a difference, if any, in the results achieved by the test mailing.
  3. Tests should be conducted for each campaign – the current one, that is, not a previous one. Yesterday’s truths are rarely reliable in marketing. Conditions change constantly for a variety of reasons. It is essential to test continuously so as to discover today’s truths. However, if you have a continuous (ongoing) performance marketing campaign and do not make major changes in the campaign, you may not have to make tests specifically, after you have made the original ones, before rolling out. In such circumstances, you should be monitoring results and compiling records that reflect and correlate the significant parameters on a regular basis. This is itself an ongoing test.

Demographic data is useful and a great many marketers rely heavily on it, but it is not an absolute by any means.

The only reliable method of coming even close to an absolute indication of market conditions – of truths and untruths about a given market at a given time – lies in testing the specific value proposition in the specific market at that specific time, observing the criteria listed here.

For example the TV networks, VOD providers like Netflix, and the producers who serve them with new programs every year. These executives are definitely in the marketing business, vying for their percentage of TV/VOD viewers and they rely heavily on demographic data.

They use the term “skew” to identify and describe what they believe to be the bulk of their audience, as when they report that their market for a given program “skews young,” for example.

This is one consideration that affects their decisions as to which programs to introduce each year and how to schedule them. They are evidently confident that they know what kind of program appeals to each skew – type of viewer – despite the heavy evidence of their mistakes each year, and they have never devised a test mechanism less costly than trying out new programs each season.

That their methodology is only marginally successful is attested to by the heavy casualty rate of new programs every year. Those that perish after a few episodes – sometimes after even a single one – are testimony to failures of market prediction and testimony to the fact that marketing is much more art than science. (More pointedly, it reveals the need to find reliable ways to test before rolling out.)

However, whereas networks make major investments to test their marketing estimates, with proper methods it is usually possible to test markets and materials for performance marketing campaigns at relatively little expense.

8 Illustrative Value Proposition / Promise Models

To bring some sort of order to this and create workable and useful tools for both copywriting and performance marketing testing, we must identify at least a tentative set of basic needs and lists of words and promises that are commonly used as value propositions and appear to be effective.

In organizing an intelligent performance marketing campaign, then, you should choose whatever promises appear to be best suited to your needs and you then test them to find out if your estimates were reliable and/or if you chose the best possible options for your copy.

The following lists, offered as worksheets, are by no means complete or exhaustive. In the relative infinity of products, services, ideas, and words, it would be virtually impossible to create lists even approaching completeness.

They are, rather, just a few representative samples. Use these as models. Add your own entries to the lists, and create other such worksheets, suitable to your own special needs.

Each model worksheet included here addresses a given basic type or class of product or service, suggests one or more basic motivators and/or bases for value propositions (promises), and lists some of the terms (word choices) that appear to be effective in this use.

And because the listings are only representative, make sure you add your own entries. In fact, to gain some facility with this, it’s a good idea to try your hand at adding items to each of these models, even if they are for totally different products or services than those of direct interest to you.

Becoming skilled in motivational analysis and appeal synthesis takes practice, as do most things, and you can only benefit from the time spent experimenting with these models.

Bear in mind that these models are not presented as perfect examples. You may very well be able to improve greatly on them, and you are encouraged to do so.

Recognize, too, that in many cases a term or item may fit into several categories. For example, many people are motivated by pride or vanity (physical appearance) in buying diets and dietary supplements, but others are motivated by a sense of insecurity, perhaps the fear that being overweight is a threat to their careers or even a threat to their general health and physical well-being.

Here again we find confirmation of the idea that security and insecurity or the drive for security is often expressed in inverse terms such as guilt, freedom from guilt, and even vanity, prestige, and self-image.

Catering to one’s self to pump up one’s self-image is a form of compensation for insecurity or feelings of inadequacy, a most common concern. You’ll see this reflected frequently in these models.

Model 1

  • Products: Locks, bolts, bars, weapons, alarms, fences
  • Basic Need: Physical security, safety for self, family, and possessions
  • Motivators: Fear, insecurity
  • Value Propositions (words/promises):
    • Protect yourself/loved ones/property/possessions
    • Keep your home/car/business safe
    • Be safe in your own home
    • It’s your duty
    • Better safe than sorry
    • You owe it to your loved ones

Model 2

  • Products/Services: Insurance, investments, savings accounts
  • Basic Need: Security
  • Motivator: Fear, insecurity, guilt/freedom from guilt
  • Value Propositions (words, promises):
    • Protect your home/possessions/family/loved ones/children
    • Provide for your/your children’s/your family’s future
    • Have an easy mind/sleep well at night
    • Better safe than sorry
    • It’s your duty/responsibility
    • You’re in good hands

Model 3

  • Products/Services: Medical services, checkups, weight control, diets, vitamins, health foods, exercise equipment, health spas
  • Basic Need: Security, ego gratification, self-image, vanity
  • Motivators: Fear, insecurity, guilt/freedom from guilt, vanity
  • Value Propositions (words, promises):
    • Keep fit; you owe it to your family/country
    • The Surgeon General of the U.S. says .. .
    • Add years to your life
    • Feel like a million
    • Have/keep that trim figure
    • Look years younger

Model 4

  • Products/Services: Career training, computer training
  • Basic Need: Security, self-image, vanity, guilt
  • Motivators: Gain, fear
  • Value Propositions (words, promises):
    • Secure your future
    • Have a better future
    • Make more money
    • Get a better job
    • Advance on the job
    • Be up-to-date
    • Be ready for the future
    • Your future is up to you

Model 5

  • Products/Services: Money-making/start your own business plans
  • Basic Need: Self-image, security, guilt/freedom from guilt
  • Motivators: Gain, money, ego gratification
  • Value Propositions (words, promises):
    • Be independent
    • Have a money-making business of your own
    • Complete guidance assures success
    • Build your own future
    • Your future is in your own hands

Model 6

  • Products/Services: Cars, low-end cost
  • Basic Need: Security, self-image
  • Motivators: Fear, gain
  • Value Propositions (words, promises):
    • Economical but reliable transportation
    • Money-saving practicality
    • The choice of sensible people
    • Built to last
    • The only bargain in cars today

Model 7

  • Products/Services: Cars, middle and high-end of cost
  • Basic Need: Self-image, ego, prestige
  • Motivators: Mark of success, keeping up with the Joneses, status symbols
  • Value Propositions (words, promises):
    • Out in front!
    • The mark of quality

Model 8

  • Products/Services: Newspapers, news portals and subscriptions
  • Basic Need: Self-image, security
  • Motivators: Need to know, be up to date
  • Value Propositions (words, promises):
    • Know what’s happening in the world today
    • Get the whole story, not little pieces of it
    • Find out what the future holds for you
    • Get the facts, the story behind the story

These models may be modified and adapted. Even these few models can be varied to fit many situations. For example, marketing of security devices such as locks and alarms may be slanted toward the protection of homes, families, businesses, vehicles, or other property.

And in most cases the broad-brush appeal that tries to sell everyone is less effective than the more narrowly focused approach that appeals to specific segments of the market.

Even when the products or services have equal application to more than one market segment, separate appeals, each directed to and focused on a specific segment, tend to be more effective.

Making the offer too general tends to dilute and weaken the appeal. In fact, the very act of structuring or wording the offer to broaden the appeal is itself a key factor in weakening it.

What If Your “B” Test Doesn’t Win

There are three possible outcomes of your initial A/B test:

1. One of your test campaigns may demonstrate that one value proposition not only produces a highly satisfactory response – volume of orders – but is also overwhelmingly superior to the others. In that case, you usually opt to roll out with that value proposition and creative.

2. Sometimes two or more tested materials pull satisfying responses, and do not show one offer significantly greater in pulling power than the others. In that case you might try either combining the most effective promises somehow or you might use the other campaign (i.e. the landing page) as a retargeting campaign, a consecutive mailing to the same database, or if you’re into online display advertising tinker with capping settings and direct some of additional traffic to it.

(There is a cumulative effect in some cases, where follow-up mailings, retargeting promos etc., even of identical material, pull better responses than predecessors have. This is often by far the most effective way to maximize the results of a campaign to a given list of prospects.)

3. It is possible that none of your test campaigns with new offers will pull an adequate response. In that case, you must decide whether to develop other offers and test further, seek other traffic sources / mailing lists, or drop the whole idea and seek something else to sell.

6 Steps For Your Initial A/B Test

Presumably, before you reach the go / no-go decision stage you will have verified the suitability of your traffic sources / mailing lists / audience (as in Facebook audience) by split testing the sources you have selected for your initial tests.

Although many stress the traffic source as the first item to be tested, I consider it the second item to be tested. In my own experience the early sequence should be generally along this line:

  1. Frame the value proposition and the offer.
  2. Choose what appears to be a suitable audience / mailing list / traffic source.
  3. Split test the offer
  4. Choose the better result
  5. Split test it again or split test the audience / mailing list / traffic source.
  6. Continue with other tests and/or roll out.

There is at least one other important thing to test: price. But here the aim is not necessarily to determine which price will bring the greatest number of orders but which price will bring the greatest profitability and how price affects response.

And here, as in all testing, results are often surprising. Lower prices do not always produce greater response. Sometimes the opposite is true.

Less Important Elements to Test (But Still Worth It)

The fact that the value proposition, the offer, traffic source / mailing lists / audience, and price are the most important items to test does not mean that all other factors are trivial.

Not at all.

The persuasiveness of the copy is important. The relative effectiveness of the offered proofs – testimonials versus rationales, for example – is important.

Response devices in print media and other elements are important, too, although in a subordinate sense, as discussed already.

Even such factors as the use of toll-free numbers, involvement devices (e.g. quizes online or stickers offline), and even, in some cases (such as when you are trying to develop an overall sense of class and quality as a necessary selling atmosphere) the quality or apparent classiness of the paper or landing page may be important.

And so may even some of the more minor items referred to earlier. But only subject to certain qualifications.

At the risk of belaboring the point (but a surprising number of performance marketers tend to forget this point), these other things cannot become important until after you have tested your value propositions and offers to verify that at least one of them works well, and to find the one that works best. If no one is interested in your basic offer or value proposition, why waste time on anything else?

There is at least one other factor: Some of these other considerations can mean as little as 0.1 or 0.01 percent difference. That difference is not significant in dollars-and-cents terms if you are driving only a low tens of thousands prospects to your promos.

But it can represent a great many dollars in the practical sense of overall cost and profitability if you are getting several million eyeballs or more to your offers.

Among the many such items that you might wish to test are the following:

A/B testable elements that drive the cost of the promotion both online and offline

  • Discounts
  • Guarantees
  • Premiums
  • Bonuses
  • Free trial offers
  • Bill-me-later schemes
  • Inclusion of toll-free number for ordering
  • Design quality

A/B testable elements that drive the cost of the promotion offline

  • Direct mail package / free standing insert’s number of elements, pages, response devices
  • Ad length (radio, TV, press, direct mail etc.)

A/B testable elements that drive the cost of the promotion online

  • Inclusion of multimedia: videos or video sales letters.

A/B testable other elements

  • Long copy versus short copy
  • All text versus text with graphics
  • Ugly design versus pretty design

As for points 1-3: even if the tested version of your promo doesn’t win response-wise (i.e. it’s a tie) maybe you’d generate additional income by slashing your costs.

Summary

There are some basic rules to observe in devising tests. The checklist following sets forth the chief ones:

  • Test Only Important Items. The two most important items are the value proposition – your promise of what you will do for customers – and the price. (Again, there may be exceptions, but these are usually the most important items.)
  • Test Significant Differences. The differences between the items tested must be great enough to be significant. Not “good health” versus “better health,” but “good health” versus “poor health,” and not $19.95 versus $18.95, but $19.95 versus $5.95 or $25.95.
  • Test Only One Thing at a Time. It is essential that you make alternative offerings identical except for the single item being tested. Otherwise, you have no way of knowing for sure what was responsible for the difference in results.
  • Keep Careful Records. The degree of detail in your records will have a great deal to do with how useful the tests are to you. You need to keep records that furnish a complete database and tell you what to do next. You need to be able to relate the results to causative factors if the tests are to be of practical value.
  • Test Continuously. Many experts in performance marketing test only representative samples when beginning a campaign, and then base the entire campaign on those initial tests. My own conviction is that it is far better to record results and study them continuously throughout the campaign. This enables you to detect changes in response – a not uncommon occurrence, especially in large or long-term campaigns – while there is yet time to react to them by making suitable changes. The data has some value as history, but has a great deal more value as real-time feedback.
  • Study the Records. An amazingly large number of performance marketing executives invest the time and money to design good tests and have the data assiduously recorded – and then rarely if ever look at the data. Of course, the best designed and most exhaustive and tests do you no good if you do not analyze the results and act on what they tell you.

If you like this blog post and want more expert advice on performance marketing, direct marketing, direct response advertising, please share it. Thank you!

Share on linkedin
LinkedIn
Share on twitter
Twitter
Share on reddit
Reddit
Share on facebook
Facebook
Share on email
Email
Share on whatsapp
WhatsApp

Leave a Reply to the Article

The expert's thoughts on direct response - growth hacking - performance-based marketing activities - DIRECT MARKETING

About Me, Rafal Lipnicki.

the direct / performance marketing consultant with a strange sounding name

Who.

Not your usual "guru" but a real-world performance marketing & innovation consultant based in Europe and an experienced senior executive at leading multinational companies.

What and Where.

I am a consultant for hire, working remotely and on-site all over the world (but Europe is always preferred). See my consulting services page for details.

How.

Contrarian advice most of the time. Document-based audits, workshops, one-off projects, mentoring programs, and more.