A/B Testing is NOT Personalization

A million visitors are going to hit your site – whether that’s over a year or over a day, you as a marketer want to figure out how to put your best story in front of them and drive engagement and conversion. Do you use A/B testing? Or is personalization the better approach?

As a marketer, you have probably used, been pitched, or at least heard of both of these methods. Marketers use them–though many confuse them–often wondering which best addresses their needs. Let’s start with a simple, stripped down definition for each of them:

A/B testing (sometimes called split testing) at its core is a simple and effective way to measure which version (A or B) of content or visual element performs better.

Personalization is the process of understanding who your audience is, figuring out what they want, and putting the best content or visual element in front of them.

How is personalization different from A/B testing?

In short, A/B testing uses basic statistical methods to report which content piece performs better for a general audience. Personalization uses data to segment audiences, serves content for each segment and reports on an audience’s engagement with specific content. One is a verification tool for content effectiveness, the other is a method to optimize audience experience.

Marketers often combine the two approaches because they both work towards the same goal: increasing engagement and conversion. However, it is important to implement personalization first, and then fine tune your message with A/B testing.

Two guys walk into a bar…

To illustrate the difference between A/B testing and personalization, let’s consider a simple use case.

Two guys walk into a bar. Let’s call them Alex and Ben. In a bar that is using A/B testing, Alex is given a wine list and Ben is given a beer list. The bar tracks whether Alex or Ben buy a drink or walk out empty handed, and attributes the result to the effectiveness of the wine or beer list.

In a bar using personalization, the bartender knows that Alex founded a brewery so he hands him the beer list. The bartender may not know Ben, but he has purple teeth, so the bartender gives Ben a wine list. Both Alex and Ben buy drinks because the bartender offered what they were each looking for. Here we have two audience segments: brewery owners and people with purple teeth. In the A/B testing bar, one or both could be mis-served. In the personalization bar, each visitor is served based on their identified need.

Content performance vs. audience performance

The A/B testing bar will only learn whether the wine list or beer list has a higher conversion rate. Because the personalization bar focuses on audience performance, the bartender will learn whether the experience for each target audience is delivering a higher conversion rate.

A/B testing is useful to optimize specific words after audience segments are implemented. For example, at the personalization bar, the bartender can A/B test which beer Alex likes or which wine Ben prefers without missing out on increased conversions from a personalized experience. The results of A/B testing are often misleading if you aren’t already using personalization to talk to the right audience with the right message.

If you want to learn more about optimizing your website through personalization, check out our Website Personalization Strategy eBook.