What is A|B Split Testing for Facebook Ads?

Campaign excellence isn’t easy. Facebook split testing is critical to achieving excellence.

High-performance, control-beating creative is a rare thing. In our experience, after spending over a billion dollars in user acquisition advertising, usually only one out of twenty ads can beat the current “control” (the top-performing ad). 

The reality is, most ads fail. The extremely high failure rate of most creative shapes advertising budgets and advertising testing. Because 95% of creative fails, if you can’t A|B test ads quickly and affordably, your campaign performance is going to be crippled.

Facebook Split Testing compares different versions of your ads so you can see what works best and improve future campaigns. For example, you can test the same ad on two different audiences to see which ad performed better. Or, to test two delivery optimizations to determine which selection yields better results.

How Facebook Split Testing Works

• Split testing divides your audience into random, non-overlapping groups.
• This randomization helps ensure the test is conducted fairly because other factors won’t skew the results of the group comparison. It also ensures each ad set is given an equal chance in the auction.
• Each ad set tested has one distinct difference, called a variable. Facebook will duplicate your ads and only change the one variable you choose.
• To get the most accurate results from your split test, you’ll only have the opportunity to test one variable at a time. For example, if you test two different audiences against each other, you can’t also test two delivery optimizations simultaneously because you wouldn’t know for sure which change affected the performance.
• Split Testing is based on people, not cookies, and gathers results across multiple devices.
• The cost per result of each ad set is calculated and compared. The ad set with the lowest cost per result, such as cost per website purchase, wins. Facebook makes these calculations with Facebook’s attribution system. Facebook uses data from the test itself and thousand of simulations based on it, which helps us determine our confidence level in the results.
• After the test is complete, you’ll get a notification and email containing results. These insights can then fuel your ad strategy and help you design your next campaign.

Objectives for Split Testing

 
Facebook split testing supports the following business objectives:

• Traffic
• App installs
• Lead generation
• Conversions
• Video views
• Catalog sales
• Reach
• Engagement
• Messages
• Brand Awareness

Variables for Split Testing

 
Advertisers will have the option to test one of the following variables. You can test 5 different strategies with one of these variables.

• Target audience
• Delivery optimization
• Placements
• Creative
• Product sets

Setting a Budget and Schedule

 
Your split test should have a budget that will produce enough results to confidently determine a winning strategy. You can use the suggested budget that Facebook provides if you’re not sure about an ideal budget. It also provides a mandatory minimum budget to help guide you. The budget and audience will then be divided between the ad sets. You can choose to divide it evenly or weigh one more than the other(s), depending on your preference.

Facebook recommends 4-day tests for the most reliable results, and if you aren’t sure about an ideal time frame, you can start with 4 days. In general, your test should run for at least 3 days and no longer than 30 days. Tests shorter than 3 days may produce insufficient data to confidently determine a winner, and tests longer than 14 days may not be an efficient use of budget since a test winner can usually be determined in 14 days or sooner.

For this reason, Facebook recommends a test between 3-14 days long, and it must have a schedule between 3 and 30 days long.

When the test is over, you’ll receive a notification in Ads Manager and get an email with the results. You can then refine your ad strategy and design your next campaign.

Getting Started: Creating a Split Test in Ads Manager

 
A split test lets you test different versions of your ads so you can see what works best and improve future campaigns.

In Facebook Ads Manager, there are 3 ways to create a split test:
 

1. Guided creation.

 
Facebook walks you through the process of creating a split test, and your ads will be ready to run once you’ve completed the workflow. You can choose this method if you’re a new advertiser or if you’re more comfortable with a step-by-step process.
 

2. Quick creation.

 
You create the structure for a split test that can be finalized later.
 

3. Duplication.

 
Add a new ad set or ad to turn an existing campaign into a split test campaign.

Create a Split Test When Editing an Active Ad Set

 
An especially useful time to run a split test is when you make an edit to an existing active ad set. Doing so allows you to test the effectiveness of your edits.

Here’s how it works:

After you edit your ad set, Facebook will prompt you to create a test. If you do this, Facebook creates a new ad set with the edits you just made and returns the ad set you edited back to how it was. We’ll split their audiences and budgets for you, and they’ll both run until they’re scheduled to stop.

To create a split test while editing an ad set:

1. Go to Ads Manager.
2. Hover over the ad set you want to edit.
3. Click Edit.
4. Edit your ad set.
5. Find the Want to Test These Changes? box that appears.
6. Click Create Test.
7. Click Confirm. 

 
The test will begin when you publish your changes. You can check how your test is progressing at any time in Ads Manager. Facebook will provide you with results as soon as we have them. You can act on or ignore the results as you feel is appropriate. It doesn’t take any action for you based on the results. You can keep both ad sets running after you end the test, but the audiences will no longer be split.

Once you receive your split test results, you can review them to discover insights about the best performing ad set. These insights can help you determine your ad strategy and design your next campaign.

Conclusion

 
Creative development, testing, and strategy are critical for campaign performance. The algorithms at Facebook may be able to test creative elements, but they still can’t create those elements. They can’t do competitive analysis, either. And they can’t plan out a coherent creative strategy.

If you’re an acquisition manager, we recommend focusing on expanding your skills in those areas, specifically, being able to distill and interpret Facebook split testing data from creative so your team can deliver better creative.

Facebook will transform UA advertising through 2020. They’re working hard on a new feature that will optimize creative, creative placements, bids, and budgets for social marketers. We expect this new capability to be out sometime in Q1 2020, so stay tuned.

In the meantime, check out Facebook’s Power5 best practices for running campaigns, and feel free to contact us with any questions.