Section Three: Creative Testing 2.0 Recommendations
And here’s what we think they mean:
Creative Testing 2.0 Recommendations:
Given the above results, those of us testing using IPM have an opportunity to re-test IPM winners that exclude the control video to determine if we’ve been killing potential winners. As such, we recommend the following three-phase testing plan.
Creative Testing Phase 1: Initial IPM Test
- Use 3~6 creatives in one ad set with MAI bidding (NEVER include the control in the ad set)
- Less expensive than Facebook split testing but not a best practice and will not achieve statistically relevance
- 5% LAL in the US (for other countries, still use 5% LAL)
- This will give you an audience reach of 10M or smaller (other geos)
- Isolate one OS (iOS or Android)
- Facebook Newsfeed only
- Generate over 100 installs (50 installs are acceptable in high CPI scenarios)
- 100 installs: 70% confidence with 5% margin of error ° 160 installs: 80% confidence with 5% margin of error
- 270 installs: 90% confidence with 5% margin of error
- Lifetime budget: $500~$1,000 to drive installs that reach more than 70% confidence level
- The goal is to kill IPM losers quickly and inexpensively and then take top 1~2 IPM winners to phase 2
Creative Testing Phase 2: Initial ROAS Test
- Once you have high IPM winners identified, you can move into initial ROAS testing to see if high IPMs also will generate revenue
- Create a new campaign
- Test IPM winners from Phase 1 with AEO or VO
- 10% LAL, auto-placement, Android or iOS but do NOT test using WW audiences, chose only one country
- 1 ad set with IPM winners from phase 1
- Create new campaigns for new IPM winners from next rounds – do not add winners from other tests
- Lifetime budget: $800~$1,500
Creative Testing Phase 3: ROAS Scale Test
- Choose winners from Phase 2 with good ROAS
- Use CBO, create new ad set and roll them out towards the target audiences that produce good results for the control
- New ad set for new creative winners from different testing rounds
- Never compete against control in an ad set, instead, have them separated and compete for more budget within the same campaign
We’re still testing many of our assumptions and non-standard practices.
- Is it helpful to warm up IPM winners and establish “creative history” by purchasing installs in inexpensive countries?
- How long should IPM winners be “isolated” from the control to allow time for learning to be built up?
- 5-10% of LaL is contingent on the population of the country being tested?
- Results don’t appear to change being run as 1 ad per ad set or many ads per ad set? • Will lifetime vs daily budgets matter?
- Does a new campaign matter?
- Does resetting the post ID matter?
- Should creative testing be isolated to a test account?
We look forward to hearing how you’re testing and sharing more of what we uncover soon.
section 2 Table of Contents