Given the above results, those of us testing using IPM have an opportunity to re-test IPM winners that exclude the control video to determine if we’ve been killing potential winners. As such, we recommend the following three-phase testing plan.
Our 3-Step Creative Testing Process
Phase 1: IPM Test
No control video
Create a new split test campaign using 3~6 new creatives (no control).
Setup campaign structure for basic App Install (No event optimization or value optimization)
Spend an equal amount on each creative. Ex: One ad per ad set.
Budget for at least 100 installs per creative
$200~$400 spend per ad is recommended (based on a CPI of $2-$4) if T1 English-speaking country
$20~$40 spend per ad/adset testing in India (based on $0.20-$0.40 CPI)
US Phase 1 testing.
10-15% LAL with a seed audience similar to past 90-day installers, or past 90 day payers.
Non-US Phase 1 testing.
Use broad targeting & English speakers only
If not available in India, try other English-speaking countries with lower CPMs than U.S. and similar results. Ex: ZA, CA, IE, AU, PH, etc.
Use the OS (iOS or Android) you intend to scale in production
Use one body text
Headline is optional
FB Newsfeed or Facebook Audience Networking placement only (not both and not auto placements)
Be sure winner has 100+ installs (50 installs acceptable in high CPI scenarios)
100 installs: 70% confidence with 5% margin of error
160 installs: 80% confidence with 5% margin of error
270 installs: 90% confidence with 5% margin of error
IAP Titles: kill losers, top 1~3 winners go to phase 2
IAA Titles: kill losers, allow top 1~3 “possible winners” to exit the learning phase and then put into “the Control’s” campaign
Which Creatives Move From Phase 1 > Phase 2?
How To Pick A Phase 1 IPM Winner
IPMs may range broadly or be clumped together
Goal: kill obvious losers and test remaining ads in phase 2
Ads (blue) have IPMs 6.77 & 6.34, move to phase 2
If all ads are very close (e.g. within 5%), increase the budget
IAA (in-app ads titles) you may need more LTV data before scaling
Phase 2: Initial ROAS
No control video
Create a new campaign with AEO or VO optimization
Place all creatives into a single adset (Multi Ads Per Adset)
Use IPM winner(s) from Phase 1 (you can combine winners from multiple Phase 1 tests into a single Phase 2 test)
OS – Android or iOS. 5-10% LALs from top seeds (purchases, frequent users + purchase) + Auto Placements
Testing can be done at a lower cost if you wish to run this campaign in other countries where ROAS is similar or higher but CPMs are much lower compared to the US – ie. South Africa, Ireland, Canada, etc.
Lifetime budget $3,500-$4,900 or daily budgets of $500-$750 over the course of 4-6 days (depending on your $/purchase).
WARNING! Skipping this step is highly likely to result in one of the following scenarios:
Challenger immediately kills the champion/control but hasn’t achieved enough statistical relevance or exited the learning phase and therefore the sustained ROAS/KPI may not be sustained.
Champion/control video has a lot more statistical history and relevance and most likely has exited the learning phases and may immediately kill the challenger before it has a chance to get enough data to properly fight for ROAS.
Phase 3: ROAS Scale
No control video
Use strong CBO campaign
Choose winner(s) from Phase 2 with good/decent ROAS
You’ve proven the ad has great IPM and “can monetize”
To win this phase, it must hit KPIs (D7 ROAS, etc.)
Create a copy of an existing ad set
Delete old ads and replace with your Phase 2 winner(s)
Allows new ads to spend in a competitive environment
Then, create a new ad set, roll out towards target audiences with solid ROAS / KPIs
CBO controls budgets between ad sets with control creatives and ad sets with new creative winners.
Intervene with adset min/max spend control only if new creatives don’t receive spend from CBO.
Require challenger to exit the learning phase before moving to challenge the control “Gladiator” video
Once the challenger has exited the learning phase, allow CBO to change budget distribution between challenger and champion
Note: We’re continuously testing our assumptions and discussing testing procedures with large Facebook advertisers.
We look forward to hearing how you’re testing and sharing more of what we uncover soon.