Section Three: Creative Testing 2.0 Recommendations



And here’s what we think they mean:


Creative Testing 2.0 Recommendations:


Given the above results, those of us testing using IPM have an opportunity to re-test IPM winners that exclude the control video to determine if we’ve been killing potential winners. As such, we recommend the following three-phase testing plan.


Our 3-Step Creative Testing Process


creative testing recommendations

Phase 1: IPM Test

  • No control video
  • Create a new split test campaign using 3~6 new creatives (no control).
    • Setup campaign structure for basic App Install (No event optimization or value optimization)
    • Spend an equal amount on each creative. Ex: One ad per ad set.
    • Budget for at least 100 installs per creative
      • $200~$400 spend per ad is recommended (based on a CPI of $2-$4) if T1 English-speaking country
      • $20~$40 spend per ad/adset testing in India (based on $0.20-$0.40 CPI)
    • US Phase 1 testing.
      • 10-15% LAL with a seed audience similar to past 90-day installers, or past 90 day payers.
    • Non-US Phase 1 testing.
      • Use broad targeting & English speakers only
      • If not available in India, try other English-speaking countries with lower CPMs than U.S. and similar results. Ex: ZA, CA, IE, AU, PH, etc.
    • Use the OS (iOS or Android) you intend to scale in production
    • Use one body text
    • Headline is optional
    • FB Newsfeed or Facebook Audience Networking placement only (not both and not auto placements)
    • Be sure winner has 100+ installs (50 installs acceptable in high CPI scenarios)
      • 100 installs: 70% confidence with 5% margin of error
      • 160 installs: 80% confidence with 5% margin of error
      • 270 installs: 90% confidence with 5% margin of error
      • IAP Titles: kill losers, top 1~3 winners go to phase 2
      • IAA Titles: kill losers, allow top 1~3 “possible winners” to exit the learning phase and then put into “the Control’s” campaign

Which Creatives Move From Phase 1 > Phase 2?

Which Creatives Move From Phase 1 > Phase 2?

  • How To Pick A Phase 1 IPM Winner
    • IPMs may range broadly or be clumped together
    • Goal: kill obvious losers and test remaining ads in phase 2
    • Ads (blue) have IPMs 6.77 & 6.34, move to phase 2
  • If all ads are very close (e.g. within 5%), increase the budget
  • IAA (in-app ads titles) you may need more LTV data before scaling


Phase 2: Initial ROAS

  • No control video
  • Create a new campaign with AEO or VO optimization
  • Place all creatives into a single adset (Multi Ads Per Adset)
  • Use IPM winner(s) from Phase 1 (you can combine winners from multiple Phase 1 tests into a single Phase 2 test)
  • OS – Android or iOS. 5-10% LALs from top seeds (purchases, frequent users + purchase) + Auto Placements
  • Testing can be done at a lower cost if you wish to run this campaign in other countries where ROAS is similar or higher but CPMs are much lower compared to the US – ie. South Africa, Ireland, Canada, etc.
  • Lifetime budget $3,500-$4,900 or daily budgets of $500-$750 over the course of 4-6 days (depending on your $/purchase).
  • WARNING! Skipping this step is highly likely to result in one of the following scenarios:
    • Challenger immediately kills the champion/control but hasn’t achieved enough statistical relevance or exited the learning phase and therefore the sustained ROAS/KPI may not be sustained.
    • Champion/control video has a lot more statistical history and relevance and most likely has exited the learning phases and may immediately kill the challenger before it has a chance to get enough data to properly fight for ROAS.


Phase 3: ROAS Scale

  • No control video
  • Use strong CBO campaign
  • Choose winner(s) from Phase 2 with good/decent ROAS
    • You’ve proven the ad has great IPM and “can monetize”
    • To win this phase, it must hit KPIs (D7 ROAS, etc.)
  • Create a copy of an existing ad set
    • Delete old ads and replace with your Phase 2 winner(s)
    • Allows new ads to spend in a competitive environment
  • Then, create a new ad set, roll out towards target audiences with solid ROAS / KPIs
  • CBO controls budgets between ad sets with control creatives and ad sets with new creative winners.
    • Intervene with adset min/max spend control only if new creatives don’t receive spend from CBO.
  • Require challenger to exit the learning phase before moving to challenge the control “Gladiator” video
  • Once the challenger has exited the learning phase, allow CBO to change budget distribution between challenger and champion


Note: We’re continuously testing our assumptions and discussing testing procedures with large Facebook advertisers.

We look forward to hearing how you’re testing and sharing more of what we uncover soon.


section 2 Table of Contents section 4

READ OUR OTHER whitepapers for creative best practices for Facebook & Google social advertising.

Download PDF Today!

Please prove you are human by selecting the Truck.

Read our Whitepapers!

Creative Best Practices for Facebook & Google social advertising.

You have Successfully Subscribed!