Plenty of ad accounts test constantly and learn nothing. New videos go in, the winners get more money, and six months later nobody can say why anything won. That’s a production schedule with a nicer name. A testing loop has one extra requirement: every round has to change what you make next.
Here’s the version we run with brands. It takes four weeks, which fits comfortably around the 7 to 10 days that finished videos take to arrive.
The five steps
- Write a hypothesis. One sentence: we think X will beat Y because Z. “A hook that names the fear of irritation will beat a hook that names the glow, because our reviews mention redness far more than radiance.”
- Change one thing. Use the same body with different hooks, or the same script with a different creator. Change everything at once and you’ll get a winner with no explanation attached.
- Run it fairly. Give tests their own campaign, a set budget and an end date. Inside your main campaign, new ads have to fight your old favourites for spend, and the platform usually sides with the old favourites.
- Read it the same way every time. Use one scorecard, on the date you agreed, and try not to peek on day two.
- Write down what you now believe. One line in a shared log, carried into the next brief.
A scorecard that shows where the leak is
A single headline number tells you that an ad lost. A few numbers taken in order tell you where it lost, which is the part you can fix.
- Stop rate: the share of impressions that watch past three seconds. If it’s low, look at the hook.
- Hold rate: the share of those viewers who stay to the halfway point. If it’s low, the body is dragging, or the hook promised something the video doesn’t deliver.
- Click-through rate: if people watch and don’t click, the offer or the call to action is weak.
- Cost per purchase: if people click and don’t buy, check the landing page and the price before you blame the video.
Compare each ad with the others in the same test and with your own account’s history. Benchmarks from someone else’s account tell you very little about yours.
What a month looks like
- Week one: read last month’s results, update the log, write the new brief.
- Week two: creators film, and you review first cuts as they land.
- Weeks three and four: tests run. Leave them alone, then pull the numbers on the agreed date.
A worked example
Picture a hypothetical sleep powder brand. In month one it tests three angles: falling asleep faster, waking up less groggy, and swapping a nightly glass of wine for something warm. The wine angle wins on stop rate by a wide margin, but viewers drop off before the product appears.
The log now says: “People are curious about the wine swap. We take too long to get to the product.” So month two keeps the angle, moves the product into the first ten seconds and tests three new hooks. Hold rate improves and cost per purchase comes down. Month three repeats the idea with different creators, because a result that depends on one person isn’t much of a finding.
Three months in, the brand has a few good ads and, more usefully, a sentence it believes about its customers. That sentence can go to the media buyer, the person writing email subject lines and the next round of creators.
If you can’t say what last month’s test taught you, you paid for the ads and skipped the lesson.Maren Okafor, head of creative
Keep the log boring
A spreadsheet with five columns is plenty: date, hypothesis, what changed, result, what we believe now. Elaborate dashboards tend to get abandoned by the second quarter. The log only has to survive staff changes and short memories.
When an ad wins, move it to your main campaign and let it earn, then write the next hypothesis. The loop never finishes, which is the annoying part. It does get quicker, because each round starts from something you already know.