Advertising insights
Putting creative pre-testing to the test
Understand how evaluating advert creative before it goes live can predict advert success and help build a better campaign.
CONTENTS
Watching TV on a laptop, streaming live video on a phone and reading news headlines on a tablet are just a few of the ways that people consume media these days. And no matter the channel, marketers want advertising campaigns that reach their audience and make an impact.
About the study
As advertisers continue to shift more of their budget to digital, and to mobile specifically, there is a need for rigorous creative evaluation. According to an Ipsos study, creative quality determines 75% of the impact, as measured by brand and advert recall.1
Advertisers have long used creative pre-testing research to optimise their adverts before launching a campaign. Facebook Marketing Science wanted to understand if the leading industry creative pre-testing solutions in the US could successfully be applied in a digital, feed-based environment.
In June 2016, we tested four advert creatives from two completed US campaigns from Frito-Lay North America brands. We used three creative pre-testing solution providers: Ad-Vance by MetrixLab, LinkNow for Digital by Millward Brown and ASI Connect by IPSOS.
We wanted to understand if:
- Testing digital creative could provide insight into how campaigns perform in-market
- Different testing solutions would provide consistent recommendations
- Digital creative testing results would hold up in the Facebook News Feed
Methodology
Working closely with Frito-Lay North America, we identified campaigns from Doritos and Ruffles that ran on Facebook during the second half of 2015. Each campaign ran for a similar time period, used comparable targeting strategies and had the goals of increasing awareness of a new flavour and driving trial.
Despite having the same goals, each campaign had different outcomes. The Doritos campaign had much stronger sales and brand lift than the Ruffles campaign.
With the campaign performance measured, we could see whether pre-testing was a good determiner of a campaign's ability to deliver on business objectives. We selected two representative videos from the Doritos and Ruffles campaigns that ran on Facebook to compare the three digital pre-testing solutions. The pre-testing solution providers were not given any background on how the adverts had performed in-market.
“
The goal of this study was to quantify creative pre-testing and its ability to predict whether an ad will be effective. Determining which ad creatives have the most potential to deliver on business objectives, such as in-store sales, will help drive more impactful campaigns.
Shawn Baron
Measurement Lead, Marketing Science, Facebook
Chips all in for consistent results
Results were consistent across the three pre-testing solutions, which all selected the advert creative that performed best in-market in 2015 as the strongest advert, but gave mixed results for the remaining three adverts.
The Doritos advert (below) was the strongest overall performer in terms of both breakthrough and response. Compared to the lowest-scoring advert, the Doritos advert scored three times higher on Ipsos's Copy Effect Index, 8% higher on Millward Brown's Impact Score and 38% higher for MetrixLab's advert recognition. The brand and product benefit are well integrated into the storyline, leading to strong brand recall. The video both stands out and grabs attention, and consumers are interested in the featured brand after watching the video.
Breaking through the screen
A feed environment requires attention-grabbing advert creative. And our research has shown that every second matters. People are consuming and recalling more content faster than ever before. To determine if pre-testing results will hold up in the Facebook News Feed, we need to look at the different testing environments.
Testing environments vary in how realistic they are for people viewing content.
Each of the three creative pre-testing solutions used a different testing environment. All three solutions measure brand presence as a factor to predict if an advert will break through. We saw that as the testing environments began to mirror how people expect to experience an advert within the Facebook News Feed, it became more difficult for otherwise successful advert creative to stand out. The more relevant and personalised the surrounding content was, the more difficult it was for the advert to break out.
This study suggests that creative has to work harder in a personalised feed environment than in an environment with content that isn't as relevant. We have seen that when advertisers optimise for the screen where the advert is being delivered, creative is able to break through more effectively.
Factors to consider
Based on this analysis, there are three factors to consider when implementing a creative pre-test for a campaign. Consider where the advert creative will appear when selecting a testing environment (desktop vs mobile) and how this will affect your results. Look at the reporting metrics of each pre-testing solution and determine which one best aligns with your business objectives. Remember that adverts are rarely flighted in isolation on Facebook or other digital platforms. Current pre-testing solutions are designed to evaluate a single advert, which could be a limitation in digital testing.
While the tests were able to consistently pick the top-performing advert, they were less clear on predicting the in-market success of the remaining three adverts. The adverts all tested average or below average compared to each solution's copy-testing norms. This suggests that if a creative's execution is average, the campaign flighting is going to have a larger impact on the overall campaign success.
What it means for marketers
As Atin Kulkarni, Senior Director of Portfolio Analytics of Frito-Lay North America, explained: "Creative development is an important focus for us to maximise the effectiveness of our marketing efforts and drive sales. Digital creative pre-testing provides us with a new way to think about evaluating creative before we go to market."
While all three pre-testing solutions provided similar results for the best advert, none were able to predict the in-market outcomes of the other three adverts. Broadly, there were things we learned that can influence how marketers should evaluate their creative.
Put your creative to the test:
Testing digital advert creative before a campaign goes live will help advertisers launch higher-impact campaigns in-market and spend more efficiently. Based on results from creative pre-testing, marketers can confidently decide which assets to support with media spend.
Think holistically:
Playing with creative can help identify the best overall combination of advert creative assets for each campaign. While testing solutions in our study were consistent in predicting the top-performing advert, when creative is average, the other factors of the campaign become even more important. Consider how the creative fits with targeting, timing and other campaign goals.
Capture attention in-feed:
While creative pre-testing worked in a feed-based environment, adverts viewed in-feed require more breakthrough creative to capture attention. Facebook News Feed is fast, frequent and personalised. Advertisers should leverage iconography and branding cues strategically to captivate consumers early and get people to stop scrolling.
Source 2: Oracle Data Cloud-measured Facebook campaigns, May-October 2015.
Source unless otherwise specified: Four Ad-Vance studies by MetrixLab, four LinkNow studies by Millward Brown Digital and four ASI Connect studies by Ipsos (Facebook-commissioned studies of people aged 18+ in the US), May 2016.