Four keys to using machine learning for campaign measurement
Simon Ejdemyr, Data Scientist at Facebook, explains how to develop machine learning-based approaches for measuring campaign performance.
CONTENTS
As the media landscape continues to grow more complicated, advertisers need to be innovative in how they measure their campaigns. With its ability to quickly interpret data and predict results, machine learning is an exciting technology that can help marketers combat complexity and understand performance.
Machine learning has many use cases for advertisers – Facebook IQ has already explored how automated systems can improve campaign planning as well as execution – and measurement can be among the most valuable when traditional methods aren't enough. Nevertheless, it can be difficult to determine exactly how to approach incorporating machine learning into your own campaign analysis.
Facebook understands these opportunities and challenges first-hand. To help marketers more accurately measure the incremental value of their marketing efforts, we built our own data-driven attribution model (DDA) that incorporates machine learning. Facebook IQ sat down with Simon Ejdemyr, a Data Scientist with our Marketing Science group to find out what lessons the team learned from this experience.
Ejdemyr shared these four key things to consider when developing and evaluating a machine learning solution for campaign measurement:
1. Find the gaps in your current strategy
Before getting too deep into the weeds, Ejdemyr recommends first stepping back and looking for gaps in your current measurement strategy. Fundamentally, machine learning should be utilised if it can help interpret the performance of campaigns in fresh ways but isn't a silver bullet to fix everything automatically. "If the end product isn't bringing anything new to the table then no matter how fancy it is, you're better off saving the time and the money", says Ejdemyr.
"We knew that marketers wanted to be able to predict the causal effects of Facebook ads quickly and at a low cost. This drove the development of a targeted tool that bridged the gap between experimental design-based approaches and simple attribution-based reporting. By starting with a clear understanding of the need, it was possible to ensure that machine learning was utilised appropriately and effectively."
2. Determine what success looks like
A potential pitfall when developing a machine learning-based measurement approach is that it will be declared a success simply because it exists, rather than because it matches your needs. To avoid this, you can build a validation framework before you see the results from a new solution. "We think that creating a validation system is just as important as model development itself", says Ejdemyr.
When we were developing our data-driven attribution model, a holdout set of data points was pulled from other experiments to see how inputs that had not yet been seen would be handled. This upfront work and definition of success gave us confidence that our approach would work well when applied to new campaigns.
3. Ensure that you have high-quality inputs
A machine learning-based measurement approach does not operate in a vacuum; it is reliant on data. "While different machine learning methods can do a lot of the heavy lifting, they can't make bad inputs good and they can't tell you how they should or shouldn't be used. For that, the solution needs good inputs and a skilled team", says Ejdemyr.
The data-driven attribution model was built, in part, off of conversion lift data, which gave the approach a strong and broad foundation. As Ejdemyr puts it: "Quality inputs will vary based on what data you have access to and what your objectives are, but without them one cannot create a good model no matter how complex the method is."
4. Continue to validate and refine your solution
After investing time and resources into creating a new machine learning-based approach, it can be tempting to pivot to that solution and declare your work done. However, models work in conjunction with other tools and can evolve. "No measurement solution is perfect. Finding the right complements and cadence with other measures ensures stronger decision-making", says Ejdemyr.
The intent for our data-driven attribution model was never to replace experiment-based testing but rather to provide a strong complement for marketers who wanted to explore causality. This means that the solution continues to be validated and refined to help it work effectively with other reporting options. As Ejdemyr notes: "Machine learning models improve over time. With continued investment, you can ensure that you are getting full measurement coverage".
What it means for marketers
Identify your needs.
Do your due diligence before investing and find gaps in your current measurement strategy that need to be filled. Utilise machine learning if it will help you interpret the performance of campaigns in new ways.
Test and learn.
Implementing a new approach is only the beginning of the process. To make your solution as effective as possible, it is necessary to learn how it works best with other measurement tools and to continue to invest in improvements over time.
Find out how fellow marketers are utilising measurement strategies such as incrementality and attribution to make smarter business decisions here.