The move, hailed as a victory for user privacy, ultimately cost major tech platforms billions of dollars.
But when marketing professor Kellogg Anna Tuchman looked into the issue, found very few studies that showed the effect on businesses that advertised on these apps.
In fact, no one had really quantified the value of user data—such as browsing activity and online purchases—that is tracked and shared across apps to help serve ads across many major advertising platforms. This type of data is called “off-site data” because it is used to target ads on a specific platform, but is generated outside of the platform.
So how effective has off-site data been in helping to acquire new customers through advertising, and if it were to disappear entirely, how much would it cost businesses?
To study the issue, he teamed up with Nils Wernerfelt, an economist at Meta who will join Kellogg as an assistant professor this fall, along with Meta’s Robert Moakler and Bradley Shapiro of the University of Chicago.
The team conducted a large-scale experiment with more than 70,000 advertisers on Facebook and Instagram to find out how companies benefit from using off-site data to serve targeted ads.
They found that removing the ability to use off-site data would result in a 35 percent increase in the cost of acquiring each new customer—and would disproportionately hurt smaller advertisers.
“A lot of the debate around privacy regulation has focused on the consumer side, and for good reason,” says Tuchman. “But we found that removing off-site data makes it harder for smaller advertisers to acquire new customers and compete with incumbents. This could have implications for market entry and could ultimately have negative consequences for consumers if it leads to less variety and higher prices.”
Helping advertisers find the right customers
Digital advertising is now the majority of advertising spend, precisely because businesses can leverage detailed user data to match ads with potential consumers. To get the data from other websites that provides the basis for this matching – called off-site data – social media platforms encourage businesses to install “pixels”: small pieces of code on their websites that collect information about users and they send them to platforms for use in ad delivery. For example, if a customer buys shoes on a retailer’s website that has a Meta pixel installed, that purchase information will be sent to Meta. This helps the platform understand who the retailer’s real customers are and helps it target ads to that business.
When businesses create ad campaigns on Meta, they set a specific goal, such as increasing purchases. Meta then uses off-site data in its algorithms to show ads to people who are most likely to buy that product. For example, if the pixel data indicates that a product is only bought by women, the platform’s algorithm knows not to show ads for it to men.
“Advertisers often don’t have a clear sense of exactly who they should be targeting,” says Wernerfelt. “One of the great strengths of digital advertising is that Meta and other platforms can dynamically display ads to users, see who is buying the product, and inform the target audience.”
Pixels are everywhere. Most social media platforms have pixel variations that businesses can install, and millions of companies use them. However, many consumers have expressed concerns about their information being tracked and shared in such ways. Both companies and governments have responded with new policies and regulations. Apple’s “Ask the App Not to Track” feature made it easy for users to opt out of being tracked by such methods. The European Union’s General Data Protection Regulation law regulates this collection of personal data, and countries around the world are considering similar policies.
“The digital advertising ecosystem has grown so quickly that it’s ahead of regulation,” says Wernerfelt. “But regulations are being enacted with very little evidence to inform compromises. We wanted to provide a piece of the puzzle that shows the cost from the advertising side.”
Assessing the value of off-site data
To find the value of off-site data, the team conducted a large-scale, randomized experiment with more than 70,000 Facebook and Instagram advertisers for one week in the fall of 2021. For each campaign, the team randomly selected 5 percent of target audience and allocated 5 percent of the campaign budget to it. Within that 5 percent, the team randomly hid ads from the 10 percent of users who would normally see those ads. They then compared the results from those who saw the ad to those who didn’t.
From this data, they could calculate the cost per incremental (new) customer for each campaign. The results were far-reaching: while the median cost per incremental customer was about $42, it ranged from $5 to more than $200 for campaigns between the 90th and 10th percentiles of effectiveness. That’s because the campaigns ran across industries, from e-commerce to consumer products, in countries around the world.
“This experiment was on a massive scale and really allowed us to feel like we were holding the effectiveness of ads across industries,” Tuchman says.
With this data in hand, the team wondered what would happen if the off-site data disappeared. With algorithms without access to browsing behavior and past purchases, ads will likely be optimized for clicks. This means they will be shown to users who are most likely to click on the ad, not to those who are most likely to buy the product.
“But when you show ads to people who just click a lot, you lose a lot of customers,” Wernerfelt says.
To determine the exact effect of this change, they repeated their previous experiment—only instead of optimizing ad delivery using off-site data, they targeted ads from the same campaigns to users who were more likely to click on them.
If advertisers were to optimize for clicks instead of purchases, the team estimated that the average cost per incremental customer would increase to $57—about a 35 percent increase from using off-site data. The average total ad spend for the week was about $1,250. This means that instead of getting 30 new customers that week, the companies would only get 22 new customers.
“We showed that losing this data would have a big impact,” says Wernerfelt. “And it negatively affected the vast majority of advertisers.”
But one group was more affected than others: smaller advertisers. The team’s analysis showed that the impact of off-site data loss was five times greater for these advertisers than for larger ones. This is because small advertisers are often little known and rely on such off-site data and ad targeting to attract niche incremental customers.
Targeted advertising: good for small advertisers and consumers?
The team also looked at purchase data six months after the experiments to better understand the long-term effect on customers. They found that ads that use off-site data to target customers are much more effective at attracting repeat customers than ads optimized for clicks.
This finding suggests that off-site data may have benefits for both advertisers, who prefer long-term customers, and consumers, Wernerfelt says.
“There is concern that these ads may persuade people to buy products they don’t need or that they will regret later,” he says. “But if consumers continue to buy this product in the future, it seems likely that they will benefit and derive utility from these products.”
These results show that policies around monitoring consumer data should be subtle and careful, the researchers say.
“You can protect privacy that way, but it can come at a real cost on the advertiser’s side,” Wernerfelt says. “There is no privacy ‘free lunch’ here.”
To protect small businesses and ensure a competitive environment, perhaps platforms, advertisers and users could find a way to allow advertisers to use the data, compensating individuals for any privacy losses. Some companies are already conducting privacy-preserving research for ad delivery and measurement solutions.
And digital advertising isn’t just limited to for-profit companies. Wernerfelt has also studied the effectiveness of digital advertising for public health interventions.
“This is a fascinating area with such far-reaching implications,” he says. “There is a lot of excitement, but there are also a lot of unanswered questions, and privacy regulation is high among them.”
