It’s a bold claim—not only because it bucks conventional wisdom, but because conducting robust research on the effects of online advertising is very difficult. To conduct a proper experiment, researchers would need to find a platform where some users see ads and others don’t — a rarity on today’s Internet.
But in reality, a small percentage of Facebook users live in such a special no-commercial zone. They’re called the “ad-free pool”—about 14.5 million people who When see ads in their Facebook News Feed. The company uses this randomly assigned segment as a control group in its own internal research, but Kellogg’s Nils Wernerfelt and colleagues saw it as a way to answer their own question.
“There are questions about how social media should be regulated, and a perennial issue that comes up is all these creepy targeted ads,” says Wernerfelt, assistant professor of marketing. “We saw an opportunity to run a fairly pure experiment and just ask, ‘How much do people really care about this?’
The surprising answer seems to be no big deal at all. When asked to put a dollar value on how much their Facebook experience was worth to them, responses were almost identical between users who saw ads in their News Feed and those who didn’t. Taken together, these findings suggest that the presence or absence of ads is of little importance to Facebook users.
“We expected that people wouldn’t like the ads,” Wernerfelt says, “but the results suggest that people might not actually care that much—and that all the regulatory information about it might be wrong.”
Data Jackpot
The main results in the team’s paper come from a research experiment conducted in 2022. At the time, Meta’s ad-free booking team had already been running for more than nine years, meaning there were millions of users on the site who had gone nearly a decade without ever seeing an ad in their News Feeds. For most researchers studying the effects of advertising, this kind of large-scale data and long-term experiments are almost impossible to find.
“Getting the resources to answer a question like this — an academic can’t do it alone,” says Wernerfelt. “You have to use the infrastructure and data that Meta has.”
But why did Facebook create the holdout group in the first place? Since the social network generates the vast majority of its revenue through advertising, it is in Meta’s best interest to understand the effect these ads have on the user experience of the product.
“If they start loading people with too many ads, at some point, they’re going to start seeing other metrics change,” explains Wernerfelt, who worked as a researcher at Meta before joining the faculty at Kellogg. “They want to monitor it, so that’s a big reason why tech companies are creating teams like this.
When conducting their study, Wernerfelt and his colleagues (Erik Brynjolfsson of Stanford University, Avinash Collis of Carnegie Mellon University, and Asad Liaqat, Daley Kutzman, Haritz Garro, and Daniel Deisenroth of Meta) recruited a representative sample of 53,166 different Facebook users from 53,166 different countries. A third of those users came from the News Feed ad-free holdout group.
Each user answered a survey asking if they would be willing to accept a certain amount of money (randomly set between $5 and $100) in exchange for deactivating their account for a month. Essentially, the survey asked users how much they value their Facebook experience by asking them how much it would cost to leave it. Users were also informed that some respondents would actually receive the dollar offer they were given.
As is common in economic research, this last part was done to incentivize honest responses from users. “If people don’t believe the offer is real, they’re more likely to say crazy things, and then your data means nothing,” says Wernerfelt. “But if I say, ‘Look, here’s the prospect of real money,’ they take it more seriously.”
Redemption
Among Facebook users who saw a typical number of ads, the median amount they accepted to temporarily terminate the service was $31.04. For users in the hold group — who had never seen an ad in News Feeds — the amount was nearly the same: $31.95. In essence, whether or not a user has Facebook ads, they tend to value the product the same.
What’s more, the researchers found that this rough equivalence held regardless of where users came from, how often they tended to spend in a single session, or how long they’d been Facebook users (about nine years).
“We do all this slicing to show that this effect is pretty consistent,” says Wernerfelt. “The average valuation for Facebook doesn’t seem to be significantly different for people who see ads versus those who don’t.”
Review of the regulation?
As is often the case with surprising findings, the researchers caution against over-interpretation.
“We have to be careful, this is a stage,” says Wernerfelt. “Extending beyond that to different contexts will naturally require hypotheses.”
For example, Wernerfelt suspects the findings would change if they looked at other ad channels: “The user experience with ads across all channels is just very different — compare, for example, how it feels to wait for a poorly targeted TV ad to end versus an ad on X or Pinterest.” With social media ads such as those studied here, it is possible that users may not engage with the ads because of the way they are tailored to each individual’s interests, potentially offering them some utility. Or perhaps both the positive and negative effects of ads are simply too small to make a noticeable difference.
On the other hand, businesses can suffer when regulators crack down on ads.
“There is growing evidence that when the effectiveness of digital advertising declines, companies shut down and prices rise for consumers,” adds Wernerfelt. “Speaking of course, if digital ads have all these downstream benefits for companies and consumers, and users don’t seem to care, maybe some of the regulatory zeal against online advertising is misplaced.”
