In 1998, Steve Jobs told BusinessWeek: “A lot of times, people don’t know what they want until you show it to them.”
The research industry spent the next 25 years calling that arrogance. They were wrong. And the science to prove it existed before Jobs was born.
1938
That’s when Paul Samuelson, a 23-year-old economist at Harvard, published a paper that should have ended the stated-preference era before it started.
His argument was straightforward: if you want to understand what people prefer, watch what they do. Don’t ask what they want. Observe what they choose. Revealed preferences, the decisions people actually make with real money and real consequences, tell you what stated preferences never can.
The research industry ignored him and kept asking.
What ‘asking’ actually gets you
When you ask people what they want, you get four things: what they think they should want, what they want you to think they want, what they wanted last time, and what they can articulate in a survey format. None of those things are what they’ll do next Tuesday when they’re standing in a store or clicking through a purchasing decision.
The meta-analytic evidence on this is consistent and damning. List and Gallet reviewed 29 studies in 2001 and found that subjects overstate their preferences by approximately a factor of 3 in hypothetical settings. Schmidt and Bijmolt’s 2020 analysis of 77 studies and over 45,000 subjects found an average hypothetical bias of 21%. Veylinx compared stated purchase interest against real-money behavioural bidding and found that 83% of respondents expressed intent to buy; 42% actually bid. That’s not a rounding error. That’s a different dataset entirely.
The gap between what people say and what people do is one of the most replicated findings in behavioural science. It has been documented across consumer goods, charitable giving, healthcare, financial decisions, and B2B purchasing. It doesn’t shrink in different cultures or categories. It is structural.
The research industry knows this. They built a $140 billion business on top of it anyway.
New Coke
In 1985, Coca-Cola ran approximately 200,000 blind taste tests at a cost of $4 million. It was one of the most exhaustive research programs in corporate history. Every data point pointed in the same direction. Respondents preferred the new formula. The science said go.
They launched. It was a catastrophe. The company pulled the product within 79 days. The research captured people’s preference for sweetness in a blind-sip test. What it missed was that Coke wasn’t a beverage. It was a 99-year-old cultural identity. People weren’t buying a drink. They were buying continuity, childhood, America. You can’t measure that in a taste test because people can’t tell you that in a taste test. They don’t know it themselves.
Coca-Cola had $4 million worth of stated preferences and zero insight into revealed behaviour. The market told them what the research couldn’t.
The industry’s real product
Here’s the uncomfortable part. The research industry isn’t primarily in the business of accuracy. It’s in the business of confidence.
Executives don’t buy research to find out what’s true. They buy it to feel certain about a decision they’ve already made, or to distribute responsibility for a decision that might go wrong. A $200,000 survey from a branded firm does something a behavioural data analysis can’t: it gives a senior leader something to point to in a post-mortem. “We followed the research.”
The effort heuristic, documented by Kruger, Wirtz, Van Boven and Altermatt in 2004, shows that people rate identical work as higher quality when they believe more effort was invested. The research industry’s methodology is slow, expensive, and human-intensive by design. Those qualities don’t produce better answers; they produce more credible ones.
That’s a positioning strategy. It’s not an accuracy strategy.
What Jobs understood
Jobs didn’t ignore customers. He watched them. He observed what they actually did with technology, not what they said they wanted from it. He looked at the gap between existing products and actual behaviour and designed into that gap.
The iPod wasn’t created because a focus group said, “I want a thousand songs in my pocket.” It was created because Jobs saw that people were carrying CDs, ripping CDs, and burning CDs, and that the real job was access to a personal music library on the move. The behaviour was already there. The product aligned with it.
That’s revealed preference thinking. Samuelson would have recognized it immediately.
The method that works
Behavioural data doesn’t lie the way stated preferences do, because it has no reason to. A job posting can’t manage your perception of it. A customer review written on a platform the company doesn’t control reflects what the customer actually experienced. A legal filing, a patent, a contract structure, a pricing change: these are decisions made with real stakes. They reveal what an organization actually prioritizes, not what it claims to prioritize.
The perception gap, the distance between what a company claims to be and what the market actually experiences, can’t be measured by asking the market. The market will give you its polite version. It will tell you what it thinks you want to hear, or what it thinks is true, which is not the same as what is true.
The only reliable map is built from behavioural evidence. What customers actually write when nobody from the company is listening. What employees actually say about how decisions get made. What competitors actually signal through hiring and investment. What regulators and courts have actually found.
That data exists. It’s uncontrolled by the company being measured. And it’s been the right methodology since at least 1938.
The industry had 87 years
Samuelson published in 1938. The behavioural science on stated versus revealed preference has been building ever since. The research industry had 87 years to change its core methodology.
It didn’t. Because the incentives run the other direction. Clients pay for confidence. Confidence comes from familiar methods. Familiar methods survive regardless of accuracy.
Jobs didn’t know the academic literature. He just trusted what he could observe over what people could articulate. That instinct was right. The science backs it. The $140 billion industry built on the alternative has been wrong for most of a century.
The market research industry’s perception gap is this: it believes it sells intelligence. What it actually sells is certainty. Those are different products, and only one of them is real.
Related:
Leave a Reply
You must be logged in to post a comment.