{"id":4330,"date":"2026-05-12T19:48:34","date_gmt":"2026-05-12T23:48:34","guid":{"rendered":"https:\/\/paulsyng.com\/blog\/?p=4330"},"modified":"2026-05-12T20:13:06","modified_gmt":"2026-05-13T00:13:06","slug":"the-intelligence-layer-theyre-all-missing","status":"publish","type":"post","link":"https:\/\/paulsyng.com\/blog\/the-intelligence-layer-theyre-all-missing\/","title":{"rendered":"The Intelligence Layer They’re All Missing"},"content":{"rendered":"\n
\n

TL;DR.<\/strong> The $153 billion global research industry (surveys, management consulting, analyst firms, social listening, and now AI search) sources its evidence from people and companies who know they are being observed. I call that performed signal. Every source biased in the same direction, toward the audience the data was produced for. The category nobody built until now is fast, scaled access to unperformed evidence \u2014 regulatory filings, litigation records, workforce sentiment, institutional and customer language produced when nobody is asking. The Bain 80\/8 study from 2005 documented the cost of getting this wrong. 80% of executives believed their company delivered superior customer experience. 8% of customers agreed. This article walks through the mechanism, the 87-year history, and what changes when intelligence is built above the line.<\/p>\n<\/blockquote>\n\n\n\n

Key Definitions<\/h2>\n\n\n\n
    \n
  • Performed signal:<\/strong> evidence produced when the source knows it is being observed \u2014 surveys, interviews, analyst briefings, press releases, solicited reviews. Shaped by the audience it was produced for.<\/li>\n\n\n\n
  • Unperformed evidence:<\/strong> evidence produced without awareness of an audience \u2014 regulatory filings, litigation records, workforce sentiment on third-party platforms, hiring velocity, patent activity, customer language nobody solicited.<\/li>\n\n\n\n
  • Perception gap:<\/strong> the measurable distance between what a company claims about itself and what unperformed evidence shows. The most expensive unmeasured variable in business.<\/li>\n\n\n\n
  • The 80\/8 Problem:<\/strong> Bain & Company\u2019s 2005 finding across 362 companies. 80% of executives believed their company delivered superior CX. 8% of customers agreed. A 72-point gap inside the same organization.<\/li>\n\n\n\n
  • Demand characteristics:<\/strong> Martin Orne\u2019s 1962 finding that the moment a person knows they are participating in a study, their answers shift toward what they perceive the researcher expects.<\/li>\n\n\n\n
  • Above the line \/ below the line:<\/strong> the structural divide between unperformed evidence (above) and performed signal (below) that organises every commercial intelligence tool ever built.<\/li>\n<\/ul>\n\n\n\n

    The Argument in One Image<\/h2>\n\n\n\n

    I put every research tool I could think of on a single chart. Surveys. Management consulting. Analyst firms. Social listening. Competitive intelligence. AI search. Two axes \u2014 speed of insight, and type of evidence. <\/p>\n\n\n\n

    What surprised me was how cleanly they grouped. They all sit below the same line.<\/p>\n\n\n\n

    \"\"<\/figure>\n\n\n\n

    The line separates two different kinds of evidence. Below the line is performed signal. Above the line is unperformed evidence. The entire commercial intelligence stack, from McKinsey to ChatGPT, operates below the line. Until now, the top-right quadrant has been empty. Not for lack of recognition. The commercial reason to build there did not exist.<\/p>\n\n\n\n

    This article explains why that quadrant was empty for 87 years, what changed, and what kind of decision-making becomes possible when intelligence finally lives above the line.<\/p>\n\n\n\n

    Why Don\u2019t Surveys Predict Actual Customer Behaviour?<\/h2>\n\n\n\n

    Because the act of asking changes the answer. This is not a methodology flaw that better sampling can fix. It is a structural feature of any research instrument built on the question-asker \/ question-answerer dyad.<\/p>\n\n\n\n

    In 1962, social psychologist Martin Orne published a paper in American Psychologist<\/em> that should have ended traditional market research. He documented demand characteristics<\/em> \u2014 the cues a research environment gives off that tell participants what answers are expected. His conclusion was uncompromising: \u201cIt is futile to imagine an experiment that could be created without demand characteristics.\u201d<\/em><\/p>\n\n\n\n

    Sixty-four years later, the research industry is still pretending it has solved this. It hasn\u2019t. It can\u2019t. The methodology requires a question-asker and a question-answerer \u2014 a dyad that is structurally performative regardless of how rigorous the sampling frame, how clever the screener, or how validated the instrument.<\/p>\n\n\n\n

    There is older evidence too. In 1938, economist Paul Samuelson published A Note on the Pure Theory of Consumer\u2019s Behavior<\/em> and introduced the distinction between stated preference<\/em> (what people say they will do) and revealed preference<\/em> (what they actually do when nobody is asking). The behavioural science on this gap has been accumulating for 88 years. <\/p>\n\n\n\n

    Daniel Kahneman won a Nobel Prize in part for documenting how unreliable introspection is as a guide to behaviour. Gerald Zaltman at Harvard estimates that 95% of cognition occurs below conscious awareness \u2014 meaning the answers people give in surveys are stories the conscious mind constructs after<\/em> the decision has been made, not reports of the decision itself.<\/p>\n\n\n\n

    The mechanism:<\/strong> survey methodology requires a question-answerer who can be asked something and produce an answer. The whole apparatus is built on the dyad that Orne identified as structurally compromised. Removing the dyad would dismantle the business model. So the industry sells what it can produce: performed signal at varying speeds and price points.<\/p>\n\n\n\n

    What Are the Four Categories of Performed Signal?<\/h2>\n\n\n\n

    Look at the chart. The cluster below the line includes tools that share almost nothing else in common. A McKinsey engagement is nothing like a Sprinklr dashboard. A Kantar tracking study is nothing like a ChatGPT query. They differ on price by four orders of magnitude. They differ on speed from minutes to months.<\/p>\n\n\n\n

    They all source from the same kind of evidence.<\/p>\n\n\n\n

    Category<\/th>Examples<\/th>Speed<\/th>Cost<\/th>Unit of Production<\/th><\/tr><\/thead>
    Traditional research<\/td>Kantar, Nielsen, Ipsos<\/td>6\u201312 weeks<\/td>$50K\u2013$500K<\/td>Human surveying<\/td><\/tr>
    Management consulting<\/td>McKinsey, BCG, Bain<\/td>3\u20136 months<\/td>$500K\u2013$5M<\/td>Human synthesis<\/td><\/tr>
    Analyst firms<\/td>Gartner, Forrester, IDC<\/td>Quarterly<\/td>$50K\u2013$250K<\/td>Vendor briefings<\/td><\/tr>
    Social & competitive intel<\/td>Sprinklr, Klue, Crayon<\/td>Days\u2013weeks<\/td>$25K\u2013$150K<\/td>Published cadence<\/td><\/tr>
    AI search<\/td>ChatGPT, Perplexity, Gemini<\/td>Seconds<\/td>$0\u2013$200\/mo<\/td>Web retrieval<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

    All five sit below the line because none of them were built on unperformed sources. The differences in speed and cost reflect differences in production efficiency<\/em>. They do not reflect differences in evidence type<\/em>. A faster way to retrieve performed signal is still a retrieval of performed signal.<\/p>\n\n\n\n

    The implication:<\/strong> the commercial intelligence stack, from the most prestigious consultancy to the latest AI search engine, operates on a corpus that companies and people produce when they know they are being observed.<\/p>\n\n\n\n

    Why Doesn\u2019t Faster Research Solve the Problem?<\/h2>\n\n\n\n

    Below the line, faster makes it worse.<\/p>\n\n\n\n

    That sentence took me a while to be willing to write. It sounds counterintuitive. Speed is supposed to be progress. Examine what speed actually does to performed signal.<\/p>\n\n\n\n

    Traditional research is slow. A Kantar tracker takes 6\u201312 weeks. A McKinsey engagement runs three to six months. The slowness is the methodology\u2019s most valuable feature. Slowness signals effort. Effort signals rigour. Rigour produces confidence.<\/p>\n\n\n\n

    This is what Norton, Mochon, and Ariely (2012) called the effort heuristic<\/em> \u2014 the documented finding that people rate identical outputs as higher quality when they believe more effort was invested. The research industry has monetized this for eighty years. Slowness was the product. What you were buying, beyond the data, was the appearance of thoroughness \u2014 and the political cover that came with it. \u201cWe commissioned Kantar.\u201d<\/em> \u201cBCG did the analysis.\u201d<\/em> \u201cForrester rated us a Leader.\u201d<\/em><\/p>\n\n\n\n

    AI search collapses the timeline. It delivers the same kind of evidence (synthesized PR, marketing pages, analyst commentary, performed customer reviews, SEO content) in seconds, for free. It strips away the slowness, which means it strips away the effort heuristic, which means it strips away the political cover.<\/p>\n\n\n\n

    What is left is the underlying signal, naked. And the underlying signal was always performed. The research industry hid that fact behind a six-figure invoice and a twelve-week delivery cycle. AI search did not make performed signal worse. It made the thinness of performed signal visible, by removing the theatrical dressing that made it feel like rigour.<\/p>\n\n\n\n

    The implication:<\/strong> the fast tools deliver the same epistemological product as the slow tools, without the institutional alibi. You pay nothing, you wait nothing, and what you get is a polished synthesis of what every company in your market wants you to believe about itself. Speed is what made the thinness visible.<\/p>\n\n\n\n

    What Did Bain\u2019s 80\/8 Study Actually Find?<\/h2>\n\n\n\n

    In 2005, Bain & Company surveyed 362 companies. They asked executives whether their company delivered a superior customer experience. 80% said yes.<\/strong> They then asked the customers of those same companies whether they agreed. 8% said yes.<\/strong><\/p>\n\n\n\n

    A 72-point gap. Not between competitors. Inside the same organization. Same company, same product, same week. The leadership and the customers were living in different realities.<\/p>\n\n\n\n

    Both groups were asked questions. Both groups answered honestly. The methodology was sound. The result was incoherent, because both answers were performed. The executives performed for their self-image and their boards. The customers performed in whatever frame the survey instrument was constructed. Neither set of answers told you what was actually happening.<\/p>\n\n\n\n

    Twenty-one years later, McKinsey\u2019s State of Organizations 2026<\/em> report, drawing on a survey of more than 10,000 senior executives across 15 countries and 16 industries, finds leaders still anchored to confidence in their own performance even as the pace of disruption accelerates. The gap is not closing. If anything, the speed of business has accelerated faster than the speed of self-correction.<\/p>\n\n\n\n

    This is not abstract. It produces real failures with real price tags:<\/p>\n\n\n\n

    Company<\/th>What Research Said<\/th>What Happened<\/th>Cost<\/th><\/tr><\/thead>
    New Coke (1985)<\/td>Taste tests confirmed superiority<\/td>79-day catastrophe<\/td>$4M research + retraction<\/td><\/tr>
    Tropicana (2009)<\/td>Focus groups approved packaging<\/td>Shoppers couldn\u2019t find product<\/td>$50M in lost sales<\/td><\/tr>
    Quaker\/Snapple (1994\u201397)<\/td>Brand equity research supported acquisition<\/td>Sold three years later<\/td>$1.4B loss<\/td><\/tr>
    Gap logo (2010)<\/td>Tested and approved<\/td>Reversed within one week<\/td>~$100M<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

    In every case, the research wasn\u2019t wrong about what people said<\/em>. It was wrong about whether what people said predicted what people would do<\/em>. Performed signal mistaken for behavioural truth.<\/p>\n\n\n\n

    What Is Unperformed Evidence?<\/h2>\n\n\n\n

    Above the line is a different kind of evidence. Regulatory filings. Litigation records. Customer complaints filed with regulators. Workforce sentiment captured on platforms the company doesn\u2019t own. Hiring velocity. Patent activity. Government procurement records. Financial disclosures with footnotes nobody reads. Customer language nobody solicited.<\/p>\n\n\n\n

    These signals share a common property: they exist because something happened, not because someone was asked.<\/strong><\/p>\n\n\n\n

    A company doesn\u2019t choose what the SEC records about its material risks. A litigation record reflects a dispute the company would prefer not to publicize. A CFPB complaint is filed by a customer who is frustrated, specific, and not acting in the company\u2019s best interest. A Glassdoor review is written by an employee for an audience of future employees, not for the company\u2019s PR team. A patent filing reveals what a company is actually building, not what it says it is building.<\/p>\n\n\n\n

    Are Unperformed Sources Free of Bias?<\/h3>\n\n\n\n

    No. The obvious counter is true. Unperformed evidence has its own distortions. Glassdoor skews toward exits. Litigation records skew toward disputes. Regulatory filings are written by lawyers to minimize exposure, not to reveal the truth.<\/p>\n\n\n\n

    That is real. It is also a different problem.<\/p>\n\n\n\n

    Performed signal is shaped for the person consuming it<\/em> \u2014 for the survey-giver, the analyst, the consultant, the buyer. Unperformed sources are shaped by their own pressures, and those pressures are not aligned with making you, the analyst, see a particular thing<\/em>. The biases are orthogonal to your investigation.<\/p>\n\n\n\n

    Performed signal is correlated noise. Every source biased in the same direction, toward the audience the data was produced for.<\/p>\n\n\n\n

    Unperformed evidence is uncorrelated noise. Each source biased differently, by pressures that have nothing to do with you.<\/p>\n\n\n\n

    You can triangulate uncorrelated noise. You cannot triangulate correlated noise. This is why above-the-line evidence converges on something real, while below-the-line evidence converges on a polished consensus that dissolves the moment behaviour is observed.<\/p>\n\n\n\n

    Why Didn\u2019t the Research Industry Fix This in 87 Years?<\/h2>\n\n\n\n

    The easy answer is technical: the data infrastructure didn\u2019t exist. SEC databases weren\u2019t machine-readable until recently. Glassdoor didn\u2019t exist until 2007. Regulatory filings weren\u2019t cross-referenceable at scale until APIs and modern search made it possible. There is some truth to this. The infrastructure to operate above the line at speed is genuinely new.<\/p>\n\n\n\n

    It isn\u2019t the full answer.<\/p>\n\n\n\n

    Samuelson published in 1938. Orne published in 1962. The behavioural science on stated versus revealed preference has been accumulating for 88 years. The industry knew. The methodologists knew. The academics knew. And the commercial research industry kept selling surveys, kept monetizing the effort heuristic, kept charging six figures for instruments that asked people questions and recorded their answers.<\/p>\n\n\n\n

    The top-right of the chart wasn\u2019t empty because the idea was missing. It was empty because the incentives<\/em> were missing.<\/p>\n\n\n\n

    There was no commercial reason to dismantle a business model that worked. The buyers wanted political cover. The sellers wanted high-margin engagements. The methodology was wrong, the methodology was profitable, and the profitability was self-reinforcing.<\/p>\n\n\n\n

    What changed is that the cost of operating above the line dropped to the point where someone outside the incumbent industry could build there. The incumbents did not have a change of heart. The cost structure that protected them collapsed.<\/p>\n\n\n\n

    The implication:<\/strong> the industry had 87 years to fix this. It didn\u2019t fix it because it didn\u2019t need to. It is being fixed now because the data became cheap enough that fixing it is possible from the outside.<\/p>\n\n\n\n

    What Does Perception Gap Intelligence Actually Look Like?<\/h2>\n\n\n\n

    The chart is doing one thing: making this argument visible in two seconds, without prose.<\/p>\n\n\n\n

    Monopoly sits in the top-right because that\u2019s the architecture I\u2019m building. Fast access to unperformed evidence. Regulatory filings, litigation records, workforce sentiment, customer language, financial disclosures, patent activity, procurement records \u2014 pulled from fifty-plus sources the company doesn\u2019t control, didn\u2019t commission, and can\u2019t edit, synthesized into a perception gap analysis in minutes rather than months.<\/p>\n\n\n\n

    This isn\u2019t an argument that other tools should disappear. The honest reading of the chart is that every tool below the line is useful for what it was built to do:<\/p>\n\n\n\n