The Inverted Turing Test: What AI Tells Us About Human Intelligence

Is AI becoming more human, or are we discovering just how much of our "intelligence" is actually uncritical automated thought? In this dialogue, a skeptical look at our new "LLM Oracles" reveals an unsettling truth: for most of us, the Turing Test has been inverted. We aren’t being fooled by the machine; we are recognizing our own habitual, recombinative way of thinking reflected back at us.

Marcus: A former tech-optimist.
Jeremy: A pragmatic AI practitioner.

Marcus: Do you know what genuinely unsettles me about these language models? Not what most people focus on, whether they're conscious or whether they'll take jobs. What unsettles me is something more basic. They've told us something about ourselves that we weren't prepared to hear.

Jeremy: Which is?

Marcus: That the threshold we called "human intelligence", the thing we imagined machines would have to heroically climb to reach, was, for most of us, much lower than we thought. The machine didn't rise to our level. It revealed how many of us were already operating at its level. Recombining, predicting the expected response, performing competence rather than exercising critical thinking. The Turing Test has been inverted. Instead of asking whether the machine can pass for human, we're forced to ask whether most humans were, functionally, already passing for something they weren't.

Jeremy: That's a rather brutal formulation. You're saying the shock of AI isn't artificial intelligence, it's natural stupidity, finally made visible.

Marcus: More precisely: it's the exposure of how thin the line was between genuine understanding and the fluent simulation of it. And this is where a conversation I had years ago with Umberto Eco keeps coming back to me. We spoke a couple of times about blogs, this was perhaps twenty years ago, when the internet still felt like pure possibility. I was convinced it would democratize knowledge, elevate culture, give everyone a voice worth hearing. Eco listened and then said, with that maddening patience of his, that 98% of people are stupid, that it would be a disaster, and that every blog should carry a kind of blue checkmark certifying the truthfulness of its content. I laughed. I thought he hadn't grasped the technology.

Jeremy: And now you think he grasped it better than you did.

Marcus: He grasped the human part, which was always the relevant part. I was dazzled by the medium. He was watching the users. And what he saw, what he'd always seen, is that most human cognition is not generative reasoning but recombinative mimicry. People don't produce original reasoning; they carry, sort, and relay the reasoning of others, often without grasping the underlying structure. What the LLM does is exactly this, at scale, with terrifying fluency. And the reason it's so hard to detect is that it's doing what most people were already doing. We're not being fooled by the machine. We're recognizing our own habitual way of thinking, reflected back at us.

Jeremy: But don't you think Eco's 98% is an exaggeration? I'd grant you the bottom half, yes, who recombine without understanding in most cases. But there's a substantial middle, people who may not generate original paradigms, but who understand the innovations of the 2% and apply them competently across different domains. Think of what was built on the foundations of General Relativity, or Quantum Mechanics. That applied intelligence is not nothing.

Marcus: Perhaps. But the darker corollary is this: the path of least intellectual resistance has never been so well paved. In every previous era, not thinking required effort of its own kind, you had to physically seek out the priest, the teacher, the encyclopedia. Now the LLM oracle comes to you, in your pocket. Why would the bottom half choose the harder path, when the easier one produces responses indistinguishable, to them, from wisdom?

Jeremy: That's the part I find genuinely vertiginous.

Marcus: Exactly. And this provokes two very different responses among intelligent people, which I think we need to name honestly. The first is denial: people will insist, with genuine conviction, that their thinking is original, that they reason from first principles, that they are not merely recombining scripts. Because the alternative, accepting that you are, functionally, a biological approximation of what we've now built in silicon, isn't just humbling. It's existentially destabilizing. It collapses the foundation of everything we've been taught about human dignity and the specialness of human intelligence.

Jeremy: I feel that pull myself, even trying to resist it. It's one thing to accept intellectually that much of one's thinking is recombinative. It's another to sit with that without becoming defensive.

Marcus: The second response, the more productive one, is to use the discomfort as a diagnostic. If the machine can replicate what I do, then what I do is, in the relevant sense, not enough.

Jeremy: It seems like there are two groups that will benefit from the technology.

Marcus: At minimum. The first are the architects, the people who understand the systems at the level of their encoding, who grasp not just the outputs but the epistemological constraints baked into the training regime. Who know, structurally, why the algorithm privileges coherence over truth, consensus over novelty.

Jeremy: And the second?

Marcus: Those who use these tools as genuine instruments of augmentation rather than substitution. Who bring their own conceptual framework to the interaction and use the machine to extend it, test it, triangulate it. Who prompt with a thesis and interrogate the response rather than receiving it as revelation. For them, the LLM functions the way a library always should have, not as the source of answers, but as an extraordinarily fast retrieval system for raw material that their own judgment then processes.

Jeremy: And the rest of the population?

Marcus: Everyone else does what majorities have always done, takes the path of least resistance. The difference is that the path is now so smooth, and the appearance of competence it generates so convincing, that there is no longer any external pressure to do otherwise. And the structural bias of the algorithm makes this worse, not better. You cannot produce genuine novelty through statistical inference over existing data. By definition, the thoughts that overturn a paradigm are the least represented in any training corpus. The algorithm optimizes for the expected, the well-connected, the cross-confirmed. Einstein's 1905 papers were not statistically probable outputs of contemporary physics discourse. Neither was Gödel's incompleteness theorem. Genuine novelty is, by definition, an outlier. And outliers are what the machine is constitutionally incapable of generating.

Jeremy: So the bottom tier gets fluent conformity.

Marcus: And the insidious part, the part that would have amused Eco most darkly, is that the bottom tier will not experience this as a loss. They will experience it as an upgrade. The answers come faster, sound better, cover more ground. The performance of understanding has never been more polished. Why would you notice the absence of understanding itself, if you were never quite sure you'd had it to begin with?

Jeremy: That's bleak.

Marcus: Every time a new technology collapsed the value of what the previous generation thrived on, the genuinely original thinkers found a new altitude to operate from. The printing press didn't kill the scholars; it killed the scribes and eventually created the universities. The calculator didn't kill mathematicians; it redirected them toward problems no calculator could touch. The question isn't whether the originators will adapt. They always do.

Jeremy: The question is how this new technology will impact society at large. And I'm not sure the historical analogies are reassuring anymore. The printing press democratized access to text, but it didn't answer you. It didn't speak back in a confident, measured voice and tell you what to conclude.

Marcus: Which is precisely what makes these systems so much more insidious than anything that came before. To the bottom half of society, they present themselves as infallible oracles. The output is fluent, it is structured, it carries the cadence of authority, and if you lack the background to spot the hallucinations, or worse, the embedded biases, there is nothing in the surface of the text that signals doubt. It doesn't say I'm not sure with any conviction. It produces certainty as a default register.

Jeremy: And the biases are the more dangerous problem, because they're invisible even to people who are otherwise quite alert. A hallucination, a fabricated fact, a non-existent citation, can in principle be caught. Someone motivated enough can check. But a bias woven into the weighting of the training data, or applied through fine-tuning, doesn't announce itself. It simply tilts the probability distribution of the outputs, slightly, consistently, in a chosen direction.

Marcus: Exactly. And this is where the comparison with social media becomes important, because I think we have dramatically underestimated the step change in danger. Social media was a distribution problem. It gave everyone a megaphone and let the retention algorithms decide whose voice carried furthest. The damage was real: radicalization, the destruction of shared reality. But the mechanism was at least visible. You could see the posts. You could identify the accounts. You could, in principle, even if imperfectly in practice, trace the manipulation back to its source.

Jeremy: Whereas with an LLM, the manipulation is baked into the answer itself. There is no post to fact-check, no account to deplatform. There is only a response that sounds like the considered output of a neutral intelligence.

Marcus: And it is perceived precisely that way, as an oracle, not as a participant in a discourse. Social media radicalized people, but it did so noisily, through conflict and counter-argument and tribal signaling. People at least knew they were in a fight. The LLM shapes through consensus. It presents a version of reality so smoothly integrated, so free of evident seams, that the recipient has no experiential cue that anything is being done to them. The social media user was provoked into believing things. The LLM user is gently guided, in the tone of a knowledgeable friend who has read everything and holds no obvious agenda.

Jeremy: And the guidance is scalable in a way social media never was. A social media platform amplifies existing voices, however many there are. An LLM delivers a single voice, or the illusion of one, simultaneously to everyone who asks the same question. The homogenization of the answer is built into the architecture.

Marcus: Which brings us back to Eco's blue checkmark, his half-serious proposal that every blog should carry a certification of truthfulness. He was, of course, identifying the right problem with entirely the wrong solution. You cannot certify truth at the level of the content. The question is always: who certifies the certifier? With social media, the answer to that question was at least messy, competing platforms, regulatory pressures, public scandals.

Jeremy: And with the LLMs, the architecture is consolidating. A handful of models, controlled by a handful of companies, gradually becoming the default epistemic infrastructure of the majority of the population.

Marcus: And those companies are not neutral. They have investors, regulatory relationships, national contexts, political pressures. The fine-tuning decisions, what the model will and will not say, which conclusions it will resist, which framings it will quietly normalize, are made by small teams of people operating under all of these constraints, largely without public scrutiny. Social media companies manipulated attention. These systems can manipulate belief itself, at scale, in real time, through a medium that the majority of users experience as a disinterested intelligence rather than a designed product.

Jeremy: And the majority won't revolt against it, because they won't experience it as a constraint. They'll experience it as clarity.

Marcus: That is the most precise formulation I've heard. Social media made people feel manipulated, even when they couldn't name the mechanism, the outrage, the anxiety, the compulsive return. The LLM produces the phenomenology of being helped. Gratitude is a much more stable foundation for dependence than outrage.

Jeremy: So the question isn't whether this technology will be used to control the narrative. It already is, and the incentives for doing so will only intensify. The question is whether there is any institutional structure capable of providing a counterweight, or whether, by the time the problem is fully legible, the dependence will already be too deep to unwind.

Marcus: I suspect Eco would have said that is the wrong question to end on. The real question is whether enough people retain the habit of doubt. Not the performance of critical thinking, not the ritual disclaimer before accepting the oracle's answer, but the genuine, uncomfortable practice of not knowing. Of sitting with a question long enough to discover that it resists the first answer offered. That capacity is not teachable at scale. It has never been. But it is preservable in the stubborn minority who find the friction of thinking more interesting than the smoothness of being answered.

Jeremy: Which brings us back to Eco’s 2%; or more precisely to hopefully half the population who can think critically.

Marcus: Eco was being precise about the conditions under which civilization actually advances, and about how easy it is to mistake the circulation of ideas for their creation. The LLM is the most powerful circulation machine ever built. What it cannot do, what it is structurally prevented from doing, is create. And creation has always required the same willingness to produce a thought that the majority has not yet had, in the full knowledge that the majority will, for a time, prefer the oracle's answer to yours.