Stochastic parrotology

There is some kind of thought-experiment fight triggered by the pre-training heavy life-cycle of LLMs. This had led to two different families of vibes about artificial intelligence. On one side, we have the Stochastic Parrots thesis: the idea that no matter how fluent a model becomes, it is fundamentally just mimicking the statistical properties of its training data—a “blurry JPEG” of the web, forever trapped in the Chinese Room of syntax without semantics. On the other side is the growing empirical reality of models that seem to reason, code, and strategize in ways that defy simple statistical explanation.

We could formalise this and make it precise, and possibly resolve the thesis (several theses) that are wrapped up here. If we did so, it would please me to call field of inquiry Stochastic Parrotology: the rigorous study of what can (and cannot) be learned from pre-training on observational data alone.

Recent history of the debate: As Emily Bender and Timnit Gebru argued in their seminal paper (Bender et al. 2021), without communicative intent or grounding in the physical world, these models may be nothing more than haphazard stitchers of linguistic form—parrots that don’t know what they are saying. This echoes John Searle’s famous “Chinese Room” argument: manipulating symbols correctly is not the same as understanding them.

Jacob Browning and Yann LeCun reinforce this in AI And The Limits Of Language, arguing that language is a low-bandwidth, lossy compression of human experience. In their view, “shallow understanding” is the ceiling for any system trained on text alone because it lacks the non-linguistic “know-how” the muscle memory of navigating physical reality—that grounds human intelligence. To them, an LLM is a mirror: it reflects our depth back at us, but is itself only a centimeter thick.

But there is a counter-current suggesting that “parroting” the structure of language might force a model to learn the structure of the world. In Blurry JPEG or Frozen Concentrate, Lance Fortnow (riffing on Ted Chiang) suggests that what these models are doing is approximating the Kolmogorov Structure Function—finding the smallest program that could generate the set of all possible human articles. If the most efficient way to compress the data of the world is to learn the rules of the world, then “mere prediction” becomes a path to understanding.

Similarly, Deontologistics argues in On Post-Searlean Critiques of LLMs that we shouldn’t be so quick to dismiss the “game” of language. Drawing on inferentialism, they suggest that being “entangled” in the social web of giving and asking for reasons might be enough to constitute a form of meaning, even without a body. If a model can reliably play the “Inaccessible Game” of information exchange—a concept explored by Neil Lawrence in Perpetual Motion, Superintelligence and the Inaccessible Game—it may be developing internal structures that are functionally indistinguishable from world models.

So, are we looking at a Benderian Octopus—a creature that can trick us until a bear attacks and it fails to hand us a stick? Are we witnessing the emergence of a Conduit (to use Michael Reddy’s metaphor) that has somehow become aware of the content it carries?

To move beyond vibes, we need to operationalize these debates. We must ask specific, falsifiable questions about what happens inside the black box during pre-training.

If we strip away the anthropomorphism, the debate seems to be about at least three things that I can discern. Can we mathematically and empirically demonstrate whether the following can arise purely from observing the text of others?

1. Can models learn intervention distributions from pre-training?
2. Can models learn world models from pre-training?
3. Can models learn agency from pre-training?

Let’s unpack these.