Economics of large language models

Various questions about economics of social changes wrought by ready access to LLMs, the latest generation of automation. This is the major “short”-“medium” effect whatever those words mean. Longer-sighted persons might also care about whether AI will replace us with grey goo.

Thought had in conversation with Richard Scalzo about Smith (2022):

“Snowmobile or bicycle?”

My mental model for disruptive technology is always in reference to snowmobiles (Pelto 1973). and an aside from ?Steve Jobs? that the PC should be a bicycle for the mind. I am interested in knowing what is more bicycle for the mind (democratising, enabling even underdogs) and what is a snowmobile (cementing disparities, increasing returns to incumbents).

Economics of collective intelligence

How do foundation models/ large language models change the economics of knowledge production? Of art production? To a first order approximation (valid at 03/2023) LLMs provide a way of massively compressing collective knowledge and synthesising the bits I need on demand. They are not yet directly generating novel knowledge (whatever that means). But they do seem to be pretty good at being “nearly as smart as everyone on the internet combined”. There is no sharp boundary between these ideas, clearly.

Deploying these models will test various hypotheses about how much of collective knowledge depends upon our participating in boring boilerplate grunt work, and what incentives are necessary to encourage us to produce and share our individual contributions to that collective intelligence.

Here is where I formulate those.

Do we need to spend time doing the boring stuff that LLMs automate?

Think of Matt Might’s iconic illustrated guide to a Ph.D..

Imagine a circle that contains all of human knowledge:

By the time you finish elementary school, you know a little:

A master’s degree deepens that specialty:

Reading research papers takes you to the edge of human knowledge:

You push at the boundary for a few years:

Until one day, the boundary gives way:

And, that dent you've made is called a Ph.D.:

Here’s my question: Does the new map look something like this? If so is that a problem?

Now OpenAI have shipped an LLM and where is the border?

Are researchers now incentivised to hide special expertise since LLMs will more readily automate published expertise?

I’m shakier on this hypothesis. TBC

What to spend my time on

Economics of production at a microscopic, individual scale:

GPT and the Economics of Cognitively Costly Writing Tasks

To analyze the effect of GPT-4 on labor efficiency and the optimal mix of capital to labor for workers who are good at using GPT versus those who aren’t when it comes to performing cognitively costly tasks, we will consider the Goldin and Katz modified Cobb-Douglas production function

Content landscape, culture landscape, spamularity, dark forest, textpocalypse

Hmm let me see; devastate, develop or devour?

Charlie Stross’s 2010 Spamularity stuck with me:

We are currently in the early days of an arms race, between the spammers and the authors of spam filters. The spammers are writing software to generate personalized, individualized wrappers for their advertising payloads that masquerade as legitimate communications. The spam cops are writing filters that automate the process of distinguishing a genuinely interesting human communication from the random effusions of a ’bot. And with each iteration, the spam gets more subtly targeted, and the spam filters get better at distinguishing human beings from software, in a bizarre parody of the imitation game popularized by Alan Turing (in which a human being tries to distinguish between another human being and a piece conversational software via textual communication) — an early ad hoc attempt to invent a pragmatic test for artificial intelligence.

We have one faction that is attempting to write software that can generate messages that can pass a Turing test, and another faction that is attempting to write software that can administer an ad-hoc Turing test. Each faction has a strong incentive to beat the other. This is the classic pattern of an evolutionary predator/prey arms race: and so I deduce that if symbol-handling, linguistic artificial intelligence is possible at all, we are on course for a very odd destination indeed—the Spamularity, in which those curious lumps of communicating meat give rise to a meta-sphere of discourse dominated by parasitic viral payloads pretending to be meat…

Maggie Appleton’s commentary on Dark Forest theory of the Internet by Yancey Strickler is another neat framing:

The dark forest theory of the web points to the increasingly life-like but life-less state of being online. Most open and publicly available spaces on the web are overrun with bots, advertisers, trolls, data scrapers, clickbait, keyword-stuffing “content creators,” and algorithmically manipulated junk.

It’s like a dark forest that seems eerily devoid of human life—all the living creatures are hidden beneath the ground or up in trees. If they reveal themselves, they risk being attacked by automated predators.

Humans who want to engage in informal, unoptimised, personal interactions have to hide in closed spaces like invite-only Slack channels, Discord groups, email newsletters, small-scale blogs, and digital gardens. Or make themselves illegible and algorithmically incoherent in public venues.

I feel like I’m going to lose this battle, but for the record, I despise the term “textpocalypse”.

Information landscape

Fake news, credibility, cryptographic verification of provenance, etc. TBC

Economic disparity and LLMs

PR, hype, marketing implications

George Hosu, in a short aside, highlights the incredible marketing advantage of AI:

People that failed to lift a finger to integrate better-than-doctors or work-with-doctors supervised medical models for half a century are stoked at a chatbot being as good as an average doctor and can’t wait to get it to triage patients

The Tweet that Sank $100bn

Google’s Bard was undone on day two by an inaccurate response in the demo video where it suggested that the James Webb Space Telescope would take the first images of exoplanets. This sounds like something the JWST would do but it’s not at all true So one tweet from an astrophysicist sank Alphabet’s value by 9%. This says a lot about how a) LLMs are like* being at the pub with friends, it can say things that sound plausible and true enough and no one really needs to check because who cares? Except we do because this is science, not a lads’ night out and b) the insane speculative volatility of this AI bubble that the hype is so razor thin it can be undermined by a tweet with 44 likes.

I had a wonder if there’s any exploration of the ‘thickness’ of hype. Jack Stilgoe suggested looking at Borup et al which is evergreen but I feel like there’s something about the resilience of hype: Like crypto was/is pretty thin in the scheme of things. High levels of hype but frenetic, unstable and quick to collapse. AI has pretty consistent if pulsating hype gradually growing over the years while something like nuclear fusion is super-thick (at least in the popular imagination) – remaining through decades of not-quite-ready and grasping the slightest indication of success. I don’t know, if there’s nothing specifically on this, maybe I should write it one day.



Andrus, McKane, Sarah Dean, Thomas Krendl Gilbert, Nathan Lambert, and Tom Zick. 2021. AI Development for the Public Interest: From Abstraction Traps to Sociotechnical Risks.” arXiv.
Barke, Shraddha, Michael B. James, and Nadia Polikarpova. 2022. Grounded Copilot: How Programmers Interact with Code-Generating Models.” arXiv.
Bowman, Samuel R. 2023. Eight Things to Know about Large Language Models.” arXiv.
Hall, Lawrence. 1975. The Snowmobile Revolution: Technology and Social Change in the Arctic. Pertti J. Pelto.” Economic Development and Cultural Change 23 (4): 769–71.
Métraux, Alfred. 1956. “A Steel Axe That Destroyed a Tribe, as an Anthropologist Sees It.” The UNESCO Courier: A Window Open on the World IX: 26–27.
Pelto, Pertti J. 1973. The snowmobile revolution: technology and social change in the Arctic. Waveland Press.
Shanahan, Murray. 2023. Talking About Large Language Models.” arXiv.
Smith, Justin E. H. 2022. The Internet Is Not What You Think It Is: A History, a Philosophy, a Warning. Princeton: Princeton University Press.

1 comment

I asked ChatGPT to summarize your piece. Since it was not able to "read" the title as it is an image, it misunderstood LLM for "Master of Laws". Hence its first pass summarizing went as follows--verbatim:

"The post is a personal account of the author's experience pursuing a Master of Laws (LL.M) in Economics program. The author discusses the challenges they faced, such as the steep learning curve of economics and the demands of the program. They also highlight the benefits of the program, including exposure to new economic theories, the development of critical thinking skills, and opportunities to engage in policy discussions. The author also provides some advice for prospective LL.M in Economics students, including the importance of prior preparation, the need to develop time-management skills, and the value of networking opportunities."

The metal axe still needs improvement but it will get there with time.

Reply to RHM

GitHub-flavored Markdown & a sane subset of HTML is supported.