Building and Running Scientific Institutions
On the design of research machines
2021-08-24 — 2025-07-27
Wherein institutional designs for collective discovery are examined, and the DARPA model of empowered temporary program managers and the rise of focused research organizations are set forth with attention to funding incentives
Science is, AFAICT, the most effective method we have for generating reliable knowledge and driving progress. It is fundamentally a collective endeavour, operating both at a planetary consensus scale and day-to-day in labs. This is a place to collect thoughts on how science is shaped by the organizations that do it. The design of these “research machines”—from funding mechanisms to management structures—is a critical variable in the equation of discovery.
This post focuses on institutional design for that collective endeavour. It is a companion to the post on creating scientific knowledge, which explores the messy, social, and philosophical process of how discoveries are actually made. The best institutions are those that create the conditions for a healthy process of discovery to flourish.
There seems to me to be a sudden surge of interest in scientific institutions. Much of this interest is driven by the intertwined fields of metascience and progress studies, which seek to turn science’s tools back on itself. Also, I suspect there has been a public upswell in interest in science due to crises of confidence in Big Pharma and vaccines, disputes over climate modelling and so on; this is also entwined in the culture wars.
1 The Metascience and Progress Studies Perspective
Metascience and progress studies analyze how we can improve the process of discovery by studying its history, sociology, and economics. This includes understanding the complex social dynamics of groupthink and consensus, which we explore more in generating science.
Some relevant snippets for the current topic:
- A Vision of Metascience: An Engine of Improvement for the Social Processes of Science (by Michael Nielsen and Kanjun Qiu)
- Accelerating science through evolvable institutions (from Roots of Progress)
- My Metascience 2022 talk on new scalable ‘technoscience’ laboratory designs (by James Phillips)
2 Lessons from the Great Research Labs of the Past
Some of the 20th century’s most significant technological leaps emerged from unique institutional experiments. By studying these case studies, we might extract enduring principles of creative collaboration and innovation management. That said, a more statistically valid approach would be to include a long list of awful labs, because learning a classifier from only positive examples is doomed.
- Bell Labs: The archetypal idea factory, explored in The Idea Factory (Gertner 2013) and The Genesis of Technoscientific Revolutions (Narayanamurti and Tsao 2021). Also, see How did places like Bell Labs know how to ask the right questions?.
- Xerox PARC: The birthplace of the personal computer, detailed in Dealers of Lightning (Hiltzik 2000).
- Tuxedo Park: A secretive community of scientists who developed critical radar technology during WWII (Conant 2003).
- RAND Corporation: A case study in applying systematic analysis to complex problems. See When RAND Made Magic in Santa Monica.
- China Lake: The famously creative culture that developed the Sidewinder missile is analyzed in Sidewinder: Creative Missile Development at China Lake (Westrum 2013).
- Du Bois’s Atlanta University: As a sharp contrast to models that emphasize competition and individual genius, W.E.B. Du Bois organized a highly centralized, collaborative, and mission-oriented research program to study Black life in America. This relatively under-explored model of holistic, planned research is detailed in (Bright 2023).
- How did NASA get to the moon? Franklin Foer, in the man who ate NASA describes a brief history of the baton passing between NASA and SpaceX, but I’d like to know how NASA functioned in the first place.
- The original Trinity project would also be worth looking into
- General analysis on “great groups” can be found in Organizing Genius (Bennis and Biederman 1998).
- I also started a notebook on highly effective teams.
3 Skunkworks
There’s a whole literature on managing innovative/disruptive projects. Scott Locklin, with his signature blend of under-sourced, passive-aggressive assertions and insightful, contrarian perspective, describes a particularly interesting era of innovative engineering and project management: the mid-C20 development of a guided missile as a Skunk Works intrapreneurial project led by a character called Bill McLean. It’s worth reading. See The Sidewinder Story, Westrum (2013) and Lenfle (2014).
Interesting quote:
How to squelch genius according to Bill McLean:
- Coordinate work carefully to avoid duplication: Everything new can be made to look like something we have done before, or are now doing.
- Keep the check reins tight; define mission clearly; follow regulations: Nothing very new will ever get a chance to be inserted.
- Concentrate on planning and scheduling, and insist on meeting time scales: New, interesting ideas may not work and always need extra time.
- Ensure full output by rigorous adherence to scheduled workday: Don’t be late. The creative man sometimes remembers his new ideas, but delay in working on them helps to dissipate them.
- Insist that all plans go through at least three review levels before starting work. Review weeds out and filters innovation. More levels will do it faster, but three is adequate, particularly if they are protected from exposure to the enthusiasm of innovator. Insist on only written proposals.
- Optimise each component to ensure that each, separately, be as near perfect as possible: This leads to a wealth of “sacred” specifications which will be supported in the mind of the creative man by the early “believe teacher” training. He will the reject any pressure to depart from his specifications.
- Centralize as many functions as possible: This creates more review levels and cuts down on direct contact between people.
- Strive to avoid mistakes: This increases the filter action of reviews.
4 Government research labs
Research is a public good. Traditionally we’ve had to rely on governments to provide under-supplied public goods. Ergo: there is probably a role for the state in science. How well do governments perform as science organizations?
TODO
4.1 The DARPA Model
The Defense Advanced Research Projects Agency (DARPA) has an influential model for fostering breakthrough innovation. Its structure of empowered, temporary program managers with a high tolerance for failure recurs in discussions about improving research.
- Why does DARPA work? (by Ben Reinhardt)
- Structure for a Private ARPA (by Ben Reinhardt)
- White paper on how to best structure and fund research organizations for “Immense Challenges” (by Alan Kay) handles ARPA and Xerox-PARC style models.
5 Universities
I should probably have a lot to say about universities, having spent much time in them. The connection between teaching and research is intuitively important, both in the positive sense (education and research are natural complements) and in the negative sense (universities become dependent on high student fees to cross-subsidize research, leading to weird distortions).
Those are my vibes. But I have little concrete or quantitative evidence to offer yet because I haven’t done the work. TODO.
6 Emerging Models: The New Research Landscape
In response to the perceived shortcomings of traditional academia and corporate R&D, a new ecosystem of research organizations is emerging. Samuel Arbesman calls these “overedge” organizations and collects examples of them.
- The Overedge Catalog: The Future of Research Organizations (by Samuel Arbesman), introduced in We need to create and foster new types of scientific organizations
6.1 Focused Research Organizations (FROs)
FROs are mission-driven non-profits designed to tackle a specific challenge not well-served by existing funding structures.
6.2 Decentralized Science (DeSci)
DeSci aims to build public infrastructure for science using web3 tools, hoping to make funding, peer review, and data sharing more transparent and permissionless.
- A Guide to DeSci, the Latest Web3 Movement
- Retroactive Public Goods Funding (An incentive mechanism often discussed in DeSci)
6.3 New Organizational Structures (Studios, Non-profits, etc.)
A Cambrian explosion of other models is being explored, from small, agile studios to new takes on the non-profit model.
- When should an idea that smells like research be a startup? (by Ben Reinhardt)
- Notes on Starting a Scientific Organization (by Sabrina Singh)
- Erika’s quick-start guide to research nonprofits (by Erika Alden DeBenedictis)
- Startup nonprofits (by Mark Lutter)
- Research Studios (by Tom Critchlow)
- DIY research studio (by Patrick Tanguay)
- An orthogonal technology lab (by Matt Webb)
7 Core Principles of Research Organization
Underlying the design of any institution are core principles about how to best manage people, money, and ideas.
7.1 Funding and Incentives
How we fund science determines what science gets done. Many reformers focus on changing the incentive landscape, from calls to Fund People Not Projects (Ioannidis 2011) to new models like crowdfunded research. This also involves a critique of the pervasive and often counterproductive “application culture” that dominates modern science, a topic explored in creating scientific knowledge.
- The argument to Fund People Not Projects (Ioannidis 2011).
- On funding scientific research (by Alan Kay)
- White paper on “fractional scholarship” (by Samuel Arbesman and Jon F. Wilkins)
- Crowdfunded research (by Andy Matuschak)
- Alt VC (by Mark McGranaghan)
7.2 Management, Freedom, and Culture
The day-to-day environment of a research organization is paramount. This includes granting autonomy, managing work effectively, and building the right culture.
- Du Bois’s early thoughts on the topic are explored in Du Bois on the Centralised Organisation of Science (Bright 2023).
- Lessons on the process of discovery from a master practitioner can be found in The Art of Doing Science and Engineering (Hamming and Victor 2020).
- Breaking Logjams in Knowledge Work: Applying operational insights to research.
- The Need for Long-term Research (by Kevin Kelly)
- Tools, Computing, and Research Labs (by James Hill-Khurana)
8 Balancing Consensus and Contrarianism
A central challenge for any scientific institution is fostering a culture that allows for productive diversity of thought. An organization must be able to explore novel, contrarian ideas while still being able to eventually form a reliable consensus and act on it. Striking this balance means resisting the pull of institutional groupthink and vested interests without succumbing to juvenile contrarianism. This is an obsession in creating scientific knowledge; the difficulty of distinguishing crackpots from pioneers is central to civilization as a whole, but science faces that issue most directly.
- Scientific Freedom: The Elixir of Civilization (Braben 2020)
9 Further Resources
Collections, lists, and people to follow in this space.
- Reading material about applied research labs (by Patrick Collison)
- A list of research labs (by Matt Webb)
- Interesting organizational structures (by Ben Reinhardt)
- Eric Gilliam (@eric_is_weird): A researcher who writes about the history and operation of R&D organizations.