Building and Running Scientific Institutions

On the design of research machines

2021-08-24 — 2025-07-27

adaptive
bounded compute
collective knowledge
distributed
economics
how do science
incentive mechanisms
institutions
learning
mind
networks
provenance
sociology
standards

Science is AFAICT most effective method we have for generating reliable knowledge and driving progress. It is fundamentally a collective endeavour, operating both at a planetary consensus scale and day-to-day in labs. This is a place to collect on how science is shaped by the organisations that do itr The design of these “research machines”—from funding mechanisms to management structures—is a critical variable in the equation of discovery.

This post focuses on institutional design for that collective endeavour. It is a companion to the post on creating scientific knowledge, which explores the messy, social, and philosophical process of how discoveries are actually made. The best institutions are those that create the conditions for a healthy process of discovery to flourish.

There seems to me to be a sudden surge of interest in scientific institutions? Much of it is driven by the intertwined fields of metascience and progress studies, which seek to turn the tools of science back onto itself. Also, I suspect there has been a public upswell in interest in science due to crises of confidence in Big Pharma, and vaccines, disputes over climate modelling adn so on; this is also entwined in culture wars.

Figure 1

1 The Metascience and Progress Studies Perspective

Metascience and progress studies analyze how we can improve the process of discovery by studying its history, sociology, and economics. This includes understanding the complex social dynamics of groupthink and consensus, which are explored more in generating science.

Some relevant snippets for the current topic:

2 Lessons from the Great Research Labs of the Past

Some of the 20th century’s most significant technological leaps emerged from unique institutional experiments. By studying these historical case studies, we might hope to extract enduring principles of creative collaboration and innovation management. That said, a more statistically valid approach would be to also have a long list of awful labs of the past, because learning a classifier from only positive examples is doomed.

  • Bell Labs: The archetypal idea factory. Explored in The Idea Factory (Gertner 2013) and The Genesis of Technoscientific Revolutions (Narayanamurti and Tsao 2021). Also, see How did places like Bell Labs know how to ask the right questions?.
  • Xerox PARC: The birthplace of the personal computer, detailed in Dealers of Lightning (Hiltzik 2000).
  • Tuxedo Park: A secretive community of scientists who developed critical radar technology during WWII (Conant 2003).
  • RAND Corporation: A case study in applying systematic analysis to complex problems. See When RAND Made Magic in Santa Monica.
  • China Lake: The famously creative culture that developed the Sidewinder missile is analyzed in Sidewinder: Creative Missile Development at China Lake (Westrum 2013).
  • Du Bois’s Atlanta University: As a sharp contrast to models that emphasize competition and individual genius, W.E.B. Du Bois organized a highly centralized, collaborative, and mission-oriented research program to study Black life in America. This relatively under-explored model of holistic, planned research is detailed in (Bright 2023).
  • How did NASA get to the moon? Franklin Foer in the man who ate NASA describes a brief history of the baton passing between NASA and SpaceX, but I would like to know how NASA function in the first place.
  • The original Trinity project would also be worth looking in to
  • General analysis on “great groups” can be found in Organizing Genius (Bennis and Biederman 1998).
  • I also started a notebook on highly effective teams.

3 Skunkworks

Figure 2

There is a whole literature of managing innovative/disruptive projects. Scott Locklin, with his signature blend of under-sourced passive-aggressive assertions and insightful contrarian perspective, mentions a particularly interesting era of innovative engineering and project management: the mid-C20 development of a guided missile as a skunk-work intrapreneurial project, headed by some character called Bill McLean. The tale is worth reading. See The Sidewinder Story, Westrum (2013) and Lenfle (2014).

Interesting quote:

How to squelch genius according to Bill McLean:

  1. Coordinate work carefully to avoid duplication: Everything new can be made to look like something we have done before, or are now doing.
  2. Keep the check reins tight; define mission clearly; follow regulations: Nothing very new will ever get a chance to be inserted.
  3. Concentrate on planning and scheduling, and insist on meeting time scales: New, interesting ideas may not work and always need extra time.
  4. Ensure full output by rigorous adherence to scheduled workday: Don’t be late. The creative man sometimes remembers his new ideas, but delay in working on them helps to dissipate them.
  5. Insist that all plans go through at least three review levels before starting work. Review weeds out and filters innovation. More levels will do it faster, but three is adequate, particularly if they are protected from exposure to the enthusiasm of innovator. Insist on only written proposals.
  6. Optimise each component to ensure that each, separately, be as near perfect as possible: This leads to a wealth of “sacred” specifications which will be supported in the mind of the creative man by the early “believe teacher” training. He will the reject any pressure to depart from his specifications.
  7. Centralize as many functions as possible: This creates more review levels and cuts down on direct contact between people.
  8. Strive to avoid mistakes: This increases the filter action of reviews.

4 Government research labs

Research is a public good. Traditionally we have needed to resort to governments to intervene to provided under-supplied public goods. Ergo: there is probably a role for the state in science. How do governments do as science organizations?

TODO

4.1 The DARPA Model

The Defense Advanced Research Projects Agency (DARPA) has an influential model for fostering breakthrough innovation. Its structure of empowered, temporary program managers with a high tolerance for failure is a recurring theme in discussions about improving research.

5 Universities

I should probably have a huge amount to say about universities, having spent much time in them. The connection between teaching and research is intuitively important, both in the positive sense (education and research are natural complements) and int he negative sense (universities become dependent on high student fees to cross-subsidise research leading to weird distortions).

Those are my vibes. But I have little concrete or quantitative to say yet because I have not done the work. TODO.

6 Emerging Models: The New Research Landscape

In response to the perceived shortcomings of traditional academia and corporate R&D, a new ecosystem of research organizations is emerging. Samuel Arbesman calls these “overedge” organizations and collects examples of them.

6.1 Focused Research Organizations (FROs)

FROs are mission-driven non-profits designed to tackle a specific challenge not well-served by existing funding structures.

6.2 Decentralized Science (DeSci)

DeSci aims to build public infrastructure for science using web3 tools, hoping to make funding, peer review, and data sharing more transparent and permissionless.

6.3 New Organizational Structures (Studios, Non-profits, etc.)

A Cambrian explosion of other models are being explored, from small, agile studios to new takes on the non-profit model.

7 Core Principles of Research Organization

Underlying the design of any institution are core principles about how to best manage people, money, and ideas.

7.1 Funding and Incentives

How we fund science determines what science gets done. Many reformers focus on changing the incentive landscape, from calls to Fund People Not Projects (Ioannidis 2011) to new models like crowdfunded research. This also involves a critique of the pervasive and often counter-productive “application culture” that dominates modern science, a topic explored in creating scientific knowledge.

7.2 Management, Freedom, and Culture

The day-to-day environment of a research organization is paramount. This includes granting autonomy, managing work effectively, and building the right culture.

8 Balancing Consensus and Contrarianism

A central challenge for any scientific institution is fostering a culture that allows for productive diversity of thought. An organization must be able to explore novel, contrarian ideas while still being able to eventually form a reliable consensus and act on it. Striking this balance means resisting the pull of institutional groupthink and vested interests without succumbing to juvenile contrarianism. This is an obsession in creating scientific knowledge; difficulties of distinguishing crackpots from pioneers are central to civilisation as a whole but science has that as the main thing.

  • Scientific Freedom: The Elixir of Civilization (Braben 2020)

9 Further Resources

Collections, lists, and people to follow in this space.

10 References

Arbesman, and Christakis. 2011. Eurekometrics: Analyzing the Nature of Discovery.” PLoS Comput Biol.
Bazzoli. 2022. Open Science and Epistemic Pluralism: A Tale of Many Perils and Some Opportunities.” Industrial and Organizational Psychology.
Bennis, and Biederman. 1998. Organizing Genius: The Secrets of Creative Collaboration.
Boulton. 2021. Science as a Global Public Good.”
Braben. 2020. Scientific Freedom: The Elixir of Civilization.
Bright. 2023. Du Bois on the Centralised Organisation of Science.” In Pluralising Philosophy’s Past.
Conant. 2003. Tuxedo Park : A Wall Street Tycoon and the Secret Palace of Science That Changed the Course of World War II.
Domain, Esanu, and Uhlir. 2003. Scientific Knowledge as a Global Public Good: Contributions to Innovation and the Economy.” In The Role of Scientific and Technical Data and Information in the Public Domain: Proceedings of a Symposium.
Gertner. 2013. The Idea Factory: Bell Labs and the Great Age of American Innovation.
Hamming, and Victor. 2020. The Art of Doing Science and Engineering: Learning to Learn.
Hertz, Romand-Monnier, Kyriakopoulou, et al. 2016. Social influence protects collective decision making from equality bias.” Journal of Experimental Psychology. Human Perception and Performance.
Hiltzik. 2000. Dealers of Lightning: Xerox PARC and the Dawn of the Computer Age.
Ioannidis. 2011. Fund People Not Projects.” Nature.
Lakatos. 1980. The Methodology of Scientific Research Programmes: Volume 1 : Philosophical Papers.
Lenfle. 2014. Toward a Genealogy of Project Management: Sidewinder and the Management of Exploratory Projects.” International Journal of Project Management.
Mandt, Seetharam, Cheng, et al. 2020. Federal R&D Funding: The Bedrock of National Innovation.” MIT Science Policy Review (blog).
Narayanamurti, and Tsao. 2021. The Genesis of Technoscientific Revolutions: Rethinking the Nature and Nurture of Research.
———. 2024. How Technoscientific Knowledge Advances: A Bell-Labs-Inspired Architecture.” Research Policy.
O’Connor, and Weatherall. 2017. Scientific Polarization.” European Journal for Philosophy of Science.
O’Connor, and Wu. 2021. How Should We Promote Transient Diversity in Science?
Rekdal. 2014. Academic Urban Legends.” Social Studies of Science.
Vazire. 2017. Our Obsession with Eminence Warps Research.” Nature News.
Vázquez, Oliveira, Dezsö, et al. 2006. Modeling Bursts and Heavy Tails in Human Dynamics.” Physical Review E.
Waldrop. 2018. The Dream Machine.
Westrum. 2013. Sidewinder: Creative Missile Development at China Lake.
Yin, Dong, Wang, et al. 2021. Science as a Public Good: Public Use and Funding of Science.” Working Paper. Working Paper Series.