Design of multi-agent systems
Distributed sensing, swarm sensing, adaptive social learning
October 13, 2014 — April 13, 2025
agents
bounded compute
collective knowledge
concurrency hell
distributed
economics
edge computing
extended self
game theory
incentive mechanisms
machine learning
networks
swarm
Designing swarms and societies to get things done. e.g. probability collectives (are they a thing?) and the nature-inspired algorithms (ant colonies, particle swarms, that one based on choirs…), distributed consistency…
1 Local versus global design
I’m interested in the constraints on what global organising can be done using local information. There are formal approaches to this, e.g. H. Wang and Rubenstein (2020), and more casual, fun ones like Mordvintsev et al. (2020), who use Cellular Automata as a case study.
2 Collective utility function design
If you have autonomous agents that need to cooperate, how do you design their private utility? Although this looks a bit like collective decisions, I am thinking of more incentive design-oriented questions, but maybe one where we get to choose the utility function, not just the mechanism. Maybe an inverse collective action problem.
3 Tooling
3.1 Neural MMO
A large game environment (Suárez et al. 2019; Suarez 2024; Suárez et al. 2023). Claim to fame: leveraging gaming-industry technology to provide a large open world for agent interaction.
3.2 AgentScope
- modelscope/agentscope: Start building LLM-empowered multi-agent applications in an easier way.
- Welcome to AgentScope’s documentation! — AgentScope Doc documentation
AgentScope is an innovative multi-agent platform designed to empower developers to build multi-agent applications with large-scale models. It features three high-level capabilities:
🤝 Easy-to-Use: Designed for developers, with fruitful components, comprehensive documentation, and broad compatibility. Besides, AgentScope Workstation provides a drag-and-drop programming platform and a copilot for beginners of AgentScope!
✅ High Robustness: Supporting customised fault-tolerance controls and retry mechanisms to enhance application stability.
🚀 Actor-Based Distribution: Building distributed multi-agent applications in a centralised programming manner for streamlined development.
AgentScope provides a list of
ModelWrapperto support both local model services and third-party model APIs.
Pan et al. (2024)
4 Graph topology
How do we connect agents together in our multi-agent system? If the graph topology of connections is not 1:N but something more complicated, what happens to learnability? What if you need to infer something while controlling for graph topology? I have arbitrarily filed that under inference on social graphs and network economics.
5 As metaphor for human systems
See wisdom of crowds versus groupthink, or possible weaponised social media.
6 Incoming
- 大トロ, Collective Intelligence for Deep Learning: A Survey of Recent Developments
- Yoav Shoham and Kevin Leyton-Brown’s textbook, Multiagent Systems Algorithmic, Game-Theoretic, and Logical Foundations is downloadable
- Distributed information processing in biological and computational systems
- Yujian, Digest: Consensus Filters is a quick intro to signal analysis using consensus filters
- Growing Neural Cellular Automata looks at differentiable automata on a grid for prettiness.
- Artificial Communication: How Algorithms Produce Social Intelligence
7 References
Acemoglu, and Ozdaglar. 2011. “Opinion Dynamics and Learning in Social Networks.” Dynamic Games and Applications.
Akbarpour, and Jackson. 2018. “Diffusion in Networks and the Virtue of Burstiness.” Proceedings of the National Academy of Sciences.
Akyildiz, Su, Sankarasubramaniam, et al. 2002. “A Survey on Sensor Networks.” Communications Magazine, IEEE.
Barfoot. 2020. “Fundamental Linear Algebra Problem of Gaussian Inference.”
Bianchi, and Jakubowicz. 2013. “Convergence of a Multi-Agent Projected Stochastic Gradient Algorithm for Non-Convex Optimization.” IEEE Transactions on Automatic Control.
Bieniawski, and Wolpert. 2004. “Adaptive, Distributed Control of Constrained Multi-Agent Systems.” In Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems-Volume 3.
Bishop, and Doucet. 2014. “Distributed Nonlinear Consensus in the Space of Probability Measures.” IFAC Proceedings Volumes, 19th IFAC World Congress,.
Cattivelli, Lopes, and Sayed. 2008. “Diffusion Recursive Least-Squares for Distributed Estimation over Adaptive Networks.” IEEE Transactions on Signal Processing.
Cattivelli, and Sayed. 2009. “Diffusion LMS Strategies for Distributed Estimation.” IEEE Transactions on Signal Processing.
———. 2010. “Diffusion Strategies for Distributed Kalman Filtering and Smoothing.” IEEE Transactions on Automatic Control.
Chen, and Sayed. 2012. “Diffusion Adaptation Strategies for Distributed Optimization and Learning over Networks.” IEEE Transactions on Signal Processing.
Codenotti, and Varadarajan. 2004. “Efficient Computation of Equilibrium Prices for Markets with Leontief Utilities.” In ICALP.
Critch, Dennis, and Russell. 2022. “Cooperative and Uncooperative Institution Designs: Surprises and Problems in Open-Source Game Theory.”
Dafoe, A., Bachrach, Hadfield, et al. 2021. “Cooperative AI: machines must learn to find common ground.” Nature.
Dafoe, Allan, Hughes, Bachrach, et al. 2020. “Open Problems in Cooperative AI.”
Degroot. 1974. “Reaching a Consensus.” Journal of the American Statistical Association.
Deng, Papadimitriou, and Safra. 2002. “On the Complexity of Equilibria.” In Proceedings of the Thiry-Fourth Annual ACM Symposium on Theory of Computing. STOC ’02.
Di Lorenzo, and Sayed. 2013. “Sparse Distributed Learning Based on Diffusion Adaptation.” IEEE Transactions on Signal Processing.
Dong, Li, Yang, et al. 2024. “Egoism, Utilitarianism and Egalitarianism in Multi-Agent Reinforcement Learning.” Neural Networks.
Freeman, Yang, and Lynch. 2006. “Stability and Convergence Properties of Dynamic Average Consensus Estimators.” In 2006 45th IEEE Conference on Decision and Control.
Galesic, Barkoczi, Berdahl, et al. 2022. “Beyond Collective Intelligence: Collective Adaptation.”
Galesic, Barkoczi, and Katsikopoulos. 2018. “Smaller Crowds Outperform Larger Crowds and Individuals in Realistic Task Conditions.” Decision.
Hadfield-Menell, and Hadfield. 2018. “Incomplete Contracting and AI Alignment.”
Hammond, Chan, Clifton, et al. 2025. “Multi-Agent Risks from Advanced AI.”
Ha, and Tang. 2022. “Collective Intelligence for Deep Learning: A Survey of Recent Developments.” Collective Intelligence.
Ho, Kastner, and Wong. 1978. “Teams, Signaling, and Information Theory.” IEEE Transactions on Automatic Control.
Hong, and Page. 2004. “Groups of Diverse Problem Solvers Can Outperform Groups of High-Ability Problem Solvers.” Proceedings of the National Academy of Sciences.
Ikeda. 1989. “Decentralized Control of Large Scale Systems.” In Three Decades of Mathematical System Theory: A Collection of Surveys at the Occasion of the 50th Birthday of Jan C. Willems. Lecture Notes in Control and Information Sciences.
Kayaalp, Bordignon, Vlaski, et al. 2021. “Hidden Markov Modeling over Graphs.” arXiv:2111.13626 [Cs, Eess].
Lalitha, Javidi, and Sarwate. 2014. “Social Learning and Distributed Hypothesis Testing.” arXiv:1410.4307 [Cs, Math, Stat].
Lee, Leibo, An, et al. 2022. “Importance of prefrontal meta control in human-like reinforcement learning.” Frontiers in Computational Neuroscience.
Linial. 1994. “Game-Theoretic Aspects of Computing.” In Handbook of Game Theory with Economic Applications.
Lopes, and Sayed. 2007. “Incremental Adaptive Strategies over Distributed Networks.” IEEE Transactions on Signal Processing.
———. 2008. “Diffusion Least-Mean Squares over Adaptive Networks: Formulation and Performance Analysis.” IEEE Transactions on Signal Processing.
Mann, and Helbing. 2016. “Minorities Report: Optimal Incentives for Collective Intelligence.” arXiv:1611.03899 [Cs, Math, Stat].
Mateo, Horsevad, Hassani, et al. 2019. “Optimal Network Topology for Responsive Collective Behavior.” Science Advances.
Mordvintsev, Randazzo, Niklasson, et al. 2020. “Growing Neural Cellular Automata.” Distill.
Ohsawa. 2021. “Unbiased Self-Play.” arXiv:2106.03007 [Cs, Econ, Stat].
Olfati-Saber, R. 2005. “Distributed Kalman Filter with Embedded Consensus Filters.” In 44th IEEE Conference on Decision and Control, 2005 and 2005 European Control Conference. CDC-ECC ’05.
Olfati-Saber, R. 2006. “Flocking for Multi-Agent Dynamic Systems: Algorithms and Theory.” Automatic Control, IEEE Transactions on.
Olfati-Saber, R., Fax, and Murray. 2007. “Consensus and Cooperation in Networked Multi-Agent Systems.” Proceedings of the IEEE.
Olfati-Saber, Reza, Franco, Frazzoli, et al. 2006. “Belief Consensus and Distributed Hypothesis Testing in Sensor Networks.” In Networked Embedded Sensing and Control.
Pan, Gao, Xie, et al. 2024. “Very Large-Scale Multi-Agent Simulation in AgentScope.”
Qi, Ban, and He. 2023. “Graph Neural Bandits.” In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining.
Ren, and Beard. 2005. “Consensus Seeking in Multiagent Systems Under Dynamically Changing Interaction Topologies.” Automatic Control, IEEE Transactions on.
Samuelson. 2001. “Analogies, Adaptation, and Anomalies.” Journal of Economic Theory.
Sayed, Ali. 2014. “Adaptation, Learning, and Optimization over Networks.” Foundations and Trends® in Machine Learning.
Sayed, Ali H. 2014. “Adaptive Networks.” Proceedings of the IEEE.
Spanos, Olfati-Saber, and Murray. 2005. “Dynamic Consensus on Mobile Networks.” In IFAC World Congress.
Stiglitz. 2006. “The Contributions of the Economics of Information to Twentieth Century Economics.” The Quarterly Journal of Economics.
Suarez. 2024. “Neural MMO: Massively Multiagent Simulation and Learning.”
Suárez, Du, Isola, et al. 2019. “Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents.”
Suárez, Isola, Choe, et al. 2023. “Neural MMO 2.0: A Massively Multi-Task Addition to Massively Multi-Agent Learning.”
Tennant, Hailes, and Musolesi. 2023. “Modeling Moral Choices in Social Dilemmas with Multi-Agent Reinforcement Learning.” In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence.
Tumer, and Wolpert. 2004. “Coordination in Large Collectives- Chapter 1.” In.
Wang, Shengdi, and Dekorsy. 2020. “A Factor Graph-Based Distributed Consensus Kalman Filter.” IEEE Signal Processing Letters.
Wang, Hanlin, and Rubenstein. 2020. “Shape Formation in Homogeneous Swarms Using Local Task Swapping.” IEEE Transactions on Robotics.
Wolpert, David H. 2006a. “Advances in Distributed Optimization Using Probability Collectives.” Advances in Complex Systems.
———. 2006b. “Information Theory — The Bridge Connecting Bounded Rational Game Theory and Statistical Physics.” In Complex Engineered Systems. Understanding Complex Systems.
Wolpert, David H, Bieniawski, and Rajnarayan. 2011. “Probability Collectives in Optimization.”
Wolpert, David H, and Lawson. 2002. “Designing Agent Collectives for Systems with Markovian Dynamics.” In.
Wolpert, David H., and Tumer. 1999. “An Introduction to Collective Intelligence.” arXiv:cs/9908014.
Wolpert, David H, Wheeler, and Tumer. 1999. “General Principles of Learning-Based Multi-Agent Systems.” In.
———. 2000. “Collective Intelligence for Control of Distributed Dynamical Systems.” EPL (Europhysics Letters).
Ye. 2008. “A Path to the Arrow–Debreu Competitive Market Equilibrium.” Mathematical Programming.
Zhang, and Zhu. 2017. “Game-Theoretic Design of Secure and Resilient Distributed Support Vector Machines with Adversaries.” arXiv:1710.04677 [Cs, Stat].
Zhao, and Sayed. 2014. “Asynchronous Adaptation and Learning over Networks — Part I: Modeling and Stability Analysis.” arXiv:1312.5434 [Cs, Math].
Zivojevic, Delalic, Raca, et al. 2021. “Distributed Weighted Least-Squares and Gaussian Belief Propagation: An Integrated Approach.” Preprint.