Multi-agent RL tooling
Simulation environments and frameworks for multi-agent research
2026-04-10 — 2026-04-10
Wherein various platforms are surveyed, including open-world game engines and social dilemma suites in which agents are evaluated on their capacity to cooperate with previously unseen partners.
Environments and frameworks for multi-agent systems research. See also the opponent shaping page for specific iterated-game setups.
1 Neural MMO
A large, open-world game environment for multi-agent research (Suárez et al. 2019; Suarez 2024; Suárez et al. 2023). Claim to fame: leveraging gaming-industry technology to provide a persistent, large-scale world for agent interaction, going beyond the typical matrix game or grid world.
2 AgentScope
See Pan et al. (2024).
modelscope/agentscope / documentation
A multi-agent platform oriented toward LLM-powered agents, with actor-based distribution and fault tolerance.
3 PettingZoo
PettingZoo is the multi-agent extension of Gymnasium (formerly OpenAI Gym), providing a standard API for multi-agent environments. Covers classic games (chess, Go, connect four), Atari multiplayer, and MPE (multi-particle environments commonly used in cooperative/competitive MARL papers).
4 Melting Pot
DeepMind’s Melting Pot is a suite of social dilemma environments for evaluating multi-agent cooperation and competition. Environments are designed to test generalisation: agents must cooperate with novel partners in scenarios inspired by the collective action literature.