Iterated and evolutionary game theory

1 Background: Game theory

See my actual basic game theory post for some more refs.

When people talk about game theory they usually mean the class of mathematically formulated two-player “games”, which are typically not the fun type of games, being much shorter and more brutal than Carcassone or whatever. The most famous one is the Prisoner’s dilemma; You’ve probably run into this one. You and I, co-conspirators, have been arrested by the cops for a crime we did commit, and we are interviewed separately. They offer us each the same choice: “Inform on your buddy and we will let you off lightly”. Obviously we want to have as little time in prison; what should we each do?

There are four possible outcomes:

1. Both defect — You and I both turn informant: We both go to prison for 10 years
2. You defect — You inform and I stay stumm: I go to prison for 10 years, you walk free in 1 year
3. I defect — You stay stumm and I inform: You go to prison for 10 years, I walk free in 1 year
4. Both cooperate — We both stay stumm: We each go to prison for 2 years on a lesser charge

The Prisoner’s dilemma is a classic. There are other games, but this one will stand in for all of them for now. The first lesson is that even though it would be better in terms of total prison time for us both to cooperate, to remain silent, given that you and I do not know if the other one will defect, it makes sense for each of us individually to defect.

This one is a toy model for the kind of cooperative dilemmas we face all the time, especially dilemmas about trust. Should I take credit for that bit of work at the expense of my colleague? Should I take more donuts from the donut tray rather than leave some for my colleagues? Why should my factory refrain from polluting if all my competitors are doing it and lowering their costs?

A lot of people have written about these kinds of games, there are some nice mathematical insights from making all this precise, theorems that you can prove and so forth. It’s a whole industry.

An important sub-field of this game theory thing is the field of iterated games, especially Iterated Prisoner’s Dilemma (“IPD”). This is the field wherein we extend the model a little to encompass the fact that these isolated one-off prisoner’s dilemmas are not a typical situation. In real life, when we interact in society, we meet with people again and again, and moreover there are a lot of people interacting in diverse situations. These are iterated games. Still a simplified model, but a whole bunch of interesting phenomena arise in the marginally more life-like setup.

2 Iterated Prisoner’s Dilemma: Basics and Strategy

• Imagine two friends, Alice and Bob, playing a game repeatedly. In each round, they can either cooperate with each other or betray (defect). The twist is they remember what happened in previous rounds.
• Strategy: This is the plan each player follows in every round. For instance, Alice might adopt the “Always Cooperate” strategy, meaning she decides to cooperate in every round, no matter what Bob does.

3 Population Structure and Interaction Model

• Now, imagine there’s a whole group of people, not just Alice and Bob, playing this game. They might randomly pair up for each round, or maybe they always play against the same few people.
• In some versions of this game, players remember not just past actions but also who they were playing against. So, if Alice played against Bob before, she might change her strategy based on what Bob did last time.

4 Competition and Evolution of Strategies

• In this game, winning isn’t about a single round; it’s about doing well over many rounds. Let’s say Alice’s “Always Cooperate” strategy works well against those who also cooperate, but it might not do well against someone who always defects.
• Strategy Evolution: Imagine a player, Charlie, who notices that “Always Defect” works well initially but then starts losing out to players using “Tit-for-Tat” (a strategy where a player cooperates in the first round and then replicates the opponent’s previous action). Charlie might switch to “Tit-for-Tat” to improve his overall score.
• Over time, the most successful strategies become more common. If “Tit-for-Tat” consistently earns higher scores, more players will start using it.

5 Implications

• This game is a simple way to understand complex real-world interactions. It shows how people (or animals, or even computer programs) might change their behavior based on past experiences and the behavior of others around them. It’s not just about winning a game; it’s about how strategies evolve and how cooperation or competition can emerge in a community.

By using these concrete examples, we can see how the Iterated Prisoner’s Dilemma is a powerful tool for understanding the dynamics of strategy, memory, and evolution in both social and biological contexts.

6 Population-Level Behaviors:

1. Clusters of Cooperation: In a mixed population of different strategies, cooperators can form clusters. If players are more likely to interact with those nearby (like in a spatial IPD), these clusters can protect cooperators from defectors, as they mostly interact with each other.

2. Cycles of Strategies: Sometimes, you’ll see cycles where different strategies rise and fall in prevalence. For instance, if “Always Defect” becomes common, “Tit-for-Tat” might rise in response, as it can protect itself against defectors. But then, a more forgiving strategy might outcompete “Tit-for-Tat” by not retaliating as harshly, and so on.

7 Dependence on Initial Conditions:

• The outcome of an IPD tournament can be highly sensitive to initial conditions. For example, if the game starts with a majority of defectors, cooperative strategies might struggle to gain a foothold. Conversely, if the game starts with many cooperators, they might establish a cooperative norm that resists invasion by defectors.
• The structure of interactions also matters. If players mostly interact with a fixed set of neighbors, it can lead to different dynamics compared to a model where they interact randomly with the entire population.

8 Unintuitive Dynamics:

1. Emergence of Cooperation: It might seem counterintuitive, but cooperation can emerge and stabilize even in a competitive environment. The success of strategies like “Tit-for-Tat” shows that cooperation can be a robust strategy, even when facing defectors.

2. The Shadow of the Future: The importance of future interactions in the IPD is often surprising. The longer the shadow of the future (i.e., the more future interactions players expect to have), the greater the incentive to cooperate. This is because the potential future payoffs from cooperation outweigh the immediate gains from defection.

3. Complexity from Simplicity: The IPD is based on simple rules, yet it can lead to incredibly complex dynamics. This complexity arising from simplicity can be quite unexpected, showing how basic interactions can lead to rich and varied behavioral patterns.

The dynamics of IPD games at the population level can be quite complex and often counterintuitive. The initial setup of the game and the interaction structure play crucial roles in determining the outcome, and the emergence of cooperation in a competitive environment highlights the nuanced interplay between individual strategies and collective behavior.

9 Incoming

• On Cooperation — Takeaways from Axelrod’s Iterated Prisoner’s Dilemma : Networks Course blog for INFO 2040/CS 2850/Econ 2040/SOC 2090
• Short history of iterated prisoner’s dilemma tournaments

10 References