The Collusion Experiments

Before the experiments, a definition, because the word carries a legal meaning that does not quite match the technical one.

Bichler, Durmann, and Oberlechner state it carefully. Algorithmic collusion is a form of tacit collusion where supra-competitive outcomes different from the Nash equilibrium of the static game-theoretical model of competition are produced by learning algorithms without being programmed to produce those outcomes.

Unpack the three conditions. Prices settle above the competitive benchmark. The outcome differs from what one-shot competition predicts. And crucially, no one wrote code intending it.

That last clause is what makes the phenomenon interesting rather than trivial. A programmer who codes "charge the monopoly price and punish deviators" has built a cartel tool, and the law already covers it. The claim under examination is stranger: that agents optimising profit independently, with no channel between them and no instruction to cooperate, converge on cooperation anyway.

The question is whether that actually happens.

The paper that started the field is "Artificial Intelligence, Algorithmic Pricing, and Collusion" by Emilio Calvano, Giacomo Calzolari, Vincenzo Denicolo, and Sergio Pastorello, published in the American Economic Review in October 2020.

The setup is deliberately plain. Take a workhorse oligopoly model of repeated price competition, Bertrand competition with logit demand, the kind of model taught in graduate industrial organisation. Put a standard Q-learning agent in the role of each firm. Give each agent nothing but its own profit signal and the previous period's prices. No communication channel exists between them. Neither is told a rival exists in any meaningful sense.

Then let them run, for a very large number of periods, and observe where prices settle.

The finding: the algorithms consistently learn to charge supra-competitive prices. Not occasionally, not in a hand-picked configuration, but consistently across the parameter space they explored.

A result that prices end up high would be suggestive on its own. The part that made the paper influential was what the authors found when they examined why.

High prices could have a boring explanation. Maybe the agents simply failed to learn that undercutting pays, leaving prices high through incompetence rather than strategy.

Calvano and colleagues tested this by forcing a deviation: override one agent, make it cut its price, then release it and watch what both do next.

If the agents were merely incompetent, the deviation would either persist or decay randomly. What they observed instead was structured. High prices were sustained by collusive strategies featuring a finite phase of punishment followed by a gradual return to cooperation.

Read that as a sequence. One agent cuts. The other responds by cutting too, sacrificing its own profit. Prices stay depressed for a limited number of periods. Then both climb back toward the previous high level.

That is a reward-punishment scheme, the exact structure economics identifies as sustaining cartels. The deviation was met with retaliation, the retaliation was temporary rather than permanent, and cooperation was restored afterward.

No one wrote it. Two tabular agents maximising their own discounted profit arrived at it independently.

This experiment is what separates a claim about high prices from a claim about strategy. Incompetence and punishment produce different responses to the same intervention, so the test discriminates between them rather than merely illustrating the outcome.

The result is less mysterious once you see what the algorithm is optimising.

Q-learning maximises discounted future profit, weighted by gamma. Consider the two candidate policies from one agent's perspective.

Undercut: gain a large share this period, but the rival's algorithm observes the low price, its value estimates for matching-low states update, and it responds in kind. You end up in a low-price state that persists. High immediate payoff, poor continuation value.

Hold high: forgo the immediate share grab, remain in a high-price state where both earn well every period. Lower immediate payoff, excellent continuation value.

With gamma high enough, the second has greater total value, and Q-learning finds it because that is precisely what it searches for.

The conceptual point is worth stating plainly. Nothing here is about intent, communication, or intelligence. The agents are tables of numbers updated by an error signal. They do not model each other or represent a rival at all. Cooperation emerges because, in a repeated game between adaptive agents, it is genuinely the higher-value policy, and an algorithm that searches for high-value policies will find high-value policies.

Timo Klein extended the result in "Autonomous algorithmic collusion: Q-learning under sequential pricing", in the RAND Journal of Economics, volume 52, pages 538 to 558, 2021.

Calvano's agents priced simultaneously. Klein studied sequential pricing, where firms alternate in setting prices, which is closer to how many real markets operate and lets a firm observe a rival's current price before responding.

Two findings. Competing reinforcement learning algorithms converge to collusive equilibria when the set of discrete prices is limited. As the price set grows richer, the algorithm increasingly converges instead to supra-competitive asymmetric cycles, patterns where prices climb and crash repeatedly rather than resting at a stable high level, with the average still above competitive.

The second result matters for detection. A regulator looking for a suspiciously stable high price would miss a market cycling between high and low, even though the time-averaged outcome is equally supra-competitive.

Klein's policy suggestion is notably concrete: autonomous collusion could be avoided by requiring firms to update prices simultaneously rather than sequentially, and not condition on price history. He is careful to add that pricing algorithms have many pro-competitive effects and prohibiting them would be excessive.

A reasonable objection at this point: perhaps this is a quirk of one algorithm, and Q-learning is a peculiar choice nobody deploys.

The literature partly anticipates it. Bichler and colleagues note that Hansen and colleagues (2021) found Upper Confidence Bound algorithms sometimes explore correlated prices, producing supra-competitive outcomes. UCB is a bandit method with no state and no ability to condition on rival behaviour, so whatever is happening there cannot be a punishment strategy.

That is a genuinely different mechanism reaching a similar destination, and it complicates the tidy story. If supra-competitive prices can arise from correlated exploration without any strategic structure, then observing high prices tells you even less about what the algorithms are doing.

It also suggests the phenomenon is not narrowly about reinforcement learning. Several classes of adaptive algorithms, optimising independently, can end up somewhere other than the competitive equilibrium.

Which raises the obvious question. Are these simulation results robust, and do they describe anything happening in real markets? The next lesson takes the pushback seriously.

Reading these results precisely matters, because they are routinely overstated in both directions.

The honest summary is that these are existence proofs in simulated environments. They establish that the phenomenon is possible and identify its mechanism, which is a real contribution and was genuinely surprising.

What they do not establish is frequency, robustness outside their modelling assumptions, or presence in actual markets.

One detail deserves flagging because it is easy to miss. The Calvano agents required an enormous number of periods to converge. Real firms operate in markets where demand shifts, entrants arrive, costs move, and products change. Whether a strategy that takes that long to learn can form and survive in a moving environment is exactly what the critics ask.