Do Pricing Algorithms Learn to Collude? What the Q-Learning Experiments Actually Show

Published 2026-09-19. Last updated 2026-09-19. Sources are listed and linked at the end of this page.

Yes, in simulation, and the number to remember is 85 percent. Emilio Calvano, Giacomo Calzolari, Vincenzo Denicolò and Sergio Pastorello ran pairs of Q-learning agents in a repeated pricing game with logit demand and found they settled on prices capturing, on average, 85 percent of the distance between the static Bertrand-Nash profit and the monopoly profit, with no communication channel between them. Their published abstract states that "the algorithms consistently learn to charge supracompetitive prices, without communicating with one another. The high prices are sustained by collusive strategies with a finite phase of punishment followed by a gradual return to cooperation." The result is real and influential. It was obtained in a market of two firms choosing among 15 discrete prices, it required about 850,000 repetitions, the converged strategies formed a Nash equilibrium only 54 percent of the time, and no published study has observed the same reward and punishment structure in a live market.

The setup that produces the result

The experiment is narrower than the phrase "AI learns to collude" suggests. Two agents face a logit demand system and each picks one of 15 prices per period. Each agent runs tabular Q-learning: it keeps a table of estimated future payoffs indexed by the current state and the action it might take, and updates one cell of that table per period. The state is the pair of prices charged in the previous period, which gives the algorithm one period of memory. That is what makes conditional punishment representable at all. The whole table has 3,375 state-action cells in the baseline duopoly.

Exploration is epsilon-greedy with a decaying rate: the agent mostly plays whichever action its table currently rates highest, and occasionally prices at random. That random pricing fills in the table, and it is also, as the critics point out, what determines most of the early profit.

The authors are direct about what is not guaranteed here: "we do not know whether the algorithms converge at all or, if they do, whether they converge to a Nash equilibrium." Convergence is therefore operational rather than proved: a session counts as converged when each agent's best action is unchanged for 100,000 consecutive periods, and the simulation is abandoned after one billion. On that definition, more than 99.9 percent of sessions converged.

What "supracompetitive" meant in numbers

The headline measure is the average profit gain, written Delta: converged per-firm profit minus static Bertrand-Nash profit, divided by monopoly profit minus static Bertrand-Nash profit. Delta of 0 is the competitive outcome, Delta of 1 perfect collusion.

In the baseline duopoly the reported Delta is 85 percent. In the same parameterisation, the static Bertrand-Nash price is 1.4729 with per-period profit 0.2229, and the monopoly price is 1.9250 with per-period profit 0.3375. Arithmetic on those three figures puts the converged profit at roughly 0.320. The agents did not reach the monopoly price. They reached most of the way there, and the authors note that prices "are rarely as high as under monopoly but are almost always higher than in the Bertrand-Nash equilibrium."

The profit gain falls slowly as the market gets less concentrated, the finding that unsettled competition economists most. Moving from two firms to three cut the average profit gain from 85 percent to 64 percent, and the paper adds: "With four agents, the profit gain is still a substantial 56%." Human subjects in laboratory oligopoly games typically fail to sustain cooperation at three or four players without talking. The Q-learners degraded more gently. Cost asymmetry also bit less than theory predicts: a 25 percent cost advantage for one firm moved Delta only from 85 percent to 81 percent.

Punishment, not price, is the actual claim

A high price is not collusion. A firm that has failed to learn its own best response will charge above the competitive level, and so will a firm whose price grid sits above cost. What distinguishes a collusive strategy is a reward and punishment structure: a deviation draws a response that makes deviating unprofitable, after which the high price returns.

The authors tested this by overriding one agent after convergence, forcing a price cut, and tracing both agents over the 25 periods that follow. Over 1,000 sessions the pattern was consistent: "Clearly, the exogenous deviation gets punished. The punishment is not as harsh as it could be" and it lasts a finite number of periods before prices drift back up. The retaliation is also fast: the rival's response wipes out the deviator's gain in the very next period, so deviating does not pay.

The sharper piece of evidence is what happens on a small deviation rather than a full undercut. There the non-deviating agent cuts below the deviation: "In response to the deviation, both algorithms now cut their prices further down, below the exogenous initial reduction." The authors' reading of that overshoot is the load-bearing inference in the whole literature: it "would be difficult to rationalize unless the non-deviating algorithm is really punishing the rival." An automatic system merely relaxing back toward a resting point would not overshoot. The authors also ran the same test with the discount factor set to zero, where collusion cannot pay, and found an immediate return to the original price with no punishment phase. That control is the strongest part of the case.

Figure: schematic of the reward and punishment pattern. The horizontal levels are reported numbers; the two paths illustrate the described sequence and are not traced from published data.

The learning protocol decides the outcome

The most useful correction comes from John Asker, Chaim Fershtman and Ariel Pakes, who show the same family of algorithms produces opposite answers depending on one design choice. An asynchronous learner updates only the value of the action it took; a synchronous learner also computes what it would have earned at the prices it did not try.

Their result: "synchronous updating leads to competitive pricing, while asynchronous can lead to pricing close to monopoly levels." That holds with the discount factor set to zero, meaning the agents place no value on future profit. Punishment strategies are impossible in that setting, because a threat is worthless to an agent that does not care about tomorrow. Supracompetitive prices appeared anyway.

Their example market has two firms selling perfect substitutes, marginal cost of 2, a monopoly price of 10 and 100 feasible prices between 0.1 and 10. The synchronous pair converged near marginal cost quickly; the asynchronous pair settled well above it. The mechanism is nothing like a cartel: both algorithms pick the same action because both currently rate it highest, and later updates never push that value down far enough to dislodge it, so the pair sticks at a rest point above the competitive price.

Supracompetitive prices with no punishment at all

The same conclusion arrives from a different direction. Karsten Hansen, Kanishka Misra and Mallesh Pai study sellers running real-time price experiments with algorithms that ignore the rival's price. When the signal-to-noise ratio of those experiments is high, long-run prices are supracompetitive: "We show this occurs via a novel channel: competitors' algorithms' prices end up running correlated experiments. Therefore, sellers' misspecified models overestimate own price sensitivity, resulting in higher prices."

Nothing in that mechanism resembles an agreement. The algorithms are wrong about their own demand curve in a way that happens to be correlated across sellers.

A third route reaches the same place. Eshwar Ram Arunachaleswaran, Natalie Collina, Sampath Kannan, Aaron Roth and Juba Ziani prove that "supra-competitive prices can robustly emerge even when both players are using algorithms which do not explicitly encode threats, and which optimize for their own revenue." Deploying an algorithm is itself a commitment, and a second mover optimising against a committed no-regret learner ends up near monopoly prices without anything that a lawyer would recognise as a threat.

Taken together, these results make one point a compliance officer should carry into any vendor meeting: a supracompetitive price is weak evidence. It is produced by collusive strategies, by failed optimisation, by misspecified demand models and by unilateral commitment alike. Only the punishment structure discriminates between them, and that is exactly what the replications attack.

Method and limits

The most serious published reanalysis is by Arnoud den Boer, Janusz Meylahn and Maarten Pieter Schinkel. Their verdict: "A detailed analysis of the inner workings of this algorithm reveals that there is no immediate reason for alarm."

First, the measure and the claim are different things. A strategy of always pricing high scores near the top on Delta yet contains no punishment at all, so a high Delta cannot by itself establish a reward and punishment scheme.

Second, in a version tractable enough to solve exactly, with only two feasible prices, more than half of simulations do not converge to collusive equilibria or even to collusive strategies. On the inference from the impulse response: "Hence, the main argument to infer that the supracompetitive prices generated by their Q-learning algorithms are 'sustained by collusive strategies' appears to be incorrect." They construct non-collusive, non-equilibrium strategies that produce the same deviation-then-recovery picture.

Third, and most damaging for anyone extrapolating to a real business, is the time horizon. At a discount factor of 0.95, profits beyond roughly 165 periods contribute almost nothing to the firm's discounted objective. Convergence takes between about 400,000 and several million periods. Within the horizon the firm actually cares about, "the Q-learning algorithm prices uniformly at random in, on average, more than 164 time periods." The short-run profit therefore depends almost entirely on where the price grid was placed. The baseline grid has an average random price of 1.6990 against a Nash price of 1.4729, so random pricing on it already beats competition before any learning happens. Centre the grid on the Nash price instead and random pricing returns about 22 percent less than Nash.

Timo Klein, whose own sequential-pricing model does produce collusive equilibria, is equally careful: "I use Q-learning as a proof of concept only: It is unlikely that pricing algorithms observed 'in the wild' are completely and only based on Q-learning." His results are also fragile to one parameter: with six price steps the agents often converge on a fixed collusive price, while with 24 almost no runs reach a Nash equilibrium and the outcome becomes an asymmetric price cycle.

Galit Askenazi-Golan, Domenico Mergoni Cecchelli, Edward Plumb and Clemens Possnig supply the theoretical frame: "We obtain a Folk Theorem-style result and characterise the set of payoff vectors that can be obtained by these dynamics." A folk theorem result means the collusive outcome is one point in a large set of attainable outcomes, not a destination the dynamics select.

The environments are the remaining limit. Demand is stationary, the price grid is small and fixed, costs are known, and both agents run the same algorithm with the same hyperparameters. Real firms change repricing vendors, adjust rules, enter and exit, and face demand shocks arriving far sooner than 850,000 periods. The joint working paper of the Bundeskartellamt and the Autorité de la concurrence puts the conclusion plainly: "While many experiments on algorithmic pricing show that some degree of collusion can be achieved, whether these results will transpose to a real-world setting seems uncertain at present." It also raises the commercial objection: reaching the state the simulations describe would mean absorbing a long loss-making learning phase in a live market.

What real markets have shown so far

Two empirical studies matter, and they point in different directions.

Stephanie Assad, Robert Clark, Daniel Ershov and Lei Xu study the German retail gasoline market, where repricing software became widely available in 2017: "In duopoly markets, margins increase only if both stations adopt, suggesting that AP has a significant effect on competition." Where both stations in a local duopoly adopted, "Mean market-level margins increase by 3.2 cents per litre after market-wide AI adoption", roughly 38 percent. Single adopters gained about 0.8 cents per litre, roughly 9 percent, and monopolist adopters gained nothing statistically significant. They also report no significant margin change for about the first year after both stations adopted, which fits a slow learning process and several other explanations equally well.

The inconvenient study is Leon Musolff's work on e-commerce. Using a dataset of online sellers: "Evidence from an event study suggests that firms that start employing repricing tools drop their prices by 16.93%, with market prices falling by 9.67%." Adoption of repricing software made that market more competitive, not less. He does find that deliberate "resetting" strategies, which raise prices hoping rivals follow, lift competitor and market prices by 11.4 percent in markets with fewer than six serious competitors, but reports that price cycling of that kind "is still relatively rare in the data".

That is the honest state of the evidence. The strongest simulation result is 85 percent of the way to monopoly profit; the strongest field result in the other direction is a 9.67 percent fall in market prices after adoption. Both are real, and an audit that looks only for the first will find patterns in markets where the second is what is happening.

Questions worth putting to a vendor or an expert witness

Ask which behaviour is detected. A price level and a correlation between rivals' prices are both consistent with ordinary competition; only an asymmetric response to deviations matches the literature's definition.

Ask whether the method separates punishment from mean reversion. The overshoot on a small deviation is the discriminating test, and it requires observing deviations, which in real markets are rare and rarely exogenous.

Ask what the null model is. Supracompetitive prices arise from misspecified demand models and from unilateral commitment as readily as from collusion, so a detector with no null model for the innocent cases reports false positives.

Ask what the algorithm is. Q-learning with one period of memory over a 15-price grid is a research instrument; commercial repricers are rule-based systems configured by humans, and the exposure usually comes from the configuration.

Frequently asked questions

What is algorithmic collusion?

A supracompetitive outcome sustained by pricing algorithms whose strategies contain a reward and punishment structure: deviations are met with a response that makes deviating unprofitable, then prices return to the high level. The term is often used loosely for any parallel or raised pricing produced by software, a much broader claim.

Can algorithms collude without communicating?

In simulation, yes. The Calvano agents have no channel to each other beyond the previous period's prices, and they still converge on supracompetitive prices with a punishment phase. Whether the same happens under non-stationary demand, with heterogeneous algorithms and far fewer repetitions, is unresolved.

Is tacit collusion illegal?

Parallel pricing without an agreement is generally not an infringement in either EU or US law, which is why this literature attracted regulators: if an algorithm learns a punishment strategy by itself, there is no meeting of minds to prove.

How do you detect algorithmic collusion in real pricing data?

By looking for structure rather than level: an overshoot below a rival's price cut, followed by a slow recovery. A high margin alone cannot separate collusion from a failure to optimise, or from a demand model wrong in the same way for everyone.

Are there real-world examples?

No case has established autonomous algorithmic collusion of the kind these experiments describe. The German gasoline study shows margins rising about 38 percent in duopolies where both stations adopted, while the e-commerce study shows market prices falling about 9.67 percent. Enforcement so far has concerned humans using algorithms to implement agreements.

Sources

1. Emilio Calvano, Giacomo Calzolari, Vincenzo Denicolò and Sergio Pastorello, "Artificial Intelligence, Algorithmic Pricing, and Collusion", American Economic Review 110(10), 3267 to 3297. https://www.aeaweb.org/articles?id=10.1257/aer.20190623
2. Emilio Calvano, Giacomo Calzolari, Vincenzo Denicolò and Sergio Pastorello, "Artificial Intelligence, Algorithmic Pricing and Collusion", working paper, April 2019. https://www.ftc.gov/system/files/documents/public_events/1494697/calzolaricalvanodenicolopastorello.pdf
3. Timo Klein, "Autonomous algorithmic collusion: Q-learning under sequential pricing", RAND Journal of Economics 52(3), 538 to 558. https://research-portal.uu.nl/ws/files/243482156/The_RAND_J_of_Economics_-_2021_-_Klein_-_Autonomous_algorithmic_collusion_Q_learning_under_sequential_pricing.pdf
4. John Asker, Chaim Fershtman and Ariel Pakes, "Artificial Intelligence and Pricing: The Impact of Algorithm Design", NBER Working Paper 28535. https://www.nber.org/system/files/working_papers/w28535/w28535.pdf
5. Arnoud V. den Boer, Janusz M. Meylahn and Maarten Pieter Schinkel, "Artificial Collusion: Examining Supracompetitive Pricing by Q-learning Algorithms", Tinbergen Institute Discussion Paper TI 2022-067/VII. https://papers.tinbergen.nl/22067.pdf
6. Karsten Hansen, Kanishka Misra and Mallesh Pai, "DP14372 Algorithmic Collusion: Supra-competitive Prices via Independent Algorithms", CEPR Discussion Paper 14372. https://cepr.org/publications/dp14372
7. Eshwar Ram Arunachaleswaran, Natalie Collina, Sampath Kannan, Aaron Roth and Juba Ziani, "Algorithmic Collusion Without Threats", arXiv:2409.03956. https://arxiv.org/pdf/2409.03956
8. Galit Askenazi-Golan, Domenico Mergoni Cecchelli, Edward Plumb and Clemens Possnig, "The Bounds of Algorithmic Collusion: Q-learning, Gradient Learning, and the Folk Theorem", arXiv:2411.12725. https://arxiv.org/pdf/2411.12725
9. Stephanie Assad, Robert Clark, Daniel Ershov and Lei Xu, "Algorithmic Pricing and Competition: Empirical Evidence from the German Retail Gasoline Market", January 2021. https://economics.yale.edu/sites/default/files/clark_acex_jan_2021.pdf
10. Leon Musolff, "Algorithmic Pricing, Price Wars and Tacit Collusion: Evidence from E-Commerce". https://lmusolff.com/papers/Algorithmic_Pricing.pdf
11. Bundeskartellamt and Autorité de la concurrence, "Algorithms and Competition", November 2019. https://www.bundeskartellamt.de/SharedDocs/Publikation/EN/Berichte/Algorithms_and_Competition_Working-Paper.pdf?__blob=publicationFile&v=3

Read this page as Markdown