Kokkon: Mythos can be retrained with far less compute

Former OpenAI and Anthropic researcher Jake Kokkon named a chart from Anthropic’s IPO presentation as one of the main reasons he left the company after about four months. In his assessment, the chart shows that algorithmic efficiency is improving at an accelerating pace: Mythos, whose capabilities took a notable leap forward this spring, can already be retrained using only a small fraction of the computing resources originally required to train it. Kokkon believes algorithmic improvements are approaching automation, and calls for a one-year pause in the development of frontier models so competitors can catch up.

Former Anthropic researcher: Mythos can be retrained at lower cost

What the chart shows

A chart in Anthropic’s IPO materials shows how the company’s model-training compute costs have changed. Kokkon noted that the scale is compressed: a tenfold and a hundredfold increase in costs look almost the same. Yet the curve still rises sharply—Kokkon sees this as evidence that algorithmic efficiency is improving not just quickly, but at an accelerating rate.

This slide from Anthropic's IPO roadshow shows the "Pre-training compute multiplier" for code over time. The y-axis is logarithmic. Former Anthropic researcher Jacob Coxon says the steep climb from July 2024 to October 2026 points to superexponential algorithmic progress. Anthropic notes that the data points are based on internal estimates and are for illustration only. | Image: Anthropic

Source: the-decoder.com

In Kokkon’s view, Mythos is not a one-off outlier but a point on a superexponential curve. He says the model can already be retrained using only a small fraction of the computing resources originally required.

Why Kokkon is calling for a pause

Kokkon believes algorithmic improvements will soon be automatable. He therefore wants development of frontier models to be paused next year, allowing competitors to catch up with the leaders. He says a pause would make human-level intelligence widely available and harm only OpenAI and Anthropic.

Kokkon trained models at OpenAI, and his most recent job was at Anthropic. In September, he drew widespread attention with public warnings about the dangers of advanced AI systems; others in the industry shared those concerns. Among the risks they cite is the possibility that advanced AI capable of improving itself could get out of control.

Dataist's take: The chart reflects Kokkon’s view that algorithmic improvements are accelerating, but it does not by itself prove that model development can already be automated. His call for a pause stems from concern that this process could give industry leaders an advantage their competitors would struggle to close.