K2-Think gets a 32B model to frontier math scores with test-time compute

The past year has settled one question: to get better at hard problems, an LLM does not have to keep growing in parameters. What matters more is teaching the model to think at length and with structure, and shifting part of the computation to inference time. K2-Think is a sharp example of that shift. The team takes a 32B model — a size anyone can afford to run — and squeezes the most out of it on math, code and science, at a level comparable to systems an order of magnitude larger.

What K2-Think is, in brief

The base is Qwen2.5-32B, with no prior reasoning fine-tuning. On top of it sits a coherent recipe built on six pillars:

• long chains of thought during SFT, to set the format and cadence of the reasoning;
• RL with verifiable rewards (tasks whose answers can be checked automatically);
• plan before reasoning: the model drafts a plan first, then thinks;
• test-time scaling: several independent attempts, then pick the best (Best-of-3);
• speculative decoding to speed up generation;
• hardware optimization on the Cerebras WSE — up to 2,000 tokens per second.

How it was taught to think

The first phase is SFT on AM-Thinking-v1-Distilled with long chains of thought. It moves the ceiling fast: on AIME 2024 the model settles around 79–80% within the first third of training, on AIME 2025 around 72%, and after that the returns fall off sharply. The final K2-Think-SFT checkpoint beats the base model comfortably at every attempt budget.

What RL with verifiable rewards actually added

The second phase is RLVR on roughly 92k problems across six domains. The picture is clear: if you start from a strong SFT checkpoint, RL adds only a few points, and it is easy to hit a plateau or slide backwards. One result deserves its own line: multi-stage RL with a shrinking context window breaks the patterns SFT established, and quality never climbs back to where it started.

How the model was strengthened at inference

The third pillar is test-time technique. An external LLM first pulls out the key concepts and drafts a short plan, then K2-Think generates several answers, and the external model picks the best one. In practice, Best-of-3 supplies most of the gain; the plan adds a couple of points more and cuts answer length by about 12% at the same time. Temperature was left at 1.0 — tuning it did not help.

How it scores on the benchmarks

• Math. 67.99 across AIME 2024/2025, HMMT25 and Omni-MATH-HARD. On the hardest of them, Omni-HARD, it gets 60.73. That is the top of the open-weight field, with the proprietary o3 High and Gemini 2.5 Pro sitting alongside it.
• Code. LiveCodeBench — 63.97, above many much larger open systems; SciCode — 39.2/12.0, comparable to Qwen3-235B-A22B.
• Science. GPQA-Diamond — 71.08 (strong), HLE — 9.95 (modest).

The frontier is reached here not by adding parameters, but by combining long-CoT SFT with test-time compute that has been thought through.

Why the work matters

The main lesson: invest not only in pretraining and parameter count, but in organizing the computation at inference time. A simple pairing of planning and Best-of-N, plus a fast runtime, lets a 32B model beat giants from another weight class and compete on math and programming. That opens the way to practical, affordable systems that can be scaled and tested in real use — which is exactly what the team is doing, through the web and an API.

💾 Demo

🤖 Model