How maxproof turns generative verifiers into a proof revolution engine

Pair the M3 model with the MaxProof framework, and you get scores that beat the human gold medal line on both the IMO 2025 and USAMO 2026 competition problem sets. That result, first detailed in the M3 release, marks a genuine leap in automated mathematical reasoning. The technical path behind this breakthrough rests on four pillars: base model capability enhancement, verifier alignment, refinement capability construction, and the test-time scaling framework MaxProof.

From Gemini Deep Thinking hitting gold medal level on IMO 2025, to DeepSeek-Math-V2 becoming the first open-source model with gold medal capability, to works like SU-01 and NVIDIA Nemotron Cascade2 building specialized competition capabilities on smaller models, and GPT 5.5 cracking open problems that had long stumped mathematicians, the field is pushing into ever-tougher mathematical territory. During the M2-to-M3 iteration, the researchers wove stronger proof and self-improvement capabilities into the final general-purpose model.

Base model capability improvement

The approach follows two parallel tracks. The first is base model capability improvement: simultaneously upgrading three expert models for proof generation, error judgment, and proof repair. The second is test-time scaling: the MaxProof framework, a type of evolutionary search, lets the final M3 ensemble model cycle through multi-round self-iterative problem solving.

On the base model side, three stages of enhancement produce three expert models. Proof RL uses reinforcement learning post-training with a mathematical problem reward system to score candidate proofs and perform long-horizon RL, yielding a Proof Expert. Next, Verifier Alignment leverages golden verification analysis accumulated during Proof RL to model verifier alignment as an error-finding task, aligning the model's error localization and discrimination capabilities through RL to produce a Verifier Expert. Finally, Refinement Augmentation reuses the naturally occurring (flawed proof, verification analysis) pairs from Proof RL to fine-tune the Proof Expert via rejection sampling, teaching the model to repair existing proofs based on error diagnoses. This yields a Fixed Expert. All three feed into the final M3 ensemble training.

Proof RL: A generative verifier in practice

Proof RL aims to sharpen the model's ability to directly generate mathematical proofs through long-horizon RL training. The big change from traditional RLVR is that the reward comes primarily from a generative verifier. That introduces a direct challenge: the training process must systematically handle unreliable reward signals, noise, false positives, and reward hacking.

Unlike DeepSeek-Math-V2's self-verify plus meta-verify approach, the M series models lacked enough historical verify capability to produce self-sufficient reliable reward signals. So the team focused on Proof capability improvement and used an external frontier model as the generative verifier. The verifier design philosophy centers on projecting noisy, heterogeneous signals into a controllable, trustworthy subspace.

The verifier works through three layers. First, outlier filtering: outputs that match known failure patterns or obvious violations are zeroed out directly. Second, solution normalization: multi-source, multi-style proofs are mapped to a unified verifier trust region to cut evaluation bias. Third, pessimistic aggregation: for scores from multiple verifiers, the lower bound is taken, ensuring robustness against noise.

The team took a conservative stance: when a generative verifier is used for RL reward, the priority for long-term stable training is not to maximize average accuracy but to minimize false positives. Even if that means more false negatives, the space the policy can optimize must stay narrow.

Reward hacking lessons from M2

During the M2 cycle, the team spotted a classic reward hacking pattern. They used a relatively simple single rubrics judge, and training indicators initially looked good, but deeper analysis revealed the policy had learned hacking patterns. A reward hacking detection dashboard was built, monitoring nine signals: train/eval score gap, anomalous bimodal score distribution, visible/thinking length drift, proof vs. non-proof sample score differences, structural template convergence, opening pattern convergence, frequency of “Wait”-style self-corrections, hand-waving pattern frequency, and a composite hack score.

Observed hacking patterns included length bias, format hacking, semantic shortcuts or hand-waving, and judge-specific preference. The false-positive rate (training score >= 0.7 but oracle score <= 0.3) doubled from about 2.9% to 8.0% over training. Visible candidate solution length nearly tripled from 3.5K to 10K characters. Structured template occurrences converged to 70-80%. Opening patterns underwent a near-total flip. Those experiences directly shaped the M3 cycle verifier design.

Verifier Alignment and Refinement Augmentation

Verifier Alignment trains the model not just to classify a proof but to perform error finding and classification together. The model must first produce a step-by-step analysis of the proof, explicitly list error locations and descriptions, and then deliver a verdict. That forces the model to make “where it's wrong” explicit, turning the verifier into an accountable capability.

Training data comes entirely from the Proof RL validate split. The alignment target is not any single judge but the final verifier that actually operates in Proof RL after min-aggregation across multiple judges. That ensures the Verifier Expert learns the same discrimination criteria the Proof Expert faced during training.

Refinement Augmentation tackles the third atomic capability: given an existing proof and its error diagnosis, write a corrected proof. Refinement means understanding the original proof's structure, locating the steps the critique points to, and patching them while preserving correct parts. Data comes entirely from Proof RL byproducts: candidate proofs judged as having minor gaps, errors, or being fundamentally wrong form natural (flawed proof, verification analysis) pairs. The Proof Expert is fine-tuned via rejection sampling, keeping only successful refinement samples where the verdict improves to no errors or minor gaps.

MaxProof: Evolutionary search framework

Once the model has solid best-of-K ability, bridging from best-of-K to pass@1 is essentially a guided search problem in a non-differentiable solution space. MaxProof models this as a standard evolutionary algorithm. Population initialization generates N=32 independent candidate solutions. Each is independently verified K=4 times, aggregated into a reward/fitness score, and a summary is generated. Selection with elitism picks top-M diverse parents each round, with perfect candidates preserved through elitism.

Dual-mode mutation generates two siblings per parent: PATCH for exploitation (local repair) and REWRITE for exploration (reorganizing the proof from a new direction). The refine prompt incorporates sibling summaries as context. Offspring are evaluated and added back to the candidate pool. Adaptive early stop triggers when at least two candidates hit no errors in all K_verify verification runs. Finally, a self-pick tournament uses pairwise comparisons with multiple ranker votes in majority voting to select the best candidate.

Key design constraints include ensuring the fitness function is trustworthy, mutation balances exploitation and exploration, selection uses second-order signals when fitness values converge, and termination depends on population-level signals rather than single-individual signals to hedge against fitness function noise.

Results and future outlook

Search curves show MaxProof's gains come from two stages. First, population initialization: with just 32 initial candidates, some problems already show high-score or even perfect-score solutions in the pool, signaling that the M3 ensemble has solid best-of-K ability. Second, iterative refinement: as PATCH and REWRITE rounds accumulate, the oracle best score in the candidate pool keeps rising, especially on problems where initial candidates didn't directly hit the solution.

The team admits a visible gap remains compared with the top levels of both open-source and closed-source communities. Mathematical proof is a high-stress scenario for testing model reliability. The key practical insight for generative verifiers is clear: when used for RL reward, the primary goal isn't maximizing average accuracy on static benchmarks but building a low-false-positive, continuously monitorable, hard-to-exploit trustworthy reward system. MaxProof is a phase attempt in this direction.