Gemma 4 just made every other open-weight model look 10x too big

Google DeepMind released Gemma 4, the newest generation of its open-weight model family, with architectural changes that challenge what small models can do. The family includes five configurations, from a 2.3B-parameter dense model to a 31B-parameter flagship, and adds a Mixture-of-Experts variant with 26B total parameters (3.8B active). Ai2's EMO makes modular AI emerge from data, not human rules

For developers and researchers working on consumer hardware, the numbers are hard to ignore. Gemma 4's smallest model, at 2.3B effective parameters, roughly matches the performance of Gemma 3's 27B model across a range of benchmarks. That is a 10x parameter efficiency improvement in a single generation.

Thinking mode arrives in open weights

The most consequential addition is thinking mode, which lets models generate internal reasoning traces before producing a final answer. Previously the domain of closed models like OpenAI's o-series, this capability is now available under the Apache 2.0 license. On the AIME 2026 mathematics benchmark, the 31B model achieves 89.2% accuracy in thinking mode. The MoE variant scores 88.3%. Both figures rival models many times their size.

"We believe that openness in AI can spread the benefits of these technologies across society," the Gemma Team writes in the technical report, "but this must be continuously evaluated against the risk of malicious uses."

The thinking mode is optional. Models can be run in both thinking and non-thinking configurations, giving developers control over when to trade latency for reasoning depth. On the OpenAI o1-inspired methodology, Gemma 4 models output a chain-of-thought trace enclosed in special tokens before the final response. The verification horizon: why verifying coding agents…

Encoder-free architecture redefines multimodal processing

The most technically audacious move is the 12B model's encoder-free design. Instead of relying on separate vision and audio encoders, typically hundreds of millions of parameters each, the 12B model projects raw 48x48 RGB patches and 40ms audio chunks directly into the LLM embedding space. The 550M vision encoder is replaced by a single 35M matrix multiplication. The 305M audio encoder is discarded entirely.

This approach reduces memory fragmentation and simplifies deployment. On FLEURS automatic speech recognition, the 12B model achieves a word error rate of 0.063 on English, competitive with dedicated encoder-based systems. On CoVoST translation tasks, it scores 26.4 BLEU from Japanese to English, without ever seeing an explicitly trained speech encoder. Aleph Alpha unveils T-Free: a tokenizer-free…

Long-context and compute efficiency

Gemma 4 tackles the memory explosion problem that plagues long-context inference through a coordinated set of architectural choices. The models use a 5:1 ratio of local sliding window attention to global self-attention (4:1 for the 2.3B model), combined with p-RoPE positional encoding. Crucially, the global attention layers reuse keys as values, reducing KV cache footprint by up to 37.5%.

On the 128k RULER accuracy benchmark, the 31B model scores 96.4%, a dramatic improvement over Gemma 3's 66.0%. The MoE variant achieves 89.8%, while even the 4.5B model scores 86.6%. The needle-in-a-haystack MRCR benchmark tells a similar story: the 31B model scores 66.4% at 128k context, nearly five times better than Gemma 3's 13.5%. Your AI model says it can read 1 million tokens. It's…

The MTP drafter head, a small autoregressive model trained for speculative decoding, accelerates inference by generating multiple tokens per forward pass. For the smallest models, a top-k clustering trick reduces the final projection from a 262,000-way softmax to just 4,096, preserving acceptance rate while dramatically cutting computation. DeepSeek's DSpark just fixed the two things that held…

Quantization without compromise

Quantization-aware training is applied not just to the language model but to the vision and audio encoders as well. The 150M vision encoder runs at W8A8 precision, halving memory from 400MB to 200MB while cutting on-device latency by 44%. The audio encoder's footprint shrinks from 390MB to 87MB, a 78% reduction, through a mix of 2-bit, 4-bit, and 8-bit precision per layer cluster.

The 2.3B model with quantized weights and int8 KV cache occupies just 0.8GB at 32k context. That is small enough to run on a smartphone. The 31B model, in Q4_0 quantization with KV cache, fits in 19.2GB, within reach of a single consumer GPU. How Local LLMs Like Gemma and Qwen Are Taming Open…

Benchmark performance and the Arena leaderboard

On Chatbot Arena, Gemma 4 31B achieves an Elo of 1451, the highest score for any dense open-weight model. The MoE variant scores 1438. Both outperform models with 10 to 50 times more parameters, including DeepSeek V4 Flash Thinking (1436) and Qwen 3.5 397B-A17B (1444). The top closed model, Claude Fable 5, sits at 1508, but it is not open. Claude Fable 5 is back after the US lifted export…

On coding benchmarks, the 31B model achieves a Codeforces Elo of 2150 and 80.0% on LiveCodeBench v6. TerminalBench Hard, evaluating agentic command-line tasks, sees the 31B model at 36.0%, compared to Gemma 3's 4.0%. On scientific reasoning (SciCode), the 31B model scores 43.0%, more than double Gemma 3's 21.0%.

A platform for the next wave

Gemma 4 is released under the Apache 2.0 license, allowing commercial use, modification, and redistribution without royalty. The models are available in bf16 and quantized formats, with support for major inference engines including llama.cpp. The training infrastructure leveraged TPU v5p and v6e pods with up to 12,288 chips for the largest models, using Pathways for data-parallel reduction and ZeRO-3 for optimizer state sharding.

The technical report notes that all models are evaluated without safety filters to assess inherent behavior, and that "major improvements in every category of content safety" were observed relative to prior Gemma generations. The team encourages developers to implement system-level mitigations tailored to specific use cases.