Ollama's Mac update just solved the three things that made local AI painful

Ollama shipped an MLX engine update for Apple Silicon with three fixes for three distinct problems: NVIDIA's NVFP4 quantization format, fused Metal kernels that push output up to 20% faster, and a snapshot system that kills the redundant prompt processing that makes agent workflows expensive on local hardware.

The release targets three pain points that have kept local LLM deployment on Mac hardware from feeling production-ready: quantization quality, inference speed, and the overhead of repeated context processing in multi-turn agent sessions. The update comes as Ollama's broader push to make open-weight models a default in local inference gathers steam, backed by its $88 million bet on the Docker playbook for AI.

NVFP4 cuts quality loss in half

Ollama's MLX engine now supports NVIDIA's model-optimized NVFP4 format, which tracks the local dynamic range of model weights more closely than the common q4_K_M quantization. On the Gemma 4 12B model, NVFP4 roughly halves the perplexity gap between the 4-bit quantized version and the unquantized bf16 baseline, while keeping the same performance footprint. That means quality stays higher without trading memory, a meaningful step for anyone running local inference on models that were originally designed for datacenter hardware, a category that includes Google's Gemma 4, which just rewrote the rules for open-weight models.

Models optimized for datacenter deployment can now be imported and run on an Apple Silicon desktop without retraining. That bridging feature lets developers test inference on local hardware before scaling to the cloud, without paying the egress penalties that typically make GPU hopping painful, a problem that one new integration just tore down for cloud workflows.

Fused Metal kernels deliver 20% speedup

By fusing several operations into single Metal kernels via MLX's just-in-time compiler, Ollama pushed output generation up to 20% faster. The GPU-backed sampling pipeline was also rewritten for more efficient execution, a change that compounds with reduced memory pressure from the new quantization format. Speed improvements at this level matter most when agents iterate rapidly, and the agent tooling landscape is moving fast enough that Cursor 2.0 already redesigned the IDE around agent-first workflows.

Snapshot caching handles messy agent sessions

The standout feature is a snapshot system that saves model state at key points in a conversation, built for the messy dynamics of real agent sessions. In an agent loop, every tool call resends the entire transcript: system prompt, tool definitions, file reads. That often means processing tens of thousands of tokens repeatedly. Standard prefix caching fails when conversations branch, a thinking model drops reasoning tokens, or an agent hands off to a subagent.

Ollama's snapshot system saves state where conversations are likely to return: at branching points, at intervals through long prompts, and just before each response. That lets multiple agents run in parallel, each resuming from its own saved state, with shared context processed only once. Thinking models that discard reasoning tokens on each turn can resume from a snapshot taken before the response started, avoiding full reprocessing. If a developer branches a conversation for a different follow-up or regenerates a response, only the new direction needs to be computed. This kind of caching infrastructure resembles what Eve 0.22 does for agent state, treating it as files on disk rather than secrets in a config.

The system is selective and incremental to avoid filling memory. Snapshots are stored only at the points most likely to be reused. That matters for newer models with sliding-window attention or recurrent layers, where state cannot be rewound after the model moves past a conversation point.

To get started, download the latest Ollama and run ollama run gemma4:12b-mlx. For use in a coding agent, use ollama launch pi, model gemma4:12b-mlx.