Ollama rewrote its Apple engine. The M5 results are finally competitive.

For the past year, running large language models locally on a Mac meant accepting a tradeoff: the convenience of Apple's unified memory versus a consistent performance gap behind NVIDIA-powered rigs. Ollama's preview release, announced March 30, 2026, aims to close that gap by rebuilding its Apple silicon backend on MLX, Apple's machine learning framework. That's a big deal for anyone who has watched local inference on Mac lag behind for years, as argued in our analysis of Ollama's enterprise adoption claims.

The result is not an incremental improvement. On an M5 Ultra Mac Studio with 512GB of unified memory, Ollama 0.19 achieves a prefill speed of 1,851 tokens per second and a decode rate of 134 tokens per second using the Qwen3.5-35B-A3B model in int4 quantization. Those figures, from benchmarks conducted March 29, more than double what the previous llama.cpp-based implementation delivered on the same hardware. The performance jump changes the calculus for developers considering local agents over cloud API calls, a topic explored in our piece on measuring AI coding agent efficiency.

Why MLX changes the game

MLX's key advantage on Apple silicon is its ability to exploit the unified memory architecture that has long been a selling point but a software bottleneck. Models that previously had to shuffle data between CPU and GPU memory regions can now operate on a single memory pool with significantly lower latency. On the M5, M5 Pro, and M5 Max chips, Ollama also taps the new GPU Neural Accelerators, dedicated silicon blocks designed specifically for transformer-style matrix operations.

This is not merely a speed play. The MLX backend brings NVFP4 quantization to Ollama for the first time. NVFP4, a format developed by NVIDIA but now adopted by MLX, uses 4-bit floating-point representation that preserves more model accuracy than the Q4_K_M scheme used in earlier Ollama builds. That matters for developers who need local inference results to match production outputs from cloud providers, exactly the use case Ollama's team called out: “As more inference providers scale using NVFP4 format, this allows Ollama users to share the same results as they would in a production environment.” This cross-platform quantization push is a theme we've tracked in NVIDIA's efforts to standardize model optimization across hardware.

The coding agent accelerator

Ollama's second major improvement in this preview is a redesigned caching system. Agentic workflows, where a coding assistant branches through multiple tool calls each with a long system prompt, have been a weak spot for local inference because every new branch triggers a full prompt re-processing. Ollama now reuses cache across conversations, stores intelligent checkpoints at strategic locations in the prompt, and applies a smarter eviction policy that keeps shared prefixes alive longer even when older branches are dropped.

For tools like Claude Code, OpenCode, or the open-source OpenClaw, this means less time waiting for prompt processing between iterations. A developer asking the model to refactor a function, test the result, then try a different approach will see the second and third attempts run with near-zero prefill latency if the prompt prefix is shared. These caching improvements directly address pain points we've documented in best practices for AI coding agents.

NVIDIA and Apple, together in one stack

One of the more unexpected aspects of this release is the collaboration between NVIDIA and Olama on the Apple-native stack. The acknowledgments section of the release names NVIDIA contributors to NVFP4 quantization, the NVFP4 model optimizer, MLX CUDA support, and Ollama optimizations and testing. For a company whose GPUs are the primary competitor to Apple silicon in the AI workstation market, NVIDIA's involvement signals that NVFP4 is becoming a standard quantization format that both GPU and Apple silicon toolchains must support.

This also opens the door for Ollama to run models optimized by NVIDIA's model optimizer, a pipeline typically restricted to CUDA environments, on Apple hardware. The question is how long the exclusivity lasts: Ollama's team notes that other precisions will be made available “based on the design and usage intent from Ollama's research and hardware partners.” The strategic implications echo those we've seen in the enterprise AI coding race, where cross-platform compatibility is becoming the decisive factor.

What this means for the local LLM ecosystem

Ollama's MLX preview is, for now, limited to a single model: Qwen3.5-35B-A3B, a 35B-parameter MoE architecture that activates only 3B parameters per token. The model ships with sampling parameters tuned for coding tasks. Users with an M-series Mac and more than 32GB of unified memory can test it immediately via ollama run qwen3.5:35b-a3b-coding-nvfp4.

But the narrow model support is temporary. The release states that Olama is “actively working to support future models” and will introduce an easier import path for custom fine-tunes. Given that Olama's previous llama.cpp backend supported hundreds of architectures, the MLX branch is likely to expand quickly, especially for the family of models, Qwen, DeepSeek, and Llama, that are already tuned for Apple hardware.

The big picture is that local inference on Mac is no longer a compromise. For developers whose daily workflow revolves around coding agents and who are willing to invest in an M5 Ultra machine, the numbers in this preview suggest parity with consumer-grade NVIDIA hardware for the first time. The next six months will determine whether that parity holds across more models and more complex workloads. For now, Ollama has drawn a clear line: the M5 generation is where Apple silicon finally arrives as a first-class inference platform.