Ifbench: the new benchmark testing AI instruction following

An AI model's job isn't just to produce a plausible answer, it must follow instructions precisely. That sounds simple, but it's deceptively hard. Consider a user who wants a three-sentence summary, a rewrite in a casual tone, or an answer that includes one word while avoiding another, often all at once. A model might grasp the topic perfectly and still flub the ask.

Accepted to NeurIPS 2025, IFBench tests how well language models follow those kinds of precise, natural-language instructions. Last year, Artificial Analysis, an independent AI benchmarking organization, added IFBench to its Intelligence Index, which combines several evaluations into a single score to measure a model's overall capability.

"We saw that a model's ability to follow user instructions was something developers cared a lot about, so we wanted to assess it explicitly," says Declan Jackson, a member of the technical staff at Artificial Analysis. "IFBench was designed to fill that gap and was challenging even for the frontier models."

Measuring adherence to a prompt

IFBench doesn't just test basic instruction following, it forces a model to juggle several rules at once. Some are straightforward, like minimum word counts or required keywords. Others are trickier: sentences must match in length, consecutive words can't start with the same letter, or a keyword has to land in a precise spot.

"That is different from many other benchmarks, where instruction following is captured only indirectly through output templates or requested answer structures," says Jackson.

Each constraint might seem arbitrary on its own. But together, they mirror a scenario that's all too familiar: users often ask a model for multiple things at once, and missing even one can ruin the response. To ground IFBench in real-world use, its prompts are pulled from actual user conversations, not written from scratch by researchers.

"IFBench measures instruction following in a way that feels closer to real-world use than earlier instruction following evals," says Jackson. "The prompts use casual, user-like language, cover a wide range of tones and lengths rather than following a fixed template, and focus on common tasks such as factual question answering, content review and summarization, and creative support. IFBench's wider coverage also makes it … a stronger overall signal of instruction-following ability."

What IFBench shows that other benchmarks miss

AI benchmarks usually have a short shelf life. Once models start scoring near the top, the evaluations stop being useful for telling systems apart. Most evaluations added to Artificial Analysis's Intelligence Index saturate within about six months, says Jackson.

But IFBench hasn't.

"While IFBench scores have improved over time, that progress has not been uniform across models, and new frontier models still do not always perform well on it," says Jackson.

That's down to a couple of factors.

The first: complex instruction following doesn't overlap much with what most labs are actively training for, says Jackson. Coding and tool use get heavy post-training investment because gains there tend to generalize across other tasks and benchmarks. Instruction following is narrower, and it rarely improves as a side effect of progress in those areas.

The second is the sheer breadth of what IFBench measures. Its wide set of constraints and prompts means progress has been slower relative to more targeted domain or capability evaluations, which labs can chip away at with focused post-training recipes.

That shows up in the numbers. IFBench scores cluster sharply by model family, says Jackson, and the rankings don't line up with Artificial Analysis's broader Intelligence Index.

xAI still leads IFBench. Grok 4.20 (0309, Reasoning) takes the top spot at 82.9%, with Grok 4.3 close behind at 81.3%. Recent Google models also score well: Gemini 3 Flash Preview (Reasoning) reaches 78.0%, while Gemini 3.1 Flash-Lite Preview and Gemini 3.1 Pro Preview land at 77.2% and 77.1%, respectively. OpenAI's GPT-5.5 (xhigh) and GPT-5.4 (xhigh) follow at 75.9% and 73.9%. The leading Claude models cluster lower on IFBench, with Claude Opus 4.7, Claude Sonnet 4.6, and Claude 4.5 Haiku scoring between 54.3% and 58.6%, even though Claude Opus 4.7 ranks near the top of the Intelligence Index at 57 points, behind GPT-5.5 (xhigh) at 60 and effectively tied with Gemini 3.1 Pro Preview and GPT-5.4 (xhigh), which also score 57.

A truly open approach to evals

IFBench is useful to Artificial Analysis for two reasons: what it measures, and the fact that it's released openly.

Openness lets Jackson's team implement the evaluation faithfully and run it across a wide range of models, feeding the leaderboards their users rely on. It also makes the benchmark itself easier to understand, since anyone can see what's being measured and why.

For Artificial Analysis, IFBench tests something that comes up in nearly every AI interaction: whether a model can keep track of what a user is asking, especially when the request has a lot going on. It's now a regular part of Artificial Analysis' evaluations, and a strong example of what Ai2's open benchmarks bring to the field.

"Beyond evaluations, Ai2 is an important leader in open source," says Jackson. "Their work not only helps advance the industry through open research, but also gives users access to research artifacts with transparency around data and methodology."