benchmark
5 published articles
AI Research
Why GPT-5.5 dominates a benchmark that tests how agents improve themselves
EvoPolicyGym isolates a critical but understudied capability: an agent's ability to refine an executable policy through repeated feedback-constrained edits. The benchmark reveals GPT-5.5 as the strongest performer across 16 environments, and provides trajectory-level diagnostics that expose how different agents allocate budget and convert feedback into tuned parameters.
2026-07-11
AI Research
Your AI assistant forgets you every morning. This benchmark proves it.
VitaBench 2.0 tests whether AI agents can continuously extract and update user preferences from fragmented interactions over time. Results show that even state-of-the-art models struggle to build persistent user models, revealing a wide gap between chatbot competence and genuine collaborative intelligence.
2026-07-11
AI Model Release
MiniMax launches M2.7 model with strong software engineering and office productivity skills
MiniMax's M2.7 model delivers strong results in software engineering benchmarks and professional office tasks, with a 97% skill adherence rate on complex instructions and an ELO score of 1495 on GDPval-AA.
2026-07-08
Climate Technology
Aimip phase 1: A new benchmark to test ai climate models
The AIMIP Phase 1 project, involving groups from NVIDIA, Google Research, and others, provides an open dataset and evaluation framework for AI climate models. While these models accurately reproduce historical climate patterns, their ability to generalize to unseen conditions remains a key challenge.
2026-07-03
Benchmark
Oxford's GauntletBench puts AI agents through 100 real tasks. They failed 81% of them.
Oxford's GauntletBench puts AI agents through 100 challenging real-world tasks. Frontier systems cap out at 19.1% success, far below human performance. The benchmark targets overlooked capabilities in temporal perception, graphical understanding, and 3D reasoning across five professional applications.
2026-07-01