AI benchmark
3 published articles
Research analysis
AI document corruption in delegated workflows: what a new stress test reveals
The DELEGATE-52 benchmark evaluates AI systems on long-horizon delegated document editing tasks, finding that frontier models accumulate semantic fidelity loss of 19–34% over 20 iterations. Python workflows showed less than 1% degradation on average, but the study underscores that reliable long-horizon delegation remains an open challenge.
2026-07-04
Benchmark Deep Dive
ARC-AGI-2: The Benchmark That Measures Fluid Intelligence in AI Systems
ARC-AGI-2 tests AI systems on fluid intelligence through visual grid puzzles that can't be solved by memorization. Top frontier models now score 75-85%, but the grand prize of $700,000 remains unclaimed. Here's a deep dive into the benchmark's design, scoring, and current leaderboard.
2026-07-01
Open source
The first open-source web agent that doesn't need HTML just beat GPT-4o
MolmoWeb agents, available in 4B and 8B sizes, were trained on MolmoWebMix, a new dataset combining over 130,000 synthetic and human demonstrations. They surpass closed models and set new standards for open web agent research.
2026-04-10