AI evaluation
3 published articles
AI Research
Your AI agent passed the test by accident. Now there's a rubric for that.
SkillCoach is a self-evolving rubric framework that evaluates and improves agentic skill-use by analyzing skill selection, following, composition, and reflection processes, providing better supervision than outcome-only metrics.
2026-07-06
AI Research
Llms corrupt your documents when you delegate: a close look at the delegate-52 benchmark
The DELEGATE-52 benchmark reveals that current LLMs accumulate fidelity degradation when entrusted with multi-step document edits. Errors affect 19–34% of artifact content over 20 iterations, though Python workflows show less than 1% loss. The study is a diagnostic tool, not a verdict on real-world AI utility.
2026-07-05
Agent Benchmark Analysis
ProgramBench: every public AI scores 0% on the hardest coding test yet
ProgramBench challenges AI agents to reconstruct programs from binaries alone, without source code or issue descriptions. All public models fail to fully resolve any task, exposing weaknesses in probing, architecture, and stopping judgment. The benchmark is a stress test for coding agents moving beyond patch-based workflows.
2026-06-29