Gpt-5.5 dominates a new benchmark for agents that rewrite their own rules

Autonomous agents are being asked to do something that looks simple on paper but turns out to be remarkably hard: improve their own behavior through trial and error. A preprint called “EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments,” posted to arXiv on July 2, 2026, offers the first controlled setting to measure this ability in isolation.

The paper comes from researchers working at the intersection of reinforcement learning and agent evaluation. They introduce the concept of Autonomous Policy Evolution: a harness-model agent repeatedly edits an executable policy system under a fixed interaction budget. The goal is not to train a policy from scratch but to start with an initial policy and iteratively improve it using feedback from the environment.

That sets EvoPolicyGym apart from standard RL benchmarks, which measure end-to-end learning performance, and from open-ended software-engineering tasks like SWE-bench, which conflate policy improvement with broader coding ability. EvoPolicyGym strips those confounders away by providing compact interactive RL environments built specifically for policy editing.

What EvoPolicyGym actually measures

The benchmark includes 16 environments, ranging from classic control tasks to more complex interactive scenarios. Each environment provides a baseline policy and a feedback signal, typically a reward or a success metric, that the agent must use to make targeted edits.

What sets EvoPolicyGym apart from prior evaluation frameworks is its emphasis on trajectory-level diagnostics. Instead of collapsing performance into a single score, the benchmark tracks how agents allocate their budget across edits, how they convert sparse feedback into parametric tuning, and whether they discover mechanisms that generalize beyond the immediate test case.

The paper’s analysis shows that strong performance correlates with an agent’s ability to identify task-appropriate tuning strategies, rather than simply applying brute-force search. Agents that spent their budget on large, risky edits early tended to plateau. Agents that calibrated their edits to feedback granularity consistently improved across the interaction horizon.

Gpt-5.5 leads the pack, but the margin matters less than the pattern

According to the published results, GPT-5.5 achieves the strongest aggregate rank score and places in the top two on all 16 environments. The claim matters both for its consistency and for what it reveals about the current frontier of autonomous policy refinement.

Still, the paper’s authors caution that the leaderboard should not be the primary takeaway. More instructive are the diagnostics that show how GPT-5.5 succeeds: it discovers task-appropriate parameterization strategies early in the budget window and sticks with them. Weaker agents tend to flit between strategies or fail to convert sparse feedback into effective parameter updates.

The missing link in current agent evaluations

The autonomous agent ecosystem is flooded with benchmarks that measure final-task accuracy or software-engineering throughput. EvoPolicyGym addresses a blind spot: the capacity to improve internal policies under bounded, actionable feedback.

This capability matters for real-world deployment of agents that must operate in partially known environments. Think autonomous trading agents, adaptive robotics controllers, or self-improving code editors. A model that can rewrite its own rules in response to changing conditions is qualitatively different from one that merely executes a fixed policy.

The paper also notes that current large language models, including GPT-5.5, still exhibit failure modes: overshooting on environments with sparse feedback, under-exploring when the budget is generous, and occasionally making edits that degrade policy quality before recovering. These are not fatal flaws, but they point toward specific architectural and training improvements that future models will need.

What comes next

EvoPolicyGym is released as an open benchmark on GitHub, and the authors expect the community to adopt it as a standard evaluation suite for agentic policy refinement. The trajectory-level diagnostics, in particular, may prove useful for fine-tuning models that need to operate in continuous improvement loops.

For now, the benchmark confirms what many in the field suspected: that GPT-5.5 can consistently discover and apply task-specific refinements across diverse environments. But the more valuable contribution may be the evaluation methodology itself. A way to measure not just what agents know at test time, but how they learn from interaction.