Your AI agent passed the test by accident. Now there's a rubric for that.

As LLM agents move from toy demos to real production workflows, the gap between what they accomplish and how they do it is turning into a serious blind spot. A new paper posted on arXiv in July 2026 introduces SkillCoach, a framework that looks at the process behind agentic skill-use instead of just the final answer.

Anyone who has watched an agent fumble through a repository of documented skills knows the problem. It might land on the right answer by brute-force trial and error. It might grab a distractor skill that happens to work. It might skip required intermediate steps or forget a final validation check. Traditional verifiers, which check only the final output, call all these trajectories successful. But those are clearly not the kind of reliable, auditable behaviors you want in a production system.

Four dimensions of process quality

SkillCoach builds skill-grounded process rubrics from real agent rollouts and measures each trajectory along four axes:

• Skill selection: Did the agent choose the right skill for the step?
• Skill following: Did the agent execute the chosen skill correctly?
• Skill composition: Did the agent combine skills in the correct sequence?
• Skill-grounded reflection: Did the agent verify its own output against the skill's expected results?

By keeping the external verifier as a separate outcome signal, SkillCoach can tell the difference between a clean, repeatable execution and an accidental success. The rubrics are self-evolving, meaning they improve over time as more trajectories are collected and adapt to the specific skill repository and domain.

Exposing hidden failures

The paper's experiments show that evolved rubrics improve evaluation quality substantially, exposing failures that final accuracy alone would miss. Take an agent that selects a distractor skill, one that happens to produce the correct answer through a different, unintended path. A final verifier would give it a pass. SkillCoach would fail it on skill selection.

That granularity matters even more for training. Outcome-only filtering, which selects only trajectories that ended in success, can accidentally reinforce sloppy process habits. SkillCoach uses the evolved rubrics as process supervision instead, picking high-quality training trajectories that show correct skill use at every step. The authors report that this signal is substantially stronger for improving agentic skill-use than outcome-only filtering.

Implications for production agents

The timing of this research lines up with a broader industry push toward agentic workflows that need to be auditable and reliable. As companies deploy agents to handle SOPs, domain rules, tool workflows, and validation routines, the ability to evaluate not just the outcome but the quality of the reasoning process becomes essential for compliance, safety, and trust.

SkillCoach points toward a future where agent evaluations are as rigorous as code reviews, with every skill selection and composition step scrutinized rather than just the final answer. The research is currently available as a preprint on arXiv, with code and data expected to follow via linked repositories.