AI document corruption in delegated workflows: what a new stress test reveals

A recent paper that circulated under the title "LLMs Corrupt Your Documents When You Delegate" has reignited a critical conversation in the AI community: just how trustworthy are large language models when left to perform extended, multi-step modifications to sensitive artifacts like documents, spreadsheets, or code without close human supervision?

The research introduces a benchmark called DELEGATE-52, specifically designed to probe the gap between strong short-horizon benchmark performance and real-world reliability in delegated workflows. The authors, who have not publicly named their institution, emphasize that the work is not meant as a broad indictment of AI-assisted work, but rather as a diagnostic tool for understanding failure modes in a constrained setting.

How the benchmark works

DELEGATE-52 evaluates a pattern the researchers call delegated work: a user entrusts an AI system with multi-step transformations to digital artifacts, with limited human verification between steps. The benchmark uses chained transformation-and-inversion tasks, the system performs a series of edits, then is asked to reverse them, to measure whether the original semantic content is preserved.

Rather than scoring superficial formatting or stylistic differences, the evaluation relies on domain-specific semantic parsing to track meaningful changes to the underlying artifact. The researchers define "corruption" strictly as degradation in semantic fidelity, not task completion or user satisfaction.

"The errors we report thus correspond to degradation in the underlying semantic content but, our measure of 'corruption' did not include task completion or user satisfaction," the authors clarify.

Key findings: 19–34% degradation over 20 iterations

The main result is stark: across the evaluated settings, state-of-the-art frontier models showed roughly 19–34% degradation in artifact fidelity over 20 delegated iterations. In other words, after 20 rounds of unsupervised editing, the content bore significantly less resemblance to the original in terms of semantic accuracy.

However, important nuance emerged depending on the type of artifact. Python workflows proved significantly more robust, with less than 1% degradation on average, a finding the researchers attribute to the structured, executable nature of code compared to natural language documents.

The paper also tested a simplified agentic harness with tool-use capabilities like Python execution and file operations. While this setup did not eliminate degradation, the authors note that it should not be interpreted as representative of production-grade systems.

Methodological caveats and production reality

The study is deliberately framed as a stress test, not a simulation of real-world deployment. DELEGATE-52 evaluates long-horizon delegated execution with limited human intervention between steps, a scenario that is narrower than most actual AI workflows.

Current production systems, the authors acknowledge, can mitigate fidelity erosion through verification loops, orchestration layers, retrieval systems, memory mechanisms, and domain-specific tooling. The paper's goal is to identify where further research and engineering are needed, not to argue against using AI in professional settings.

Implications for the AI industry

The primary takeaway, according to the researchers, is that reliable long-horizon delegation remains an open challenge for both research and engineering. Strong performance on short-horizon benchmarks does not automatically translate to dependable behavior when a model is left unsupervised across many steps.

At the same time, the findings do not undermine the practical value of AI systems deployed today. Enterprises increasingly combine models with specialized harnesses and human-in-the-loop verification. The authors expect continued improvements in models, workflow-aware training, memory systems, and production-grade agentic harnesses to reduce these failure modes over time.

For CTOs and AI architects, the message is clear: treat long-horizon delegation with caution, implement verification checkpoints, and do not assume that benchmark excellence guarantees reliability at scale. The research serves as a timely reminder that trust in AI systems must be earned through rigorous, task-specific evaluation, not taken for granted.