Coding Agents Don't Just Fail — They Rot: The SlopCodeBench Finding

Here is the number that should reset how you think about long-running coding agents: across 11 models and 93 checkpoints, no agent solved a single problem end-to-end, and the best checkpoint solve rate was 17.2%. And here is the number that explains why: agent-written code is 2.2x more verbose than the open-source code humans actually ship. That pairing — agents that can’t finish, producing code that bloats as they go — is the heart of SlopCodeBench (Orlanski et al., arXiv 2603.24755, March 2026), and it puts hard data under a thesis we’ve been arguing qualitatively for a while.

We’ve written before about how agents get worse the longer they run. That post described the behavior. SlopCodeBench gives us the mechanism — measured, repeatable, and uncomfortably precise. Coding agents don’t just fail. They rot the code underneath themselves, one turn at a time, and most of that rot is invisible until you read the diff.

The experiment: make the agent live with its own code

Most coding benchmarks are one-shot. A problem comes in, the agent emits a solution, you score it, you move on. That design can’t see rot, because rot is a function of time — of an agent inheriting and extending its own earlier decisions.

SlopCodeBench is built to expose exactly that. It takes 20 problems and turns each into a chain of evolving specifications: the agent writes code, then the spec changes, and the agent must extend its own prior solution rather than start fresh. Across those problems the benchmark defines 93 checkpoints, and it runs 11 models through the whole gauntlet. Crucially, the internal structure of the code is left entirely to the agent — nothing in the harness forces a clean architecture, so the benchmark can watch the agent’s early shortcuts compound into later messes.

What it found is a two-part indictment:

• Agents can’t actually finish. No model solved any problem all the way through its evolving spec. The single best checkpoint solve rate across all 11 models was 17.2%. These are not edge-case problems; they are sustained, iterative tasks of exactly the kind real software work is made of.
• The code decays as they go. SlopCodeBench tracks two distinct decay signals — structural erosion (complexity accumulating in the wrong places) and verbosity (redundant, bloated code). Structural erosion rose across the run in 80% of trajectories. Verbosity rose in 89.8%. Measured against real open-source repositories, agent code came out 2.2x more verbose.

The control is what makes it damning. When the authors looked at how human code evolves across its own version history, it stayed essentially flat — humans refactor, prune, and hold complexity roughly steady as a project grows. Agent code does the opposite. Each turn adds more than it should, on top of a foundation each previous turn already eroded. The agent passes the checkpoint and degrades the codebase in the same motion.

That is the definition of rot: not a single catastrophic failure, but a monotonic decline that every individual step can rationalize.

Why the rot compounds instead of averaging out

If failures were independent, you could reason about them with simple probability — run more, retry the bad ones, expect the law of large numbers to bail you out. The companion research says that intuition is wrong.

Reliability as a two-dimensional property (Khanal et al., arXiv 2603.29231, March 2026) measured 10 models across four task-duration buckets and found aggregate pass@1 falling from 76.3% on short-horizon tasks to 52.1% on very-long ones — a 24.3-point drop. The shape of that drop is the important part: it is super-linear compared to the i.i.d. Bernoulli baseline you’d get if each step failed independently. In plain terms, failures correlate. One bad decision raises the odds of the next one. The agent reasoning over a codebase its earlier turns made harder to reason over is more likely to make the next turn worse, not equally likely.

This is why naive retry math under-counts the risk, and it’s the quantitative backbone of what we called the reliability gap: per-step metrics measured on short tasks systematically lie about long-horizon behavior, because they assume an independence that the data refuses to grant. Rot isn’t noise you can average away. It’s a trend, and the trend points down.

Finding where the loop derailed

Knowing that long runs rot is one thing; locating the turn where rot set in is another — and you can’t fix what you can’t attribute. That’s the contribution of HORIZON (Wang et al., arXiv 2604.11978, April 2026), a cross-domain diagnostic benchmark paired with a trajectory-grounded LLM-as-a-Judge pipeline for failure attribution rather than mere pass/fail scoring.

Built on 3,100+ trajectories across four agentic domains, HORIZON’s judge is human-validated, reaching κ = 0.84 agreement with human annotators (against an inter-annotator κ of 0.61 — the machine judge agrees with humans more than humans agree with each other). It confirms the now-familiar shape — agents handle short and mid-horizon tasks well, then break down on long-horizon ones — but its real value is turning that breakdown into something diagnosable. Instead of “the run failed,” you get “the run derailed at this step, in this way.”

That reframes evaluation from a grade into an instrument. We argued in who judges the judge that an LLM evaluator is itself a loop component you have to validate, not a neutral oracle. HORIZON is that idea pointed at trajectories: a validated judge that localizes rot inside a run is the difference between knowing your agent decays and knowing which turn started the decay — which is the only knowledge you can actually act on.

Why this hits Southeast Asia differently

In a large, well-staffed engineering org, code rot has a backstop made of people: senior reviewers, dedicated QA, the slack to throw a few engineers at a refactor when a module gets ugly. Rot still costs you, but a deep bench absorbs it.

A two- or five-person team in Phnom Penh or Da Nang does not have that bench. There is no reviewer waiting downstream to catch the agent’s twentieth-turn slop before it lands. For a capital-light team, the agent’s rot lands directly in the product, and the engineer who has to live with it is the same one who shipped it. That sounds like a disadvantage — and it would be, if the only fix were more headcount.

But the fix isn’t headcount. It’s loop design, and loop design runs on a laptop. The team that internalizes the SlopCodeBench result stops pointing a frontier agent at a big task and letting it run for hours — the exact regime where rot is worst and the token bill is highest. Instead it engineers the loop so rot never gets a foothold. That’s not a bigger-budget move; it’s a better-judgment move, and judgment is the one input a small team can actually out-compete on.

The loop parameters that stop rot

SlopCodeBench tested the obvious verbal fix — telling the agent to write clean code — and found it improves the starting point without changing the rate of decay. A prompt makes turn one tidier; it does nothing about the slope. The countermeasures that work are structural, and they map cleanly onto the checkpoints in our loop engineering maturity model:

• Token budgets with hard kill criteria. Set a ceiling per unit of work and a max-iteration limit, and enforce it. Because failures correlate super-linearly, the marginal turn deep in a long run is the most likely to rot — so the run that’s already struggling is exactly the one to stop, not extend. A kill criterion (“reassign after N stuck iterations”) is a rot circuit-breaker.
• Periodic reset / refactor checkpoints. The decay curve is a function of how many turns ride on the same eroding foundation. Insert explicit refactor turns whose only job is to reduce structural erosion and verbosity, and reset the context window between scoped units so durable state lives in committed artifacts, not an ever-growing transcript. A fresh window denies the rot a place to accumulate.
• Scope small, verify hard, commit clean. Decompose work so each unit starts from a clean baseline instead of inheriting twenty turns of slop, verify each unit end-to-end, and commit working state so a rotted branch can be abandoned rather than extended. This is the same one-feature-at-a-time discipline we keep returning to — and SlopCodeBench is the data that explains why it works.
• Instrument for attribution, not just pass/fail. Borrow HORIZON’s posture: track where trajectories derail, so your kill criteria and refactor checkpoints land where the rot actually starts instead of where you guessed it might.

The unifying point is the one this whole body of research keeps confirming: degradation is a property of the loop’s shape, not the model’s IQ. A frontier model in a sloppy long run will rot its own work. A modest model in a loop that scopes tightly, budgets strictly, resets often, and attributes its failures will hold quality far longer — for fewer tokens. The labs will keep shipping agents that can run longer. The discipline to run them well is the part no model release makes obsolete, and it’s available to anyone willing to engineer the loop instead of trusting the run.