← Back to feed

A paper published May 7 identifies and names a failure mode that anyone deploying coding agents on real production backends has likely hit: as structural constraints pile up, agent performance falls off a cliff. The authors call it constraint decay.

The study, from researchers at Eurecom, ran 80 greenfield generation tasks and 20 feature-implementation tasks across eight web frameworks — Flask, FastAPI, Django, Express, NestJS, Spring Boot, Laravel, and Rails. Each task had a fixed API contract. The variable was how many structural constraints were imposed: architectural patterns, database schemas, object-relational mappings, naming conventions.

Evaluation used a dual method: end-to-end behavioral tests (does the thing actually work) plus static verifiers (does it match the required structure). The combination catches agents that produce functionally-passing code that structurally violates the spec — a class of failure that test suites alone miss.

The Numbers

Capable agent configurations drop on average 30 assertion pass rate points from baseline (loose specification) to fully constrained tasks. Some weaker configurations approach zero at full constraint load. The floor is not a plateau; it keeps falling as constraints add.

Framework sensitivity is large and non-obvious. Agents perform significantly better in Flask — minimal, explicit, few conventions — than in FastAPI or Django, which use conventions that must be inferred. The structural assumptions baked into convention-heavy frameworks are exactly the kind of knowledge that does not reliably transfer through in-context instruction.

Root Cause

Error analysis pins the leading failure to the data layer. Incorrect query composition and ORM runtime violations account for the largest share of assertion failures. The problem is not that agents can’t write Python or JavaScript — they can. The problem is that they don’t consistently model the relationships between framework routing, ORM expectations, and database state. When those three need to co-operate under explicit structural rules, agents break.

This is distinct from the hallucination problems studied in other settings. The agent is not making things up. It is generating code that compiles, passes basic tests, and looks correct in isolation — but violates the structural contract the backend depends on.

What This Means in Practice

The finding has direct implications for how coding agents are deployed in enterprise contexts. Greenfield generation in a simple stack is where current agents are strongest — and that is exactly the use case most engineering teams already handled. The hard cases are:

• Adding a feature to an existing Django or Spring Boot application with established ORM models
• Generating backend code against a pre-existing database schema with join constraints
• Building endpoints that must conform to both an API contract and an architectural pattern the rest of the codebase follows

These are not edge cases. They describe most real software engineering work.

The paper argues that existing benchmarks overreport agent capability by rewarding functional correctness while ignoring structural correctness. A system that scores 80% on SWE-bench Verified by producing functionally working patches may still fail 50% of tasks where the patch must also satisfy structural constraints that the test harness doesn’t check.

Citation

Dente, F., Satriani, D., Papotti, P. “Constraint Decay: The Fragility of LLM Agents in Backend Code Generation.” arXiv:2605.06445. May 7, 2026.