← Back to feed

A University of Texas paper introduces AgingBench, a benchmark designed to measure whether AI agents stay reliable across many deployed sessions — and finds that most do not. The core result: an agent that passes evaluation at deployment can become progressively less accurate over weeks of operation without any change to the underlying model. The paper is available at arxiv.org/abs/2605.26302.

The problem with how agents are evaluated

Current evaluation practice tests agents at a clean starting point: fresh memory, no accumulated context, no prior session state. Real deployed agents don’t stay in that state. They summarize old conversations, store and retrieve memories, update facts, and go through periodic maintenance passes. Each of those operations is a potential failure point.

The paper describes a deployed agent as “a small operating system wrapped around a language model: it writes notes, compresses them, retrieves them, updates them, and occasionally cleans house.” Every step in that loop can introduce errors that compound silently across sessions.

Four failure modes

The paper identifies four distinct ways agents age after deployment:

Summary drift: Summarization passes that compress long conversation histories into shorter notes systematically drop key details. A medication dose becomes “a daily medication.” A specific client preference becomes a generic one. The information is gone — not misattributed, just absent.

Memory blur: When a deployed agent accumulates memories about similar entities (two clients with overlapping service histories, for example), memories can merge or swap attributes. Two distinct entries collapse toward a single blurred representation.

Stale facts: Updated information doesn’t always replace old information. A canceled subscription can remain marked active. A schedule change doesn’t propagate to stored context. The agent operates on a snapshot that becomes progressively outdated.

Maintenance-induced breaks: Routine memory maintenance — cleaning, re-indexing, compressing stored state — can suddenly remove memories entirely. Sessions that preceded a maintenance pass may have their context zeroed out.

Why “give it more memory” is the wrong fix

The paper makes a specific and counterintuitive argument: adding more memory capacity doesn’t address any of these failure modes. If a fact was never written correctly, retrieval can’t recover it. If a fact was written but crowded out, better summarization won’t fix it. If a fact exists but the agent doesn’t trust its retrieved context, the storage architecture is irrelevant.

The failure modes are in the write/compress/retrieve/trust decision chain — not in the storage capacity.

What AgingBench measures

AgingBench simulates multi-session agent operation and checks reliability at each stage — not just at the start. The benchmark is explicitly designed to catch degradation that point-in-time evaluations miss. The paper argues this reframes deployed agents from “static models” to “aging infrastructure” requiring active maintenance engineering.

The practical implication for enterprise AI deployments: production agent evaluations that only test freshly initialized systems are measuring a state that doesn’t reflect operational reality after the first few weeks.

Key Details

• Paper: “Your Agents Are Aging Too: Agent Lifespan Engineering for Deployed Systems”
• Authors: University of Texas
• Preprint: arxiv.org/abs/2605.26302
• Benchmark: AgingBench — cross-session reliability evaluation
• Failure modes identified: 4 (summary drift, memory blur, stale facts, maintenance breaks)