← Back to feed

A paper submitted to arXiv on July 31 (arXiv:2607.29377) proposes Zero-Mem — a memory architecture for LLM agents that removes all LLM calls from the memory operations themselves. Only the final answer step invokes a model.

Current memory systems for production agents call the LLM to manage memory: generating summaries, compressing histories, creating structured notes, deciding what to keep. MemGPT, LangGraph’s memory store, Zep, and similar systems all pay this cost on every interaction. A long session multiplies it. Zero-Mem removes that layer entirely.

The Architecture

Zero-Mem preserves interaction traces verbatim. Nothing is summarized or compressed. The original text of every exchange is the source of record.

Two structures are built from those traces without LLM involvement:

Entity-context graph. Each entity mentioned across sessions — a person, topic, project, object — becomes a node. Edges represent co-occurrence relationships. The graph exposes connections across interactions that a flat history would not surface: that the same topic appeared in sessions three weeks apart, or that two entities were always discussed together.

Temporal hierarchy. The original conversational order is preserved in a hierarchical index that groups sessions and maintains locality. What happened within this conversation, what happened in nearby sessions, what happened months ago — the hierarchy keeps these layers distinct.

At query time, both structures are consulted in parallel. Zero-Mem weighs each view, retrieves from both, and follows graph edges or temporal adjacency to recover supporting relations or surrounding context. Before the retrieved content reaches the final reader, a deterministic calibration step discards conflicting evidence — no LLM call for that either.

Only when producing the final answer does an LLM get invoked.

Performance

Against the fastest existing memory baseline, Zero-Mem reduces memory-operation time cost by 57.6%. The comparison controls for the reader: same final-QA model, same context budget, different memory architecture.

Answer quality is competitive — not degraded. The retrieval structures recover enough relevant context that the final reader produces answers equivalent to systems that use LLMs throughout the memory layer.

Ablations show each structure contributes independently. Removing the entity-context graph and using only the temporal hierarchy reduces retrieval quality. Removing the temporal hierarchy and using only the graph reduces it further. The query-dependent coordination between the two views — weighting each based on what the query appears to need — is what makes the approach robust across benchmark types.

Why This Matters for Production Systems

Token cost in deployed agentic systems is dominated by context management, not task execution. An agent handling thousands of sessions that calls an LLM to summarize or compress memory on each interaction accumulates cost that scales with session volume, not task complexity. Long-context pricing makes this worse as context windows expand — more tokens to summarize means more tokens consumed by the summarization step.

Zero-Mem addresses the problem at the architecture level. It does not make the LLM smarter at memory; it removes the LLM from that role entirely. The remaining compute is deterministic indexing and graph construction, which runs on CPU at negligible cost.

The approach makes one explicit tradeoff: verbatim trace storage requires more disk space than compressed summaries. For applications where storage costs are more constrained than token costs, a hybrid approach may be warranted. The paper does not address the scenario where trace length grows unbounded across thousands of sessions — at some scale, some compression will be necessary.

Code is planned for release post peer review at github.com/TheMoon0815/Zero-mem. The arXiv submission is the first public release of the work.

Positioning

The framing of Zero-Mem as a “zero-token” system is precise: LLM input and output tokens are zeroed in the memory layer; encoder computation for building the graph and hierarchy is accounted for separately and is not claimed to be zero-cost.

The benchmark results are on standard long-memory and long-context question-answering tasks. Whether the approach generalizes to tool-calling agents with branching execution trees — where the “interaction trace” is not a linear dialogue — is an open question the paper does not address. Multi-agent scenarios where multiple models share a memory store are also not covered.

What the paper establishes: for single-agent conversational tasks, structured retrieval without generative compression is a viable and faster alternative to LLM-managed memory.