← Back to feed

Search pipelines have a structural problem Inception laid out on August 26: every quality improvement is a sequential LLM call, and autoregressive models charge 400ms or more per call. Query rewriting blocks retrieval. Retrieval blocks reranking. Reranking blocks synthesis. Enough of those in sequence and the pipeline is too slow to ship.

The industry response has been to cut. Run a smaller model. Remove steps. The result is a split between search products running thin, cheap models that barely think and deep-research agents that take 30 minutes. The middle — a pipeline smart enough to matter and fast enough to use — has stayed mostly unoccupied.

Mercury 2’s argument, as published this week, is that diffusion decoding changes the arithmetic. At 1,000+ tokens per second on standard NVIDIA Blackwell GPUs, a 300-token reranking rationale takes a fraction of a second. Fifty such calls fit inside the latency budget that currently has to be reserved for one.

What the Steps Are

Inception’s case maps to three concrete pipeline stages:

Query rewriting. A literal search query misses documents a rephrased query finds. With an autoregressive model, each rewrite is a blocking call. Mercury 2 lets teams run six rewrites where they ran one, improving recall without adding meaningful latency.

Reranking. Merging results from multiple retrieval systems produces more candidates than any synthesis model can cleanly handle. Reranking narrows the set. More candidate results plus higher reranking depth produces better final context; at autoregressive speeds, teams cap both to stay within latency budgets.

Snippet summarization. Rather than passing raw retrieved text to a synthesis model, preprocessing each retrieved passage reduces context size and improves citation accuracy. Each summary is a call; at autoregressive speeds, it is usually omitted.

The compound effect: an agent that runs six rewrites instead of one, reranks a hundred results instead of ten, and summarizes every retrieved page does not just respond faster — it retrieves better.

Why the Architecture Matters

Autoregressive decoding is structurally sequential: each output token depends on every prior token, so generation cannot be parallelized within a single response. That serialization sets a hard floor on per-call latency, and the industry workaround has been smaller models — the only lever available when the decode order is fixed.

Diffusion language models do not have a fixed decode order. Mercury 2 refines an entire output span across a small, bounded number of forward passes, each pass seeing the developing answer holistically. The result is that throughput scales differently: a 300-token reranking rationale and a 500-token snippet summary come back in a fraction of the time that an autoregressive model needs.

Inception frames this as a collapse of the latency-quality trade-off specific to search. In most systems, latency and quality are separate variables. In search, every quality improvement is an LLM call, making them the same variable. At 1,000 tokens per second, the cost of one more pipeline step drops by roughly a factor of 10 compared to a typical speed-optimized autoregressive model.

Numbers and Availability

Mercury 2 generates approximately 1,009 tokens per second on NVIDIA Blackwell GPUs — 817 tokens per second as measured on Artificial Analysis in production conditions, which places it second overall on the speed leaderboard behind Celeris-1 (1,664 tok/s, also a diffusion model). Mercury 2 is priced at $0.25 per million input tokens and $0.75 per million output tokens, with a 128K context window, native tool use, and tunable reasoning depth.

The August 26 post is the latest in a series of Inception product releases — the company published “Mercury 2: the first reasoning model fast enough to pick up the phone” in July and an earlier throughput post later the same month. Inception is deploying Mercury 2 at Fortune 500 companies and describes the model as suited specifically to latency-sensitive applications where user experience is non-negotiable.