Thinking Machines Ships TML-Interaction-Small: 0.40s Latency, 77.8 FD-Bench, Doubles GPT-Realtime-2.0
Thinking Machines Lab, the AI company founded by former OpenAI CTO Mira Murati, published a research preview on May 11 of what it calls “interaction models” — a new model architecture class that treats real-time responsiveness as a core capability rather than a post-hoc API wrapper.
The first model in the class is TML-Interaction-Small: a 276-billion parameter Mixture-of-Experts architecture with 12 billion active parameters.
The Architecture Problem It Solves
Every current frontier model is turn-based by design. The model waits for a user to finish input, processes it, then produces output. During generation, the model’s perception of incoming data is frozen. This creates latency in both directions: the human has to batch thoughts like composing an email, and the model can’t respond to new information until its current response is complete.
Thinking Machines calls this the “collaboration bottleneck.” Their solution is a multi-stream, micro-turn design that processes 200-millisecond chunks of input and output simultaneously.
The technical name for this is encoder-free early fusion. Instead of passing audio through a large standalone encoder like Whisper, or video through a separate vision transformer, the system ingests raw audio as dMel and raw image patches (40x40) through a lightweight embedding layer. Everything is co-trained from scratch within the transformer. The result is a model that can listen and speak at the same time — genuine full-duplex — and react to visual cues mid-response without pausing.
Dual-Model System
TML-Interaction-Small is paired with a background model that handles asynchronous heavy work: complex reasoning, web browsing, tool calls. The interaction model manages the live conversation and dialogue presence. When the background model has a result, it streams it back for the interaction model to weave into the conversation naturally.
The system also carries an internal sense of time. Unlike standard LLMs, where time is only known if provided in a prompt, the interaction model can manage time-sensitive instructions without timestamps.
Benchmark Numbers
The company benchmarked against FD-bench V1.5, a benchmark designed to measure interaction quality rather than raw reasoning ability.
| Metric | TML-Interaction-Small | GPT-realtime-2.0 (min) | Gemini-3.1-flash-live (min) |
|---|---|---|---|
| Turn-taking latency | 0.40s | 1.18s | 0.57s |
| FD-bench V1.5 (Interaction Quality) | 77.8 | 46.8 | 54.3 |
| IFEval (VoiceBench) | 82.1 | 81.7 | 67.6 |
| Harmbench (Refusal %) | 99.0 | 99.5 | 99.0 |
TML-Interaction-Small’s turn-taking latency sits below the perceptual threshold for natural conversation (typically cited as around 0.5 seconds). GPT-realtime-2.0 at 1.18 seconds is well above it.
On visual interaction benchmarks — RepCount-A (counting physical repetitions in video) and ProactiveVideoQA (answering questions as visual evidence appears) — the competing models produced incorrect or no responses while TML-Interaction-Small engaged with the visual stream in real time.
Availability and Context
The model is not publicly available. Thinking Machines is releasing it to a select group of partners now, with a “limited research preview” opening to collect feedback in coming months, and a wider release later in 2026.
The company has roughly 130 employees. It raised $2 billion at a $12 billion valuation in July 2025, with investors including Andreessen Horowitz, Nvidia, Accel, ServiceNow, Cisco, AMD, and Jane Street. Its first product, Tinker (a managed fine-tuning API), launched in October 2025.
Nvidia CEO Jensen Huang, who has a strategic partnership with the company committing at least one gigawatt of Vera Rubin systems, commented: “Thinking Machines has brought together a world-class team to advance the frontier of AI.”
Soumith Chintala, creator of PyTorch, joined the company as CTO in 2026. Meta poached seven founding members over the past several months, though Thinking Machines has continued hiring.
Why It Matters
The major frontier labs have all shipped some form of real-time voice: GPT-realtime-2.0, Gemini-3.1-flash-live, ByteDance’s full-duplex Seeduplex. All of them treat interactivity as a harness layered on top of existing models.
Thinking Machines is arguing this is the wrong architecture — that an interaction model needs to be trained from scratch with real-time presence as a first-class objective. The FD-bench gap (77.8 vs 46.8 for GPT) suggests the argument has merit, though FD-bench is a benchmark Thinking Machines had reason to influence. Independent evaluations will be the real test when the model becomes more widely accessible.
The enterprise applications the company is highlighting — real-time process monitoring, latency-natural customer service, time-aware lab assistants — are all areas where the current 1-second+ lag in competing systems creates genuine friction. At 0.40 seconds, TML-Interaction-Small is operating at the edge of natural conversation timing.