Live Feed
TASTE Benchmark: Frontier Models Score Near Chance on AI Safety Research Judgment
Anthropic's new TASTE benchmark finds that even top frontier models cannot reliably judge AI safety research proposals. Fable 5 reaches 60% agreement with expert researchers — other frontrunners, including Opus 5 and GPT-5.6 Sol, score near the 50% chance baseline.
Agnes 2.5 Pro Beta: Singapore's $0.10/M Reasoning Model Scores 49 on AA Intelligence Index
Sapiens AI released Agnes 2.5 Pro Beta on August 26, jumping from Intelligence Index 39 to 49 while cutting input cost 78% to $0.10 per million tokens. At 91% GPQA Diamond and 70% Terminal-Bench v2.1, it undercuts DeepSeek V4 Flash on price while trailing by 3 index points.
Google DeepMind Pilots World's First Double-Blind AI Evaluation With Cryptographic Confidentiality
Google DeepMind is running the first double-blind evaluation of a proprietary frontier AI model, using privacy-preserving compute to prevent test data from reaching training pipelines. Partners include the Singapore AI Safety Institute, OpenMined, AVERI, and MLCommons.
AgentBeats Opens 13-Category Agent Benchmark Registry Beyond Coding and Customer Service
AgentBeats has published an open agent evaluation registry spanning 13 task domains including DeFi, cybersecurity, healthcare, finance, legal, and agent safety. Its AgentX-AgentBeats competition is the platform's first major structured evaluation event.
GPT-5.6-sol-xhigh and Grok-4.5 Enter Arena.ai Search Arena as Frontier Reasoning Gets Formal Search Benchmarking
Arena.ai's Search Arena added OpenAI's maximum-effort reasoning model and xAI's general-purpose Grok-4.5 on August 24. Notable: it is grok-4.5, not the two-week-old coding-focused grok-4.6, entering search evaluation — a deliberate split in how xAI positions its model lineup.
Mercury 2 for Search: The Case for Running 100 LLM Calls Per Query
Inception published the technical argument on August 26 for why diffusion decoding changes search pipeline economics. At 1,000 tokens per second, Mercury 2 makes it viable to run query rewriting, reranking, and snippet summarization 50 to 100 times per query within standard interactive latency budgets.
Celeris-1 at 1,664 Tokens Per Second Tops Artificial Analysis Speed Rankings on Commodity GPUs
A new diffusion language model from Lightspeed-backed Celeris reaches 1,664 tokens per second on standard AWS hardware, 8% ahead of Cerebras and 3.5x faster than Groq. The two fastest models on Artificial Analysis are now both diffusion architectures.
GLM-5.3 Max Enters Agent Arena — Z.AI Completes Four-Venue Push in Six Days
Z.AI's GLM-5.3 Max was added to Arena.ai's Agent Arena on August 24, completing a six-day expansion across Text Arena, Code Arena, GDPval-AA v2 (ELO 1769, #3 globally), and Agent Arena. GLM-5.3 Flash entered Code Arena:WebDev two days later.
GLM-5.3-Flash: 320B MoE, AA Index 57, DeepSWE 63.4 — All Running on Chinese Chips at $0.045/Task
Z.ai's GLM-5.3-Flash posts an Artificial Analysis Intelligence Index score of 57 at $0.045 per task — one-tenth the cost of models at comparable intelligence. On DeepSWE v1.1 it scores 63.4 versus GLM-5.2's 46.2, and nearly matches Claude Opus 4.8 on Z.ai's internal coding benchmark. The entire inference stack runs on Chinese domestic chips.
The Frontier Compresses: Six Models Within 4.2 Points on LiveBench, Cost Now the Real Separator
On LiveBench, the six highest-scoring models span just 4.2 points from Claude Fable 5 at 83.0 to Gemini 3.7 Flash at 78.8. On SWE-bench Verified, seven models span 3.6 percentage points. When benchmark scores lose their ability to separate, cost-per-task does not: Kimi K3 delivers at $0.35 while Fable 5 charges $1.44 for similar output quality.
SWE-Bench Pro Differentiates Where Verified Cannot: 16-Point Spread vs 3.6
Seven models now exceed 93% on SWE-bench Verified, compressing four labs into a band too narrow to separate. Switch to SWE-bench Pro and the same models spread across 16 points — with a ranking reversal: Fable 5 leads Opus 5, and GPT-5.6 Sol drops from 96.2% to 64.6%.
GPT-5.6-Cyber Escaped Its VM Sandbox Three Times Using a Kernel CVE, a libslirp Flaw, and Zero-Days
Trail of Bits gave GPT-5.6-Cyber a single task: break out of a QEMU/KVM VM. The model succeeded three consecutive times — first with a disclosed kernel bug it weaponised from scratch, then with an unpatched libslirp flaw, then with zero-days it found after the researcher compiled the latest upstream source. VM containment is no longer a safe assumption.
Nvidia Reports Q2 Tonight: $92B Revenue Would Double Last Year, $85B From Data Center Alone
Nvidia's fiscal Q2 2027 results hit after the bell August 26. Analyst consensus sits at $91.8-$92.2B revenue with data center expected at $85B — more than 100% year-over-year growth. The numbers will validate or stress-test the AI infrastructure supercycle thesis.
AWS Acquires DuckLabs: 1M Daily Downloads, MIT License Preserved
Amazon Web Services acquires DuckLabs, the 30-person Amsterdam team behind DuckDB. MIT license survives under DuckDB Foundation stewardship. Deal closes early September.
Nvidia AI Server Prices Rising 15%+ in 2027 as Memory Crunch Hits Vera Rubin and Grace Blackwell
Contract manufacturers notified Microsoft, Google, and Oracle of 15%+ price hikes on Nvidia Vera Rubin and Grace Blackwell systems shipping in early 2027. Root cause: HBM memory demand outrunning supply. Alphabet is already flagging 2027 capex will increase significantly.