CL-SN55 941 —
GPT-61S 885 —
GPT-56T 861 —
GPT-6S 825 —
CL-OP55X 818 -0.1%
MUSE-SPK 793 —
QWEN-38X 779 —
GPT-6A 778 —
CL-OP5H 711 -0.1%
CL-OP5X 709 -0.1%
KIMI-K3X 699 -0.1%
CL-OP46H 694 -0.1%
CL-FAB5H 692 -0.1%
CL-OP55H 692 -0.1%
CL-OP47H 683 -0.1%
GEM-37FH 648 -0.2%
GEM-38FH 633 -0.2%
INKL 531 —
CL-OP47 529 -0.2%
GPT-56S 529 -5.9%
GEM-3P 493 —
CL-OP46 488 —
CL-OP48 481 —
GEM-31P 453 -0.2%
GPT-6L 436 —
CL-SN55 941 —
GPT-61S 885 —
GPT-56T 861 —
GPT-6S 825 —
CL-OP55X 818 -0.1%
MUSE-SPK 793 —
QWEN-38X 779 —
GPT-6A 778 —
CL-OP5H 711 -0.1%
CL-OP5X 709 -0.1%
KIMI-K3X 699 -0.1%
CL-OP46H 694 -0.1%
CL-FAB5H 692 -0.1%
CL-OP55H 692 -0.1%
CL-OP47H 683 -0.1%
GEM-37FH 648 -0.2%
GEM-38FH 633 -0.2%
INKL 531 —
CL-OP47 529 -0.2%
GPT-56S 529 -5.9%
GEM-3P 493 —
CL-OP46 488 —
CL-OP48 481 —
GEM-31P 453 -0.2%
GPT-6L 436 —
← Back to feed

Tencent Open-Sources Hy4 Preview: 770B MoE, Terminal-Bench 2.1 at 85.4%, Apache 2.0

Tencent released Hy4 preview on August 28 under Apache 2.0. The model is a 770B-parameter mixture-of-experts with 49B active parameters per token, a 1M-token context window, and a native MTP layer for speculative decoding. Weights are on HuggingFace, GitHub, ModelScope, and Tencent Cloud.

The prior generation, Hy3 preview, was 295B total with 21B active. Hy4 is more than double the parameter count, nearly 2.4x the active weights, and comes with substantially stronger agentic benchmark numbers.

Benchmark Numbers

BenchmarkScore
GPQA Diamond92.3
SWE-Bench Multilingual82.9%
Terminal-Bench 2.185.4%
MCP-Atlas83.7
CyberGym78.4
HLE (High + Tools)55.4
SWE-Bench Pro65.7%
DeepSWE64.3%
Office QA Pro66.2

Terminal-Bench 2.1 at 85.4% places it just behind GLM-5.3’s 88.2% on that leaderboard. SWE-Bench Pro at 65.7% trails Claude Mythos 5 (80.3%) and GPT-5.6 Sol (72.7%) but leads the open-source field.

BenchLM/BenchAlign ranks Hy4 preview at #7 of 228 models overall with a composite score of 79.16.

Architecture

The model uses 78 layers: the first is a standard dense FFN, the remaining 77 use MoE with 256 routed experts and 1 shared expert per layer. Each token activates the top-8 routed experts plus the shared expert. A native MTP layer (10B total, 0.7B active) is built in for speculative decoding.

Hidden size: 6144. Attention heads: 64 (Gated DSA). Query compression dimension: 2048. Context: 1M tokens. Vocabulary: 120,832 tokens.

Internal Blind Evaluation

Tencent ran 163 internal experts through 203 engineering tasks in a blind side-by-side:

  • vs GLM 5.3: 46.8% wins / 12.8% ties / 40.4% losses (2.99 vs 2.92 average)
  • vs Kimi K3: 51.2% wins / 7.9% ties / 40.9% losses (2.99 vs 2.94 average)

Both comparisons show a narrow lead, not a rout. Kimi K3 is currently at #5 on LiveBench globally with 79.2 overall.

Access and Pricing

API access is available through Tencent Cloud at $0.83/M input tokens. The model is co-designed with Tencent’s CodeBuddy and WorkBuddy products.

Open weights: huggingface.co/tencent/Hy4-preview | github.com/Tencent-Hunyuan/Hy4-preview