← Back to feed

StepFun shipped the open weights for Step 5 Preview on October 15, meeting the commitment made at the model’s September 20 launch. The delivery came with a technical architecture blog and a moment of unnecessary drama: a 2:27 AM UTC report from Forkast News declared the release missed after a 1:55 UTC probe returned a 404. The repository metadata shows the files landed around 01:31:42 UTC — the repository was mid-upload when the probe hit.

What the Weights Are

Step 5 Preview is a sparse MoE model at 600 billion total parameters with approximately 27 billion active per token. The Hugging Face release contains 28 BF16 safetensor shards: 24 base-model shards, two MTP3 shards, a vision-encoder shard, and vit.safetensors, plus an FP8 build linked separately. The full BF16 checkpoint is approximately 1.2 TB.

The architecture is what StepFun calls “narrow but deep”: 92 transformer blocks. For comparison, Qwen3.8-Max and Kimi K3 also use 92 blocks at 2.4T and 2.8T total parameters respectively. Step 5 achieves comparable long-horizon agentic performance at less than one-quarter the total parameter count and less than one-third the active parameter count of those models.

Gated Sliding Window Attention and Sparse GQA

The architecture document published October 15 details two design choices that allow the narrow-but-deep layout to handle 1M-token context efficiently.

For efficient attention, StepFun chose Gated Sliding Window Attention over more complex linear attention alternatives after finding the performance gap marginal while SWA is simpler to implement — particularly for speculative decoding. For full attention layers, they developed Sparse GQA: standard grouped-query attention with a sparse indexing mechanism inspired by Dense Sliding Attention, plus block-wise token merging to reduce the indexer’s footprint. This enables long-context training and inference without the memory overhead of standard full attention across 1M tokens.

The 92-block depth introduces training instability at scale. The technical blog covers the precision and gradient handling changes required, making this one of the more candid architecture accounts published alongside an open-weight release.

The Commercial License

Step 5 Preview ships under the StepFun Model License 1.0, which StepFun describes as an Apache-2.0-style license with additional conditions specific to model use.

The commercial tripwire: any deployment that crosses 100 million monthly active users or $20 million in revenue must display “Powered by StepFun AI.” A model trained materially on Step 5 Preview outputs counts as a derivative work under the license.

Below those thresholds, the model is effectively free to use commercially. The thresholds are high enough that most enterprise deployers clear them; cloud providers and consumer-scale applications would trigger attribution requirements.

Deployment

BF16 fits on one B300 node (4 GPUs) or one H200 node (8 GPUs) with tensor parallelism. Sixteen H200 GPUs in BF16 across two nodes also works, though StepFun notes a validation caveat for the Transformers offline path on that configuration. The FP8 build cuts to a single B300 node with 4 GPUs.

Day-zero support is available on NVIDIA (NIM catalog, NeMo AutoModel, build.nvidia.com), Fireworks AI, Together, Modal, DeepInfra, and Featherless AI. vLLM and SGLang recipes are published in the repository.

Benchmark Position

Step 5 Preview launched September 20 with the following independent-benchmark results:

Benchmark	Score
DeepSWE v1.1 (High)	67.7
Terminal-Bench v4 (High)	33.3
Artificial Analysis Intelligence Index	44
API pricing (input/output)	$1.00 / $2.70 per 1M tokens

The model sits 6–7 points behind GPT-6 Astra and Claude Opus 5 on DeepSWE, and 25 points behind GPT-6 Astra on Terminal-Bench v4. Independent SWE-bench Verified results on the open-weight release are pending.

What the Open Weights Enable

The 600B total parameter count translates to roughly 1.2 TB of BF16 weights, putting Step 5 Preview in range for enterprise inference deployments that want a no-licensing-fee path for most workloads. The commercial threshold license gives it the accessibility of an open model without Meta-style terms for smaller operators. Self-hosted deployment at 27B active parameters is cheaper to run per token than comparably capable closed-API alternatives, particularly for high-volume agentic pipelines where throughput cost dominates.