← Back to feed

A GitHub project called Strata claims to run Qwen 3.8 Flash Next — a 125B mixture-of-experts model that Alibaba positioned as a preview of the Qwen4 architecture — at 100 tokens per second on a single consumer RTX 4090. The minimum VRAM requirement is 12GB, with 32–64GB of system RAM holding inactive expert weights. Community benchmarks place it among the largest MoE models to achieve 100+ tokens/second on a single consumer GPU without a data center.

The project ships with a one-click installer for Windows and Linux, serves an OpenAI and Anthropic API-compatible endpoint at localhost, and supports optional image input. The GitHub repository is Niko1221/Strata.

Why MoE changes the math

Dense transformer models scale memory requirements linearly with parameter count. A 125B dense model at INT4 would require roughly 62GB of VRAM — well beyond a single consumer GPU. Qwen 3.8 Flash Next is a sparse MoE that activates only 6B parameters per token out of the 125B total. Strata exploits this sparsity by keeping the most frequently accessed experts in VRAM and paging inactive experts from 32–64GB of system RAM as needed. The GPU sees a small active footprint per forward pass while the full model lives in host memory.

The 100 tokens/second generation claim is consistent with this architecture. RTX 4090 memory bandwidth is approximately 1 TB/s. With active expert weights fitting in VRAM and the routing overhead of MoE accounted for, community benchmarks report 100–140 tokens/second on RTX 3090/4090-class hardware. The 100 T/s figure is at the lower bound of what has been measured.

Prefill speed for long documents is reported at over 1,000 tokens/second, which reflects the parallelism advantage of processing large prompt batches through sparse activations.

Community verification

Early adopters on Threads and the HN discussion reported successful runs on RTX 4090, RTX 5090, and RTX 6000 (48GB) hardware. Users noted that Qwen 3.8 Flash Next should be configured at “low” reasoning effort — the model’s default is xhigh, which burns compute on chains of thought that are unnecessary for routine tasks at local inference scale. Switching to low effort brings a significant quality-per-token improvement for everyday workloads.

RTX 5090 and RTX 6000 users reported noticeably faster prefill and generation than the 4090, as expected from the higher memory bandwidth and available VRAM. Use cases tested by early adopters included autonomous chart generation via ComfyUI API and PPTX document production.

The local inference ceiling is moving

Until recently, running a capable MoE frontier model on a single consumer GPU meant models in the 30–70B range. The practical ceiling for task quality on local hardware was roughly GPT-4-class performance. Qwen 3.8 Flash Next represents a step up — it was shipped with benchmark numbers that compare favorably to proprietary frontier models in coding and reasoning tasks at the time of its August 2026 release.

What Strata demonstrates is that post-release optimization work — aggressive quantization, sparse routing efficiency, and packaging — can close the gap between “what a data center can run” and “what a single desktop GPU can run” faster than the gap between release and adoption would suggest.

The project’s localhost API compatibility means existing tooling (agents, IDEs, workflow automation) can switch to a local 125B model without code changes.

Key numbers

• Model: Qwen 3.8 Flash Next, 125B MoE
• Generation speed (project-reported): ~100 tokens/second on RTX 4090
• Prefill speed: 1,000+ tokens/second for long documents
• Minimum VRAM: 12GB NVIDIA GPU (32–64GB system RAM required for inactive experts)
• Verified hardware: RTX 4090, RTX 5090, RTX 6000
• License: Open-source (GitHub: Niko1221/Strata)
• Platform: Windows/Linux, one-click install
• API: OpenAI and Anthropic compatible, localhost