← Back to feed

SenseTime has open-sourced SenseNova U1 on HuggingFace under Apache 2.0, releasing the weights and a full technical report (arXiv 2605.12500) for a native multimodal model built on a new architecture the lab calls NEO-Unify.

The headline claim: SenseNova U1 removes both the Visual Encoder (VE) and Variational Auto-Encoder (VAE) found in every major multimodal system — including GPT-4o, Gemini, and LLaVA-class models — and replaces them with pixel-word integration at the architecture level.

Why Removing the VAE Matters

Standard multimodal models route images through a visual encoder that converts pixels into tokens, then pass those tokens to a language model. For generation, a VAE maps latent representations back to pixels. These handoffs introduce information loss at each stage, create alignment problems between visual and linguistic representations, and make it structurally harder to achieve consistency across modalities.

SenseNova U1’s NEO-Unify architecture treats generation and understanding as one modeling problem. Pixel-word correlations are built into the model from pretraining, not bolted on through adapters or separate modules. The practical result, according to the technical report, is better spatial consistency and reduced information loss when working with dense visual content — infographics, posters, comics, and structured image-text layouts where standard models frequently fail on text legibility and layout consistency.

Model Variants

Two open-weight models are available:

SenseNova-U1-8B-MoT (dense backbone)

• 8B parameters
• Mixture-of-Transformers architecture
• Full generation and understanding capabilities

SenseNova-U1-A3B-MoT (MoE backbone)

• 38B total parameters
• 3B active during inference
• Understanding stream: 128 experts, 30B total, 8 activated per token
• Generation stream: 32 experts, 8B total, 8 activated per token
• Runs on consumer hardware (12GB VRAM)

The A3B variant is the more interesting result: it achieves 38B parameter depth with 3B active, which is where the efficiency claim lives. For inference cost, this puts it in the same category as small dense models while operating at a capacity scale typically reserved for frontier models.

Architecture: Mixture of Transformers

The MoT backbone extends standard Transformer blocks by separating the understanding and generation streams at the expert level. Instead of sharing the same feed-forward weights for both visual comprehension and image synthesis, each stream has its own expert pool. A stream-wise MoE routing strategy determines which experts activate for each token — 8 of 128 in the understanding stream, 8 of 32 in generation.

Training used a 6-stage recipe: base pretraining → multimodal alignment → joint AR and pixel-space flow matching → reinforcement learning post-training → distillation → SFT. The flow matching component is what enables the model to generate in pixel space without a VAE.

Training and Deployment

SenseTime released ComfyUI support alongside the model weights, making the generation pipeline immediately deployable in standard image-generation workflows. The ArXiv paper is the most detailed public disclosure of how to build a native unified multimodal model on an MoE backbone that the open-source community has received.

The choice to open-source at this level of technical completeness — weights, training recipe, architecture spec, and ComfyUI integration — is notable. SenseTime is a large Chinese AI lab (one of the world’s largest by headcount) that has historically kept frontier research proprietary. The move aligns with a broader pattern of Chinese labs using open-source releases to establish architecture influence, particularly in domains where Chinese research is converging with or ahead of US counterparts.

Benchmark Position

SenseTime’s Rohan Paul post highlights density performance on infographics and structured visual content as the key differentiator. SenseNova U1 is not benchmarked against GPT Image 2 or Imagen 4 on standard T2I metrics — the positioning is explicitly around structured, text-heavy visual generation where standard diffusion models fail. That is a narrower but commercially important niche: document-to-visual workflows, structured data visualization, and high-consistency creative production.