Bonsai 27B: First 27B-Class Model to Fit on an iPhone, 90% Accuracy at 3.9 GB
A 27B-parameter model has never run on a phone. The minimum viable build at 4-bit precision is 18 GB — three times the memory budget an app gets on a 12 GB iPhone. PrismML has closed that gap.
Bonsai 27B ships in two operating points, both based on Qwen3.6 27B, both under Apache 2.0.
The ternary variant uses {-1, 0, +1} weights with FP16 group-wise scaling at 1.71 effective bits per weight. At 5.9 GB, it runs on any modern laptop. On a 15-benchmark suite spanning math, coding, reasoning, tool-calling, and vision, it retains 95% of full-precision accuracy.
The 1-bit variant uses binary {-1, +1} weights at 1.125 bits per weight. At 3.9 GB, it clears the ~6 GB per-app iOS memory ceiling with headroom for KV cache and activations. It retains 90% of baseline on the same suite. Both run natively on Apple Silicon via MLX and on NVIDIA GPUs via CUDA.
The Memory Arithmetic
A phone never allocates its full RAM to an app. On a 12 GB iPhone, the available per-app budget is roughly 6 GB. The model occupies most of that; the KV cache, activations, and runtime take the rest. No conventional 27B build has come close to fitting. At 3.9 GB, 1-bit Bonsai 27B is the first to pass through with room to work.
Context length is 262K tokens on-device, enabled by the Qwen3.6 27B hybrid-attention backbone, which uses approximately 75% linear attention across its 64 blocks, constraining the full-attention KV cache to 16 layers rather than all 64.
Benchmark Numbers
| Variant | Bits/weight | Size | Overall (15-bench) |
|---|---|---|---|
| Qwen3.6-27B baseline | 16 | ~54 GB | 85.0 |
| Ternary Bonsai 27B | 1.71 | 5.9 GB | 80.5 (95%) |
| 1-bit Bonsai 27B | 1.125 | 3.9 GB | 76.1 (90%) |
Math held best across both variants. Agentic and tool-calling took the largest hit, from 80.0 on baseline to 66.0 on 1-bit. Vision dropped to 59.6 on 1-bit.
Inference speed on a MacBook M5 Max: 44 tokens/second (ternary), 87 tokens/second (1-bit). On an NVIDIA H100: 98 t/s ternary, 163 t/s 1-bit. On an NVIDIA RTX 5090: 134 t/s ternary, 163 t/s 1-bit.
What 27B on Device Enables
The capability tier that fits at 27B includes multi-step reasoning, structured tool calls, vision tasks, and computer-use agentic loops that stay coherent across many steps — the kind of sequential task coordination that sub-10B edge models do not sustain. Running that tier locally removes the latency and privacy costs of cloud round-trips and eliminates the dependency on network availability.
PrismML built custom low-bit kernels for both backends. The ternary and 1-bit weights are consumed directly by the kernels without expanding back to FP16 during inference.
Apple Interest
PrismML’s CEO told CNBC that Apple is evaluating the company’s technology. Apple currently uses a small on-device model for basic Siri tasks and routes more demanding inference to Google Gemini in the cloud. If Apple integrates PrismML’s quantization approach, it could enable a genuinely on-device reasoning tier for iOS 27 without the cloud dependency the current architecture requires.
No formal partnership has been disclosed. Bonsai 27B is available today. Free developer preview API is live.