Cactus Needle 3: An 8-29MB Model That Matches DeepSeek V4 Flash on Tool Calls After One Fine-Tuning Epoch
Cactus Compute released Needle 3 today: a foundation model for mobile, wearables, robotics, smart home, and automotive applications. The entire model is a single binary ranging from 8 to 29 MB depending on the number of layers selected. It runs locally with no API call.
The key claim: the 4-layer (4L) subnetwork matches DeepSeek V4 Flash on downstream automation tasks after one fine-tuning epoch on task-specific data.
Architecture
Needle 3 is built on Cactus’s Simple Attention Network architecture. The full model spans 2 to 20 layers, each layer being a sub-network of monotonically increasing capacity. Cactus calls this “intelligence laddering” — developers pick the subnetwork depth that matches their device constraints:
- 2L: smallest, lowest capacity
- 4L: DeepSeek V4 Flash parity on tool calls (tuned)
- 20L: 29 MB, highest capacity
The CQ2 quantisation (2-bit) produces 9-29 MB binaries across the depth range. Parameter counts run from 25M (2L) to 121M (20L) before quantisation. Training was on 360 billion tokens of proprietary structured data.
Decode speed on a Raspberry Pi 5: 400 to 4,000 tokens per second. Prefill: 1,000 to 10,000 tokens per second.
What It Does
Needle 3 is not a general chat model. Cactus explicitly traded general conversational ability for precision on three tasks:
Tool calls — given function definitions, the model selects the right ones and fills arguments from the user’s input. Multi-step requests produce ordered call sequences. Requests that don’t map to any available tool return an empty list rather than a hallucinated call.
Structured extraction — given a schema and unstructured text, the model returns typed fields. A constrained decode grammar guarantees the output parses. The same capability handles classification.
Text embedding — the same weights produce sentence vectors for local semantic search, tool routing, and deduplication.
How the Benchmark Claim Works
The “matches DeepSeek V4 Flash” claim is conditional: it applies to the 4L subnetwork after one epoch of fine-tuning on the target downstream task. It’s not a zero-shot comparison and it’s not a general benchmark — it’s a claim about what happens when you specialize the model for a specific automation problem.
That caveat is significant. DeepSeek V4 Flash is a cloud API billing per token. A tuned Needle 3 running at 4,000 tokens per second on a Raspberry Pi 5 (starting at $50) has a marginal cost of effectively zero at inference. For latency-sensitive on-device applications where the task distribution is stable, the comparison is meaningful. For general-purpose assistant tasks, it isn’t.
Cactus’s prior model, Needle 2, was a 14MB single-file model covered here in August. Needle 3 extends the range downward to 8MB and adds the structured embedding output, making it viable for wearables and microcontrollers that couldn’t run Needle 2.