M3 Neural Engine RTL Bug Cuts AI Inference Throughput by 60% — Workaround Doubles Llama 3.2 Speed
A hardware erratum in Apple M3’s Neural Engine has been silently cutting AI inference throughput by 60% on a class of model configurations that includes Llama 3.2 1B by default. A researcher profiling ANE DMA performance discovered the bug, characterised its root cause, and published a workaround.
The erratum: whenever the transformer weight matrix input dimension D is an exact multiple of 2,048, the ANE kernel DMA’s speculative prefetch ring forces all 16 cores into a serialized memory access path. Throughput drops from the nominal 45–60 GB/s to a floor of 17–19 GB/s. At D = 2,048 — the default for Llama 3.2 — DRAM utilisation falls from the 60 GB/s ceiling to 16.93 GB/s. At D = 2,016, the same chip delivers 44.5 GB/s: a 27.6 GB/s difference from shifting a single dimension by 32.
All multiples of 2,048 produce the same bandwidth notch. Seven of ANEMLL’s 15 compiled model variants hit at least one affected dimension.
Before and after the workaround
Padding or adjusting weight dimensions to avoid 1 MiB multiples restores parallel DRAM bank access:
| Model | Before | After | Speedup |
|---|---|---|---|
| Llama 3.2 1B | 10.0 tok/s | 24.3 tok/s | 2.43× |
| Qwen3-8B | 1.36 tok/s | 2.97 tok/s | 2.18× |
DRAM utilisation for Llama 3.2 1B recovered from 24.7 GB/s to 60.0 GB/s. Qwen3-8B went from 22.4 to 48.7 GB/s. Both are now operating near nominal bandwidth.
Root cause
M3’s ANE has 16 cores that fetch weight slices from DRAM in parallel, each assigned a different memory region. At power-of-two D values, all 16 cores alias onto the same DRAM bank at the same stride. Rather than being serviced in parallel, their requests serialize, saturating a single DRAM bank while leaving others idle.
The affected addresses enter what the researcher describes as a “credit-starved issue regime” in M3’s memory controller. The behavior is RTL-level and does not manifest as a software bug — transfers complete correctly, just at one-third the expected speed. An FFT of throughput against tensor dimension reveals a dominant harmonic with wavelength 2,048 in D-space, with throughput notches at every multiple.
The fix does not require kernel changes. Recompiling ANE model artifacts with non-multiples-of-2,048 weight dimensions is sufficient. ANEMLL has identified the seven affected model compilations.
Practical impact
MacBook Pro M3 machines are a standard inference platform for developers running sub-10B parameter models locally. Llama 3.2 1B and Qwen3-8B sit squarely in the affected size class. A 2.4× throughput loss is the difference between 10 tokens per second — borderline usable — and 24 tokens per second, which is conversationally fluid.
The Llama 3.2 1B case is the most immediately relevant: it runs on ANEMLL’s public toolchain, it hits D = 2,048 by default, and it is one of the most widely deployed local inference models on Apple silicon. Anyone running the unpatched build has been leaving more than half the hardware’s capability on the table.
The finding was published in August 2026 and is drawing attention on Hacker News in September as the ANEMLL project prepares updated compilations.