Unlocking Apple's Black Box: The Imprisoned Chip Finally Learned to Learn
The Imprisoned Chip
Every MacBook, iPhone, and Mac mini since 2017's A11 Bionic contains a dedicated slice of silicon: the Apple Neural Engine (ANE). Officially, it accelerates "machine learning inference"—in plain terms, it can only *run* pre-trained models, never *train* them.
Think of it as a brilliant student locked in a glass room: you can hand him exams and he finishes them at astonishing speed, but he can never open a textbook and learn something new. Apple strictly confines the ANE to inference mode via the CoreML framework. Want to run image recognition on an iPhone? Fine. Want to train a model on your Mac? Sorry—go rent an NVIDIA GPU.
Until early 2026, when a developer named Manjeet Singh decided to pry open that glass door.
The Intruder: The Art of Reverse Engineering
Manjeet Singh (GitHub: maderix) didn't hack Apple's servers or steal secrets. He practiced the oldest, most romantic craft in programming: reverse engineering.
Like examining a welded-shut Swiss watch by observing every vibration and tick to deduce its inner mechanism, he reasoned that the ANE hardware is capable of training—Apple merely locked it in software.
Over a single weekend, collaborating with Claude Opus, he traced from the CoreML framework all the way down to the IOKit kernel drivers (the low-level device driver layer of macOS/iOS). Then he found it.
Dissecting the Black Box: 67 Private APIs
In Apple's "private" world, Singh discovered 67 Objective-C classes forming the ANE's true control interface. The three most central:
_ANEClient— the client that establishes sessions with the hardware_ANECompiler— compiles computation graphs into ANE-executable instructions_ANEInMemoryModelDescriptor— an in-memory model descriptor allowing models to load without going through CoreML- Apple claims the M4's ANE delivers 38 TOPS.
- Actual measurement: 19 TFLOPS FP16 peak.
- RMSNorm, QKV projections, and scaled dot-product attention (SDPA)—forward and backward—ran entirely on the ANE
- ~9.3 ms per training step
- ~11.2% ANE utilization, sustaining 1.78 TFLOPS
- Privacy: personal data never leaves the device
- Cost: no expensive cloud GPUs needed for small/medium models
- Energy efficiency: training emissions could be a fraction of datacenter equivalents
These APIs never appeared in Apple's developer documentation. He also found that the ANE uses an intermediate format called MIL (Model Intermediate Language) to translate neural network operations into ANE instructions.
A Startling Discovery: Hidden Compute
When Singh bypassed CoreML and talked to the ANE directly, a surprising fact emerged:
The discrepancy is a numbers game: when the ANE performs INT8 operations, it dequantizes them to FP16 before computing—so INT8 offers no real speed advantage, only memory bandwidth savings. The "38 TOPS" is essentially 19 TFLOPS multiplied by 2, a marketing figure.
Still, at 2.8W peak power, the ANE achieves roughly 6.6 TFLOPS/W—about 80x the energy efficiency of an NVIDIA A100 GPU (~0.08 TFLOPS/W).
First Time Ever: Llama2 Learned on a Mac
With low-level control secured, Singh trained a full neural network on the ANE: a 109M-parameter Llama2 Transformer.
Training requires forward pass, loss computation, backward pass, and weight updates. Apple says the ANE can only do inference. Singh proved it can also do backpropagation.
In his implementation:
It's not perfect: weight-gradient accumulation and the Adam optimizer still run on the CPU. But this was a complete training loop—the first time a neural network has been trained on Apple's Neural Engine.
The Bottleneck: A Clogged Pipeline
The ANE has 16 processing cores arranged as a pipeline—like a factory line where each station does one step. Submitting a single operation leaves most stations idle: a single op reaches only ~30% of peak performance.
To saturate the ANE, you must chain 16–64 operations into one large computation graph submitted at once, pushing utilization up to 94%.
Another quirk: each process can compile ANE programs at most 119 times due to a resource leak—so training requires careful management of a compilation cache or it will crash mid-run.
The Legal Gray Zone
Singh's README cites two legal bases:
1. Sega v. Accolade (1992) — the Ninth Circuit held that reverse engineering for interoperability is legal 2. DMCA §1201(f) — the exemption allowing circumvention of technical measures to achieve interoperability with independently developed programs
In short: he didn't steal Apple's code—he studied how hardware he owns actually works.
Future Possibilities
The ANE Training project is a research prototype, not production-ready. But it proves something bigger: the barrier to AI training was never the hardware—it was the software.
If Apple officially opened ANE training:
As Feynman put it:
> "What I cannot create, I do not understand."
Singh created ANE training capability. Now we can say: we are beginning to understand it.
References
1. GitHub: maderix/ANE — Training neural networks on Apple Neural Engine via reverse-engineered private APIs 2. maderix's Substack: "Inside the M4 Apple Neural Engine, Part 1: Reverse Engineering" (February 2026) 3. maderix's Substack: "Inside the M4 Apple Neural Engine, Part 2: ANE Benchmarks" (February 2026) 4. Hollance/neural-engine — community ANE documentation (GitHub) 5. apple/ml-ane-transformers (GitHub)
*(This article is based on the above open-source project and technical blog posts; all technical figures come from the author's published measurements.)*