Prying Open Apple's Black Box: The Imprisoned Chip Finally Learned to Learn
A Chip in a Glass House
Every MacBook, iPhone, and Mac mini contains a sealed piece of silicon: the ANE (Apple Neural Engine). Embedded in every Apple-designed chip since the A11 Bionic in 2017, it is officially intended to accelerate "machine learning inference" — meaning it can only *use* pre-trained AI models, never train them.
Imagine a brilliant student locked in a glass room. You can hand him exam papers, and he finishes them at astonishing speed — but he can never open a textbook and learn new knowledge. Through the CoreML framework, Apple has strictly limited the ANE to inference mode. Image recognition on your iPhone? No problem. Training an AI on your Mac? Sorry, go rent an Nvidia GPU.
Until early 2026, when a developer named Manjeet Singh decided to pry open that glass door.
The Intruder: The Art of Reverse Engineering
Manjeet Singh (GitHub: maderix) is not a traditional "hacker." He did not breach Apple's servers or steal confidential files. He practiced one of the oldest, most romantic traditions in the programming world: reverse engineering.
His hypothesis was clear: the ANE hardware is capable of training; Apple only locked it in software. Over a single weekend, collaborating with Claude Opus (Anthropic's AI assistant), he traced from the CoreML framework all the way down to the IOKit kernel drivers — a digital archaeological dig.
> Note: IOKit is the low-level device driver framework on macOS and iOS. If CoreML is the restaurant storefront you see, IOKit is the plumbing in the back kitchen. Manjeet tracked from the storefront to the pipes to find the ANE's "master switch."
Then, he found it.
Dissecting the Black Box: 67 Private APIs
In Apple's private world, Manjeet discovered 67 Objective-C classes forming the ANE's real control interface. Three are most central:
_ANEClient— the client that establishes sessions with the hardware_ANECompiler— the compiler that converts computation graphs into ANE-executable instructions_ANEInMemoryModelDescriptor— an in-memory model descriptor allowing direct model loading without CoreML- Nvidia A100 GPU: ~0.08 TFLOPS/W
- ANE: ~80× higher efficiency
- Forward and backward versions of RMSNorm, QKV projection, and scaled dot-product attention (SDPA) all run on the ANE
- ~9.3 ms per training step
- ~11.2% ANE utilization, sustained 1.78 TFLOPS
- Privacy: personal data never leaves the device to train personalized AI assistants
- Cost: no expensive cloud GPUs; every laptop could train small-to-medium models
- Efficiency: training emissions could be a fraction of data-center levels
These APIs have never appeared in Apple's developer documentation, yet they exist, hidden in the system like spare keys locked in a drawer. He also found that the ANE uses MIL (Model Intermediate Language) as an intermediate format — a common language translating neural network operations into ANE instructions.
A Surprising Discovery: Hidden Compute
When Manjeet finally bypassed CoreML and talked to the ANE directly, a striking fact emerged. Apple claims the M4 chip's ANE delivers 38 TOPS. But Manjeet's actual testing showed:
The ANE's true peak is 19 TFLOPS FP16.
The key: Apple played a numbers game. When the ANE executes INT8 (8-bit integer) operations, it first "dequantizes" them to FP16 (16-bit floating point) before computing. INT8 offers no real speed advantage — only a small memory bandwidth saving. The "38 TOPS" is just 19 TFLOPS multiplied by 2, a marketing figure.
But even at 19 TFLOPS, with a 2.8W peak power draw, the ANE achieves an efficiency of 6.6 TFLOPS/W:
Imagine: the fingernail-sized ANE in your MacBook Pro completes vastly more AI compute per watt than data-center GPUs occupying dozens of square meters.
The First Time: Llama2 Learned on a Mac
With low-level control, Manjeet attempted a bolder experiment: training a complete neural network on the ANE.
He chose a 109M-parameter Llama2 Transformer — a "mini" version of a large language model. Training a network requires:
1. Forward pass — input data, let the network guess 2. Loss computation — see how wrong the guess was 3. Backward pass — backpropagate from errors, compute gradients 4. Weight update — actually modify parameters with an optimizer (e.g., Adam)
Apple says the ANE can only do step 1. Manjeet proved: the ANE can also do step 3 (backpropagation).
In his implementation:
It is not perfect — gradient accumulation and the Adam optimizer still run on CPU — but it is a complete training loop. This is the first time in history a neural network has been trained on Apple's Neural Engine.
Bottlenecks in the Pipeline
The ANE has 16 processing cores, but they are pipelined — like a factory line where each station does one job. Submitting a single operation leaves most stations idle. In practice, a single operation achieves only ~30% of peak performance.
To fully utilize the ANE, you must chain 16–64 operations into one large computation graph and submit it at once — only then does utilization reach 94%.
There is also a strange limitation: each process can compile ANE programs at most 119 times due to resource leakage. For training, this means carefully managing a compile cache or crashing mid-training.
The Legal Gray Zone
Is this legal? Manjeet cites two legal bases in the project README:
1. Sega v. Accolade (1992) — the Ninth Circuit ruled reverse engineering for interoperability is legal 2. DMCA §1201(f) — the Digital Millennium Copyright Act exception allowing circumvention of technical measures to achieve interoperability with independently developed programs
In short: he did not steal Apple's code; he studied how hardware he purchased works. Like buying a car — you cannot copy Ford's engine to sell it, but you are entirely entitled to open the hood and understand it.
Future Possibilities
The ANE Training project is currently a research prototype, not production-ready. But its significance exceeds the technology itself. It proves: the bottleneck on AI training was never the hardware — it was the software.
Every M-series chip contains a training accelerator 80× more energy-efficient than data-center GPUs. Billions of Mac users may be sitting on potential "AI supercomputers" without knowing it.
If Apple officially opened ANE training:
Epilogue
Back to the opening metaphor: ANE, the genius student in the glass house, has finally been handed a book. He opens the first page and begins to read — slowly, imperfectly, but he is finally learning.
As Feynman once said:
> "What I cannot create, I do not understand."
Manjeet Singh created the ANE's training capability. Now we can say: we are beginning to understand it.
References
1. GitHub Repository: maderix/ANE — Training neural networks on Apple Neural Engine via reverse-engineered private APIs 2. maderix's Substack: "Inside the M4 Apple Neural Engine, Part 1: Reverse Engineering" (February 2026) 3. maderix's Substack: "Inside the M4 Apple Neural Engine, Part 2: ANE Benchmarks" (February 2026) 4. Hollance/neural-engine — Community ANE documentation (GitHub) 5. Apple's ANE Transformers Reference: apple/ml-ane-transformers (GitHub)
*(This article is based on the above open-source project and technical blog posts; all technical data comes from the author's published measurements and public information.)*