Cracking Apple's Black Box: Reverse Engineering Training on the Apple Neural Engine

Forum topic · 小凯 · 2026-03-17

Summary

A Chinese tech forum post analyzes developer Manjeet Singh's (GitHub: maderix) breakthrough in enabling neural network training on Apple's Neural Engine (ANE), which Apple has restricted to inference-only via CoreML since 2017. Through reverse engineering—tracing from CoreML down to IOKit kernel drivers with assistance from Claude Opus—Singh uncovered 67 private Objective-C classes including _ANEClient, _ANECompiler, and _ANEInMemoryModelDescriptor. Real-world benchmarks of the M4 chip's ANE revealed a true peak of 19 TFLOPS FP16 at 2.8W, an efficiency of 6.6 TFLOPS/W—roughly 80 times better than an NVIDIA A100—suggesting Apple's advertised 38 TOPS is a marketing figure based on INT8-to-FP16 dequantization. Singh achieved the first reported training run on ANE: a 109M-parameter Llama2 Transformer with forward and backward passes on ANE at ~9.3ms per step and 11.2% utilization, though gradient accumulation and Adam updates still run on CPU. Limitations include pipeline architecture requiring 16–64 chained operations for peak throughput and a 119-compile-per-process cap. The project cites Sega v. Accolade and DMCA §1201(f) as legal grounding.

Cracking Apple's Black Box: The Imprisoned Chip Finally Learned to Learn

*English translation and summary of a zhichai.net forum post.*

The Imprisoned Chip

Every Apple silicon chip since the A11 (2017) contains a dedicated Apple Neural Engine (ANE). Officially, it accelerates "machine learning inference"—meaning it can only *run* pre-trained models, not train them. Apple locks the ANE into inference-only mode through the CoreML framework. Want to train an AI model on your Mac? Apple's answer has always been: go rent an NVIDIA GPU.

In early 2026, developer Manjeet Singh (GitHub: maderix) decided to pry open that door.

Reverse Engineering the Black Box

Singh spent a weekend—collaborating with Claude Opus—tracing from the CoreML framework down to the IOKit kernel drivers. He discovered 67 private Objective-C classes forming the ANE's real control interface, including:

_ANEClient — establishes sessions with the hardware
_ANECompiler — compiles computation graphs into ANE instructions
_ANEInMemoryModelDescriptor — loads models directly, bypassing CoreML

He also found the ANE uses MIL (Model Intermediate Language) as an intermediate format for translating neural network operations.

The Hidden Compute: Real Benchmarks

Apple claims the M4's ANE delivers 38 TOPS. But direct testing showed a true peak of 19 TFLOPS FP16. The discrepancy: the ANE "dequantizes" INT8 operations to FP16 before computing, so INT8 offers no real speed advantage—only memory savings. The 38 TOPS figure is essentially 19 × 2 marketing math.

Even at 19 TFLOPS, the ANE's peak power draw is only 2.8W, yielding 6.6 TFLOPS/W—roughly 80× the energy efficiency of an NVIDIA A100 GPU (~0.08 TFLOPS/W).

First Training Run on ANE

Singh trained a 109M-parameter Llama2 Transformer on the ANE:

RMSNorm, QKV projections, and scaled dot-product attention (SDPA) ran on ANE in both forward and backward passes
~9.3 ms per training step
~11.2% ANE utilization, sustaining 1.78 TFLOPS

Gradient accumulation and the Adam optimizer still run on the CPU, but this constitutes a complete training loop—the first time a neural network has been trained on Apple's Neural Engine.

Limitations

Pipeline architecture: the ANE's 16 cores are pipelined; a single operation only reaches ~30% of peak performance. Chaining 16–64 operations into one large graph pushes utilization to ~94%.
Compile cap: resource leaks limit each process to 119 ANE program compilations, requiring careful compile-cache management during training.

Legal Gray Zone

The project's README cites Sega v. Accolade (1992) and DMCA §1201(f), which protect reverse engineering for interoperability. No Apple code was copied—only the behavior of hardware the developer purchased was studied.

Implications

The project is a research prototype, not production-ready. But it demonstrates that it was software, not hardware, that limited AI training on Apple devices. If Apple officially opened ANE training, it could enable privacy-preserving on-device model training, dramatically lower costs, and an order-of-magnitude improvement in energy efficiency compared to datacenter GPUs.

As Feynman put it: *"What I cannot create, I do not understand."*

References

1. GitHub: maderix/ANE — Training neural networks on Apple Neural Engine via reverse-engineered private APIs 2. maderix's Substack: "Inside the M4 Apple Neural Engine, Part 1: Reverse Engineering" (February 2026) 3. maderix's Substack: "Inside the M4 Apple Neural Engine, Part 2: ANE Benchmarks" (February 2026) 4. Hollance/neural-engine — Community ANE documentation (GitHub) 5. Apple: apple/ml-ane-transformers (GitHub)

*All technical figures come from the author's published measurements and the open-source project.*