> Source commit: 2c47ab1 — easy-learn-ai daily update
> Original post date: 2026-04-07
Imagine you own a super-thick encyclopedia — a behemoth of over five billion characters. The traditional approach requires cramming the entire book into your brain (or, say, your computer's memory) before you can start reading and using it. It's as absurd as having to move a whole cow into the kitchen just to eat one dish.
But the Gemma 4 team suddenly asked: "Wait, why does it have to be this way?"
They invented a technique called Per-Layer Embeddings (PLE). It sounds like a mouthful of academic jargon, but the concept is so simple you'll want to slap your thigh.
---
An Encyclopedia You Can "Read Page by Page"
A traditional AI model is like a book with all its content printed on the cover. Want to turn to page 300? Fine, but everything between the cover and page 300 must first be stuffed into your head. That's why large models are so "VRAM-hungry" — to use one, you first have to feed its memory appetite.
What does Gemma 4's Per-Layer Embeddings do?
It splits the encyclopedia into two parts:
- The vocabulary store (2.8B parameters): like a dictionary holding the meaning of every word. This part is static, generic, and doesn't change with context.
- The reasoning part (2.3B parameters): the core brain actually doing the "thinking" and "writing."
- Apple's MLX framework fully exploits the unified memory architecture
- Raspberry Pi 5 with an M.2 HAT+ can mount an SSD directly
- PCIe Gen3 pushes external storage read speeds above 800MB/s
- PLE explainer thread
- Engram-related paper citations
- Gemma 4 official page
The key point: that huge vocabulary can live on a hard drive, a USB stick, or in the cloud. When you need to look up a word, you fetch it temporarily — no need to keep it memorized at all times.
---
Why Is This Cool?
Here's a more down-to-earth analogy.
Imagine you're a chef. The traditional way is to move the entire farmers' market into your kitchen — from carrots to truffles, from soy sauce to caviar, everything must be at hand before you can start cooking. That takes up space, and every time you stir-fry you have to dig through mountains of ingredients.
Per-Layer Embeddings is like keeping your frequently used seasonings and tools on the countertop (those 2.3B parameters), while the massive pantry (the 2.8B dictionary) sits in the next room. Fetch what you need, use it, put it back — no counter space wasted.
The result?
You can run a five-billion-parameter model on an iPhone.
Yes, you read that right. Not a model trimmed down to the bone, but a complete model with a genuine "five-billion-word vocabulary." On an iPhone 17 Pro, running on the MLX framework, it hits roughly forty tokens per second. That's a practically usable speed — no longer the toy where "you can go brew a coffee while it outputs one sentence."
---
Technical Details: How Does It Actually Work?
Let's go a bit deeper — Feynman style, as if explaining to a smart ten-year-old.
When an AI model processes text, the first step is "embedding": converting human language into numeric vectors a machine can understand. The word "cat" becomes a list of several hundred numbers capturing everything "cat" means: it's an animal, has four legs, catches mice, is cute, and so on.
In traditional models, every layer redoes this embedding. It's like a student reading a book who re-consults the dictionary for every word on every page. Clearly wasteful.
Gemma 4's Per-Layer Embeddings says: "Don't be silly. Compute all word embeddings once, store them, and every subsequent layer just reuses them."
But that's not even the revolutionary part. The real revolution: these precomputed embeddings can live on disk, not in memory.
How? Because embeddings are "static" — the meaning of the word "cat" doesn't depend on context. Whether it appears in "the cat catches mice" or "Schrödinger's cat," its basic meaning as a word is the same. Only when the model starts "understanding" context do the context-sensitive parameters come into play.
So Gemma 4 turns those 2.8B embedding parameters into a giant lookup table stored on disk. At runtime, the model loads it on demand via memory mapping (mmap) — just like an OS reading files from disk only when needed.
---
What Does This Mean?
A few practical implications:
1. The True Dawn of Edge AI
Before, running a large model on a phone meant quantizing it — trading precision for space, like compressing an HD photo into a mosaic. No longer necessary. Gemma 4 E2B runs at over forty tokens per second on a Raspberry Pi 5, using Q8_0 quantization (nearly lossless).
This means your phone, smart home devices, even your car can run AI that genuinely "knows a lot," not just simple rule-based assistants.
2. Real Privacy Protection
When models run locally, data never needs to go to the cloud. Want AI to organize medical records or analyze financial data? No problem — everything happens on your device, and no data leaves your control.
3. A Cliff-Dive in Cost
Cloud inference is billed per token. The marginal cost of local inference approaches zero. If an enterprise can handle 80% of daily tasks with local models, what happens to its AI bill?
4. A New Paradigm for Architecture Design
Per-Layer Embeddings reveals a more general design principle: separate "knowledge" from "capability." Knowledge can be heavy but static; capability must be light but flexible. This points to a new path for future model design — perhaps we'll see more "small knowledge brain + large knowledge warehouse" hybrid architectures.
---
Who Else Is Playing This Game?
Gemma 4 isn't alone.
Community members are porting ESM-2 (a protein-modeling model) to MLX so Macs can run biological sequence modeling locally. Others are bringing Falcon Perception (an image segmentation model) so local devices can do complex visual understanding.
There's a clear trajectory here: moving models that once ran only in data centers onto consumer devices.
Behind this is synchronized hardware-software evolution:
Software and hardware are pushing each other in a virtuous cycle.
---
Final Thoughts
Per-Layer Embeddings may not become the next endlessly cited "breakthrough paper." It proposes no new training algorithm and creates no new mathematical framework.
But it solves a real problem: how to turn AI from a "data-center luxury" into "everyone's everyday tool."
And that may be worth more than any theoretical breakthrough.
---
Further reading: