An Elephant in Your Pocket: How Gemma 4 Brings AI from the Cloud to Your Jeans

*An English translation of a Chinese tech forum post (zhichai.net), written in a Feynman-style explainer voice.*

1. A Strange Phenomenon

In the first week after Gemma 4's release, the model hit 2 million downloads - not views, not likes, but 2 million people actually installing it on their own devices. The author compares it to a new restaurant with queues three blocks long on opening day.

What's odd is that this reverses years of "bigger is better" narrative. GPT-4 requires an internet connection; Claude's Opus runs in the cloud. But Gemma 4? People are running it on an iPhone 17 Pro at ~40 tokens/second (roughly human speaking speed), on a Raspberry Pi 5 (a board cheaper than a smart speaker), and - in the author's words, by absolute madlads - on a 1998 iMac G3 with 32MB of RAM (a TinyStories variant, not full Gemma 4).

2. Parameters Are Weight, Not Wisdom

More parameters ≠ smarter. Parameters are like dictionary entries: stored potential, not knowledge. And they must all fit in compute memory when the model runs - which is why large models have lived in the cloud.

Gemma 4 splits its 5.1 billion parameters into two piles:

• ~2.3 billion "active" parameters - kept in memory and invoked in real time.
• ~2.8 billion "sleeping" parameters - the Per-Layer Embeddings: static, position-independent token embeddings, like dictionary entries that don't change with context. These are offloaded to storage/flash memory and only paged in when needed.

That's the secret of stuffing the elephant into the fridge: you don't put the whole elephant in at once. You put in one leg, and fetch the rest as needed.

3. MoE: Not Every Expert Needs to Show Up to Work

Mixture of Experts works like a hospital with 26 departments: for a headache, only neurology sees you - the other 25 departments exist but aren't activated.

Gemma 4's 26B MoE version has 26 expert parameter subsets. For each input, a router wakes only the 2 most relevant experts (~2.6B active parameters per query out of 26B total). A community comparison: a code audit took 30-50 minutes with the dense 31B model, but 2 minutes with the 26B MoE - a full order of magnitude.

The author stresses this isn't cutting corners: each expert is still deep and complete in its domain. It's *smarter use* of parameters, not fewer parameters.

4. Speed Isn't Everything, But It Changes Everything

40 tokens/second on iPhone is over the threshold where humans stop perceiving latency - interaction feels continuous, like talking to a real person. The Raspberry Pi 5 (credit-card-sized, fanless, cheap) previously couldn't run LLMs at all; now it can, if slower. As the author puts it: previously only F1 cars could race the track; now your family grocery-car can finish a lap too. That's transformative for people who can't own the F1 car.

5. $0.20 and 1100% ROI

On FoodTruck Bench (an agentic benchmark testing planning and execution, not just Q&A), Gemma 4's 31B version ranked third, behind GPT-4 and Claude 3 Opus - at $0.20 per run, cited as over 1100% ROI.

The author notes $0.20 is arguably conservative: Gemma 4 is open source, so self-hosting on existing hardware pushes marginal cost toward zero. For companies running thousands of audits daily, the savings versus multi-dollar GPT-4 calls could reach hundreds of thousands of dollars a year.

6. The 1998 iMac and TinyStories

A 1998 iMac G3 (32MB RAM - smaller than a photo from your phone) can run TinyStories, a model compressed to generate children's stories. As a party trick, it's fun; as a signal, it shows AI is becoming ubiquitous - not just in GPU datacenters, but on any device with electricity. Future fridges, door locks, and hearing aids with tens of megabytes of memory could converse, authenticate voices, and translate.

7. So What?

The real change is the distribution of power. Cloud AI has been feudal: lords (OpenAI, Google, Anthropic) own the land (compute and models); peasants rent (API calls) under their rules and prices. Gemma 4 is a democratizing counterweight: open weights, no permission needed, fine-tunable, yours.

Practical benefits:

• Privacy: data never leaves your device - novels, medical records, financials can be processed fully offline.
• Reliability: no dependence on remote datacenters that can go down (as ChatGPT has done at scale).
• Accessibility: billions of people lack reliable broadband; local AI reaches them without waiting for fiber.

8. But It Doesn't Mean Everything Gets Easy

A dose of cold water:

• Local deployment still requires technical knowledge (MLX, quantization, memory mapping) - there's a barrier for ordinary users.
• Local isn't free: hardware costs money, and Raspberry Pi performance is limited.
• Local models still trail the biggest cloud models on complex tasks. Gemma 4 is "good enough", not "the best".

This isn't a "local replaces cloud" story - it's a "more choices" story. Need peak performance? Cloud. Privacy and reliability? Local. Tight budget? MoE version. That diversity, not monopoly, is what a healthy ecosystem looks like.

9. What Does the Future Look Like?

The author closes in Feynman fashion: admitting not knowing. Unknown whether Gemma 4 marks the pivot from cloud era to local era; unknown Google's true motives; unknown whether everyone will carry a personal AI in five years. But a decade ago nobody believed phones could run neural networks; a year ago nobody believed open models could approach closed ones. The "impossibles" are temporary. Two million people have already voted with their downloads - driven not by work or profit, but curiosity. In Feynman's view, that's how all good things begin.

---

*Translation note: all figures, benchmarks, and claims above are reported as stated in the original Chinese forum post and have not been independently verified.*