Google DeepMind Leaders Roundtable: A Strategic Retrospective, Not a Product Launch

On May 30, Google released a conversation video featuring four key figures who shaped its AI over the past decade: Jeff Dean (founder of Google Brain), Noam Shazeer (co-author of the Transformer paper), Oriol Vinyals (who leads Gemini), and Koray Kavukcuoglu (DeepMind CTO), hosted by Logan Kilpatrick. The nearly two-hour talk, nominally about Gemini 3.5 Flash, is really a strategic retrospective on where Google's AI is heading over the next five years.

1. Gemini Is Not a Model — It's an Organizational Restructuring

Jeff Dean was blunt: before Gemini, Google's internal teams (Pathways, PaLM, PaLM 2) each pulled their own compute and built their own models. "This was stupid," he said. To build a sufficiently strong model, people, compute, infrastructure, and data all had to be consolidated. Koray added that early AI research resembled academia with parallel exploration, but at scale it had to become one centralized mega-project. Gemini, then, is essentially the merger of Google Brain and DeepMind's scattered capabilities.

2. Flash Is About Coding and Agents, Not Speed

Oriol emphasized that Gemini 3.5 Flash's headline improvement is coding ability. Koray: "This is the era where coding capability and agent experiences define AI experiences." Competition is no longer about chat quality — whoever gets models into real workflows (writing code, calling tools, executing multi-step tasks) wins. Noam noted Gemini first becomes Google's internal R&D tool, then an external product.

3. Benchmarks vs. Real Users

Noam's memorable line: "The real test is whether users are using it. Optimizing in a box on benchmarks gets you benchmark optimization — and possibly data contamination." This explains Google's push to embed Gemini into Search, Workspace, Android, and glasses. The frontier, Koray argued, is both research capability and what products let users do.

4. World Models: From Understanding the Internet to Understanding the World

Koray described Omni not as text-to-video, but as a model understanding all physical modalities — vision, dynamics, physical laws — and able to simulate future states for decision-making. Oriol noted that large-scale joint training now yields emergent temporal consistency, 3D understanding, and audio. Jeff extended multimodality beyond text/image/audio/video to gene sequences, chemical structures, robot manipulation data, and LiDAR.

5. Squeezing the Lemon: Distilling Pro into Flash

Oriol's biggest surprise: Pro-level capability can now be compressed into Flash generation after generation — sometimes new Flash beats the previous Pro. Koray's metaphor: "Like squeezing a lemon — the juice goes into a glass, and the glass is the small model." Jeff explained the teacher-student distillation logic is essentially unchanged from the original paper, except one top teacher now replaces a 50-model ensemble. Google's ambition is making strong models cheap enough for billions of users.

6. What's Still Unsolved

• Continual learning: current MoE experts are too structurally similar; Jeff wants more "organic," fluid architectures.
• Evaluation: Oriol argued evaluation is underrated — measuring decomposed capabilities, data leakage, and user validation are all hard.
• Sample efficiency: humans hear ~1 billion words in a lifetime; models need trillions of tokens. Noam quipped humans have only a few GB of "source code" (DNA). Oriol countered that humans are also "pretrained" by evolution — a point of debate.

7. Self-Learning: Gemini Improving Gemini by 2027

Koray predicted self-learning discussions will begin around next year: increasingly agentic, code-fluent models doing research, reaching a point where model improvements to Gemini itself are made by models. Noam: experiments previously assigned to team members will be handed to models. Gemini shifts from product to part of Google's R&D system.

8. The 30-Day Agent: A Tooling Problem, Not a Model Problem

Logan floated a goal: by the 2027 I/O, a model running autonomously for 30 days. But Noam pointed out ~29.5 of those days could be waiting — today's tools are designed for human latency. The agent era requires rethinking the entire software toolchain for machine-speed invocation; many "advanced" tools are just electronic furniture built for slow human clicking.

9. How Many Products Will Google Have in Five Years?

• Koray: one — the product is the model.
• Jeff: many product "outlets," but fewer, more concentrated foundations underneath.
• Oriol: cautiously disagrees — human agency in choosing tasks may keep multiple products alive.
• Noam: information products will be consumed via text, vision, glasses, brain-computer interfaces — and eventually models will move atoms, not just bits.

Conclusion

Gemini is not a single-model competition. It's increasingly Google's core operating layer for future AI: models for understanding and generation, agents for long-horizon execution, products for real feedback, hardware for cost and latency, and toolchains for real work. Whoever plugs models into their own research, product, and infrastructure loop — letting them compound — takes the lead.

*Source: Official Google conversation video (May 30), hosted by Logan Kilpatrick; compiled from multiple sources.*