Imagine a golf ball. Place it at the bottom of a smooth bowl—however you nudge it, it rolls back to the center. Now place it on the tip of a needle—one sneeze and it falls into an unpredictable corner.
Training a neural network is, at its core, about finding a bowl bottom, not a needle tip.
That's the core idea of Sharpness-Aware Minimization—SAM. Instead of using gradient descent to find the lowest point, you use gradient descent to find a *flat* lowest point. A flat bottom means: even if the test distribution shifts slightly, your accuracy doesn't fall off a cliff. A needle tip means: you happened to land exactly on that point during training, but a tiny shift at test time ruins everything.
This is clever, and it's been around for several years. But Dufort-Labbé, Hamidi, Pascanu, Mitliagkas, Scieur, and Baratin—six names, three from DeepMind, one from Samsung, two from Canada—recently pointed out a problem.
🧭 SAM Has a Blind Spot
What SAM does is straightforward: at each training step, it first takes a small step in the direction of steepest ascent of the loss ("how much would it hurt to be perturbed this way?"), then uses that pain signal to update the model. The scheme treats all parameter directions equally.
But the geometry of the loss landscape is not flat. It undulates violently in some directions and is mirror-smooth in others. Treating everything equally means: when you hit a gentle slope, you use the same SAM perturbation size as when you hit a cliff face.
It's like driving a road with the same suspension setting whether the surface has a giant pothole or a small crack. You jolt through the big pit, and you jolt through the little crack too. But if you knew what the road surface was like ahead—couldn't you adjust the suspension before driving through?
🕳️ The Problem Is Called "PotHole"
That's exactly the problem this paper tackles. They propose a method called LLQR+SAM. LLQR is a previously proposed second-order optimization method—it reformulates gradient descent as a layer-wise linear-quadratic regulator (LQR) problem. It sounds complicated, but the essence is: LLQR learns the *average curvature* of the terrain—which stretches are gentle and which are steep.
Their key insight: use the curvature information learned by LLQR as a *preconditioner* for SAM—first learn the terrain of this stretch of road, then decide the suspension setting.
The concrete approach uses two timescales. The LLQR part updates on a slow timescale—it maintains a smooth, low-resolution schematic map of the loss terrain via exponential moving averages. The SAM part operates on a fast timescale—it performs sharpness probing on the terrain map provided by LLQR. The two timescales aren't a coincidental convenience—they're necessary.
Because regions that look flat on the terrain map—directions that don't curve under the averaged geometry—may not actually be true bowl bottoms. They may be *pits*: flat from afar, sharp up close. The paper calls these "PotHoles."
> When I read this, I chuckled—"PotHole" is the perfect word. Loss landscapes really do have these: at "one epoch" resolution, it's smooth plains; at "one batch" resolution, it's full of cracks. If you use only one timescale—fast or slow—you'll miss the information from the other side.
The slow-timescale LLQR first draws an "averaged terrain map" showing where things look flat. The fast-timescale SAM then probes locally on that map—places that are flat on the average map but sharp under local probing are the pits that need extra attention. Genuinely wide bowl bottoms—flat at both coarse and fine granularity—won't be pushed away by SAM.
📊 What Are the Results?
They tested on standard vision benchmarks and sequence-modeling benchmarks. LLQR+SAM consistently outperforms SAM alone and LLQR alone. Not by a little—by sustained, reproducible margins.
This supports their core claim: slow-learned geometry and fast sharpness correction are genuinely complementary. The geometry tells you "the general shape of the road"; sharpness tells you "don't be fooled by surface illusions."
🤷 What I Don't Know Yet
A few things I couldn't figure out.
First, the paper says LLQR's preconditioner is "sparsely updated" and maintained via exponential moving averages. But I haven't fully grasped LLQR itself. I'm familiar with LQR's use in control theory, but what exactly does it get reformulated into in the context of layer-wise gradient descent for neural networks? How sparse is the preconditioner, exactly? I couldn't confirm these details from the abstract and title alone.
Second, does "PotHole" work equally well on large models—say, LLM training? The paper tested on standard vision and sequence-modeling benchmarks, but doesn't mention LLM-scale training. In billion-parameter models, estimating "flat" versus "locally sharp" could be very expensive. My guess is the computational overhead may not be negligible at LLM scale.
Third, I'm a bit unclear on how exactly the "slow and fast timescales" are interleaved during training. Does SAM run for several steps after each LLQR update? Or do both update in parallel with different step sizes? The title and abstract don't specify the scheduling details.
🛣️ But the Core Idea Is Clear
The loss landscape isn't a smooth slope. It has big undulations, small cracks, and pits that look flat but are actually sharp. A single suspension setting—a single perturbation size—can't handle all of them. Learn the terrain first, then adjust the suspension. That's the whole story.
---
References
1. Dufort-Labbé, S., Hamidi, M., Pascanu, R., Mitliagkas, I., Scieur, D., & Baratin, A. (2026). *Navigating Potholes with Geometry-Aware Sharpness Minimization*. arXiv:2605.16134 [cs.LG]. https://arxiv.org/abs/2605.16134 2. Foret, P., Kleiner, A., Mobahi, H., & Neyshabur, B. (2021). *Sharpness-Aware Minimization for Efficiently Improving Generalization*. ICLR 2021. 3. Hochreiter, S., & Schmidhuber, J. (1997). *Flat Minima*. Neural Computation, 9(1), 1-42. 4. Martens, J., & Grosse, R. (2015). *Optimizing Neural Networks with Kronecker-Factored Approximate Curvature*. ICML 2015. 5. LeCun, Y., Bottou, L., Orr, G. B., & Müller, K.-R. (1998). *Efficient BackProp*. In: Neural Networks: Tricks of the Trade, 9-50.