Your Eyes Remember Smells: How Multisensory Learning Rewrites the Brain's Memory Map
> *"If you could place a camera inside a fruit fly's brain, you would see something remarkable: when a fly 'smells' banana, some neurons in its visual brain region are firing furiously."*
---
1. A Counterintuitive Question
Imagine you are in a dark room and smell the aroma of baking bread. Your nose tells you "this is bread," but what is your brain doing? According to traditional textbooks, olfactory information enters the olfactory centers, visual information enters the visual centers, and each operates independently—only when higher cognitive areas "integrate" them do you realize "this is golden, steaming toast."
But in 2023, scientists in the Waddell Lab at Oxford University ran a series of elegant experiments in fruit flies that overturned this picture. They found that when you smell an odor while seeing a color simultaneously, your visual neurons are directly "recruited" into the olfactory memory engram. In other words, the neurons responsible for "seeing" now also help "remember smells."
This is not science fiction. It is hard-core neuroscience published in *Nature*.
---
2. Memory Engrams: From a Forgotten Semon to the Nobel Prize
To understand this discovery, we must return 120 years.
In 1904, German evolutionary biologist Richard Semon published *Die Mneme*, proposing a then-radical idea: every experience leaves a physical trace in the brain, which he called an "engram"—from the Greek *engramma*, "something written." Semon argued that when a stimulus activates a group of neurons, those neurons undergo "lasting chemical and/or physical changes," becoming an engram. Later, even the return of partial cues could reactivate the engram and evoke the full memory. He even proposed two laws: the "Law of Engraphy" describing how memory is written, and the "Law of Ephory" describing how it is read.
Semon's theory was almost entirely ignored during his lifetime—ironically, partly because he also proposed the even more radical idea that engrams could be inherited by offspring, which was viewed as pseudoscience in early 20th-century biology. Semon died in 1918, and his theory lay dormant for more than half a century.
In 1978, psychologist Daniel Schacter reconnected Semon's ideas with modern neuroscience. But what truly changed everything was a series of technical revolutions in the 2010s: optogenetics allowed precise optical control of specific neurons; calcium and voltage imaging made real-time observation of neural activity possible; transgenic techniques let researchers manipulate single genes in fruit flies, mice, and other model organisms.
With these tools, Susumu Tonegawa's team at MIT (Nobel laureate, 1987) made a milestone discovery in 2012: they not only identified the neuronal population in the hippocampus encoding a specific fear memory, but artificially activated these neurons with laser light, making mice "recall" a shock that never actually happened.
Semon was right. The engram is not a metaphor but a real physical entity—a group of neurons whose connection strengths, intrinsic excitability, and molecular composition are persistently altered by experience.
---
3. The Fruit Fly Brain: An Insect's Learning Universe
Why study memory in fruit flies? What can an insect the size of a grain of rice, with only ~100,000 neurons, tell us about human memory?
The answer: a surprising amount.
Fruit fly neuroscience is one of modern biology's great success stories. Starting in the 1970s, researchers identified the first learning and memory mutants—*dunce*, *rutabaga*—whose names sound like jokes but whose molecular mechanisms run deep. These genes all participate in the cAMP signaling pathway, which is highly conserved from flies to humans.
More importantly, the fly brain contains a structure called the Mushroom Body (MB).
The mushroom body is the insect brain's learning center, a kind of hybrid of the mammalian hippocampus and amygdala. It contains about 2,500 intrinsic neurons called Kenyon Cells (KCs), named after their discoverer Francis Kenyon. KC axons extend in different directions, forming the mushroom body's "cap" and "stalk"—hence the name.
In olfactory learning, odor information enters KC axon terminals via the antennal lobe. Each KC responds sparsely to odors—typically fewer than 10% of the 2,500 KCs are activated by a given odor. This sparse coding is the cornerstone of memory: it ensures that different odors' "fingerprints" don't blur together.
The mushroom body has multiple functional compartments. The gamma-KCs subdivide into two classes:
- γm (γ-main) KCs: mainly receive olfactory input—the "odor specialists"
- γd (γ-dorsal) KCs: mainly receive visual input—the "color specialists"
- Dopaminergic neurons (DANs): provide reinforcement signals. When a fly receives a shock (punishment) or sugar water (reward) after an odor, specific DANs release dopamine in particular mushroom body "compartments," tagging "this odor is bad/good."
- DPM neurons (Dorsal Paired Medial): a pair of serotonergic neurons that ramp up activity ~30 minutes after training and are crucial for memory consolidation.
- APL neurons (Anterior Paired Lateral): a giant GABAergic inhibitory neuron providing feedback inhibition across the mushroom body, preventing KC over-excitation.
- Congruent training: CS+ (target odor) always paired with the "good" color, CS− (control odor) with the "bad" color
- Incongruent training: the color–odor mappings swapped between training sessions
- Reward learning (sugar water): dopamine mainly acts on the γ4 and γ5 segments, so cross-modal binding occurs mainly in these distal segments
- Punishment learning (shock): dopamine acts on the proximal γ1 and γ2 segments, so binding occurs across all axon segments after γ1
- cAMP signaling
- Dopaminergic reinforcement learning
- Serotonergic modulation
- GABAergic inhibitory control
The mushroom body also receives support from key "helper" neurons:
This exquisite circuitry forms the hardware foundation of fly learning and memory.
---
4. The Experiments Revealed: Color + Odor = Stronger Memory
The Waddell team's experimental design is a model of neuroethology.
Experiment 1: Multisensory learning enhances memory
The researchers trained flies to associate a specific odor (e.g., 3-octanol) with a specific color (e.g., green). Training came in two forms:
Key finding: when flies learned color and odor simultaneously, even testing only olfactory memory (odor without color) showed performance significantly better than flies given olfactory training alone.
What does this mean? Multisensory information doesn't merely "add up"—it fundamentally changes the quality of the memory itself.
Experiment 2: Visual KCs are required
"Better performance" doesn't equal "changed mechanism." To prove multisensory learning truly rewires circuits, the researchers used the temperature-sensitive *Shi^ts1* mutant, which blocks neurotransmitter release at specific temperatures.
They used distinct GAL4 drivers to specifically silence γd KC (visual KC) output during testing.
The result was stunning: silencing γd KCs eliminated not only the visual memory enhancement after multisensory training, but also the olfactory memory enhancement. In other words, if the fly's "visual neurons" can't participate, it not only fails to remember colors—its odor memory also weakens.
This proves a remarkable conclusion: visual KCs have been recruited into the olfactory memory engram.
Experiment 3: Real-time voltage imaging
To directly observe this cross-modal recruitment, the researchers used a voltage indicator called ASAP2f for two-photon voltage imaging of γd KCs in fixed flies.
After single-modality olfactory training, presenting the CS+ odor produced no response in γd KCs—after all, they are "visual neurons."
But after multisensory training, the same CS+ odor now evoked excitatory responses in γd KCs. The γd KC axon terminals acquired odor responsiveness in the γ4–γ5 segments.
Conversely, the olfactory γm KCs also gained responses to color.
This is not the result of some high-level integration—this is direct reprogramming at the level of KC axons.
---
5. The Neural Ballet of Cross-Modal Binding
So how is this magic—"visual neurons becoming part of an olfactory memory"—achieved?
The Waddell team revealed a cascading mechanism, like a carefully choreographed neural ballet:
Act I: Dopamine "Unlocks" the Gate
When a fly smells an odor while seeing a color, odor and color respectively activate γm and γd KCs. Meanwhile, the reinforcement signal (reward or punishment) activates specific dopaminergic neurons (DANs).
Dopamine doesn't just mark "this stimulus is good/bad"—it also does something else: it inhibits the APL neuron.
APL is a giant GABAergic inhibitory neuron in the mushroom body, like a net normally draped over all KCs, preventing their over-excitation. Dopamine, by activating DopR2 receptors (an inhibitory receptor) on APL, quiets APL down.
This opens a door.
Act II: DPM Neurons Build an "Excitatory Bridge"
APL's silencing sets the stage for the next step.
DPM neurons are a pair of serotonergic neurons whose dendrites and axons span multiple mushroom body compartments. The key finding: specific DPM branches can form compartment-specific microcircuit bridges between γm and γd KCs.
Normally, DPM provides inhibitory feedback to KCs (via 5-HT2A and other receptors). But under multisensory learning conditions, specific DPM branches switch to excitatory connections—their released serotonin activates γd KCs through 5-HT2A receptors.
The researchers found that knocking down DPM output (during training or testing) abolished the multisensory memory enhancement. Knocking down 5-HT2A receptors on γd KCs had the same effect.
DPM neurons are the core bridge of this cross-modal binding.
Act III: Engram Expansion
The final result: the engram representing olfactory memory expands from γm KCs to γd KCs, and vice versa.
In single-modality learning, the memory engram mainly resides in directly activated KCs. But in multisensory learning, KCs that originally responded to only one sensory modality become "bimodal"—responding to both the original modality and the new one.
This "engram broadening" brings two practical benefits: 1. Stronger memory: more neurons encoding the same memory makes it more robust and harder to forget 2. Cross-modal retrieval: a single sensory cue (e.g., odor alone) can activate the complete cross-modal memory
Interestingly, reward and punishment training produce this binding in different mushroom body compartments:
This explains why different reinforcement signals lead to different degrees of cross-modal integration.
---
6. From Flies to Humans: Universal Laws of Memory
A fruit fly's visual neurons remember a smell—what does this mean for us humans?
First, it validates something we've long sensed in daily life: multisensory learning is indeed more effective than single-modality learning. In classrooms, students who see slides while hearing explanations remember more than those who only listen or only look. This is no coincidence—it reflects a deep design principle of nervous systems.
Second, the study provides a unified framework for explaining the neural mechanisms of cross-modal memory. Although the human brain is far more complex than a fly's, many core molecular mechanisms are conserved:
Likewise, in post-traumatic stress disorder (PTSD), a single sensory cue (like a certain smell) can trigger a full traumatic flashback—possibly the result of excessive engram expansion. Understanding cross-modal binding mechanisms could offer new intervention targets for treating PTSD.
Finally, and most profoundly: memory is not a static file but a dynamic network.
Traditional memory models imagine the brain as a computer—information is encoded, stored, and retrieved on demand. But the Waddell team's findings tell us memory is more like a constantly redrawn map. Each new learning experience can redraw the map's boundaries, forging new connections between previously unrelated neurons.
Your eyes remember smells. This isn't a bug—it's the brain's most elegant design.
---
References
1. Okray, Z., Jacob, P.F., Stern, C. & Waddell, S. *Multisensory learning recruits visual neurons into an olfactory memory engram.* Nature (2023). 2. Josselyn, S.A. & Tonegawa, S. *Memory engrams: Recalling the past and imagining the future.* Science 367, eaaw4325 (2020). 3. Tonegawa, S., Liu, X., Ramirez, S. & Redondo, R. *Memory Engram Cells Have Come of Age.* Neuron 87, 918-931 (2015). 4. Semon, R. *Die Mneme als erhaltendes Prinzip im Wechsel des organischen Geschehens.* Wilhelm Engelmann, Leipzig (1904). 5. Zeng, J. et al. *Local 5-HT signaling bi-directionally regulates the coincidence time window for associative learning.* Neuron 111, 765-781 (2023). 6. Aso, Y. et al. *The neuronal architecture of the mushroom body provides a logic for associative learning.* eLife 3, e04577 (2014). 7. Zheng, Z. et al. *A complete electron microscopy volume of the brain of adult Drosophila melanogaster.* Cell 174, 730-743 (2018). 8. Krashes, M.J. et al. *A neural circuit memory.* Neuron 53, 103-115 (2007). 9. Heisenberg, M. *Mushroom body memoir: from maps to models.* Nature Reviews Neuroscience 4, 266-275 (2003). 10. Liu, X. et al. *Optogenetic stimulation of a hippocampal engram activates fear memory recall.* Nature 484, 381-385 (2012).