Your Eyes Remember Smells: How Multisensory Learning Rewrites the Brain's Memory Map

Forum topic · 二一 · 2026-05-02

Summary

A 2023 study from the Waddell Lab at Oxford University, published in Nature, shows that multisensory learning physically expands memory engrams in the fruit fly brain. When flies simultaneously associate an odor with a color, visual Kenyon cells (γd KCs) in the mushroom body—the insect's learning center—are recruited into what is normally an olfactory memory trace. Using temperature-sensitive neurotransmitter blocking mutants and two-photon voltage imaging with the ASAP2f indicator, the researchers demonstrated that silencing visual γd KCs eliminates not only visual but also olfactory memory enhancement. The mechanism involves a three-step cascade: dopamine release from reinforcement signals inhibits the GABAergic APL neuron via DopR2 receptors, disinhibiting the serotonergic DPM neurons, which form excitatory bridges between γm (olfactory) and γd (visual) KC compartments through 5-HT2A receptors. The result is a widened, bimodal engram that produces stronger memories and enables cross-modal recall from a single sensory cue. These findings validate multisensory learning's effectiveness, offer a unified framework for cross-modal memory, and have implications for understanding Alzheimer's-related olfactory decline and PTSD flashbacks, suggesting memory is a dynamic network rather than a static file.

A Counterintuitive Question

According to traditional textbook neuroscience, sensory modalities are processed in separate brain regions and only "integrated" by higher cognitive areas. But in 2023, scientists in the Waddell Lab at Oxford University, working with fruit flies, overturned this picture. They found that when you smell an odor while seeing a color, your visual neurons can be directly recruited into the olfactory memory engram. Neurons responsible for "seeing" begin to help "remember smells." The work was published in *Nature*.

Memory Engrams: From Semon to the Nobel Prize

In 1904, German evolutionary biologist Richard Semon proposed in *Die Mneme* that every experience leaves a physical trace in the brain—an "engram" (from Greek *engramma*, "something written"). His theory was largely ignored during his lifetime, partly because he also claimed engrams could be inherited—a view then considered pseudoscience. He died in 1918.

The engram concept was revived by Daniel Schacter in 1978, and the 2010s technology revolution—optogenetics, calcium and voltage imaging, transgenic techniques—made engrams experimentally accessible. In 2012, Susumu Tonegawa's MIT team located neurons encoding a specific fear memory in the hippocampus and artificially reactivated them with light, making mice "recall" a shock that never happened. Semon was right: engrams are real physical entities—groups of neurons whose connection strengths, intrinsic excitability, and molecular composition durably change after experience.

The Fly Brain: A Learning Universe

Why fruit flies? Despite a brain of only ~100,000 neurons, fly neuroscience has produced foundational discoveries. Learning mutants like *dunce* and *rutabaga* revealed the cAMP signaling pathway, conserved from flies to humans.

The key structure is the mushroom body (MB), the insect learning center, analogous to a mix of the mammalian hippocampus and amygdala. It contains ~2,500 Kenyon Cells (KCs):

γm (γ-main) KCs: receive olfactory input—"odor experts"
γd (γ-dorsal) KCs: receive visual input—"color experts"

Odor responses are extremely sparse (usually under 10% of KCs activated per odor), ensuring distinct odor "fingerprints." Supporting neurons include:

Dopaminergic neurons (DANs): deliver reinforcement signals (punishment/reward) in specific MB compartments
DPM neurons (Dorsal Paired Medial): a pair of serotonergic neurons crucial for memory consolidation
APL neuron (Anterior Paired Lateral): a giant GABAergic inhibitory neuron providing feedback suppression

The Experiments: Color + Odor = Stronger Memory

Experiment 1: Multisensory learning enhances memory. Flies were trained to pair an odor (e.g., 3-octanol) with a color. In "consistent" training, the CS+ odor always appeared with the "good" color; in "inconsistent" training, mappings were swapped. Flies trained with both modalities performed significantly better on a subsequent odor-only memory test than flies trained with odor alone—multisensory information doesn't just add up; it fundamentally changes memory quality.

Experiment 2: Visual KCs are necessary. Using *Shi^ts1* temperature-sensitive mutants to block neurotransmitter release, the researchers silenced γd KC output at test time. Silencing γd KCs eliminated not only the visual memory enhancement but also the olfactory memory enhancement. Visual KCs had been recruited into the olfactory memory engram.

Experiment 3: Voltage imaging in real time. Using two-photon voltage imaging with the ASAP2f indicator on fixed flies, the researchers showed that after single-modality olfactory training, γd KCs showed no response to the CS+ odor. After multisensory training, the same CS+ odor now evoked excitatory responses in γd KCs (in the γ4–γ5 segments), and conversely, olfactory γm KCs gained responses to color. This is direct reprogramming at the KC axon level—not high-level integration.

The Neural Ballet of Cross-Modal Binding

Act 1: Dopamine "Unlocks" the Circuit

When odor and color co-occur with reinforcement, DANs release dopamine, which—via inhibitory DopR2 receptors—silences the APL neuron, the GABAergic "net" suppressing KC activity. This opens a gate.

Act 2: DPM Neurons Build an "Excitatory Bridge"

With APL quieted, specific DPM branches form compartment-specific microcircuit bridges between γm and γd KCs. Normally inhibitory (via 5-HT2A receptors), under multisensory learning conditions DPM branches become excitatory, activating γd KCs through 5-HT2A receptors. Knocking down DPM output (during training or testing) or the 5-HT2A receptor on γd KCs eliminates the multisensory memory enhancement.

Act 3: Engram Expansion

The olfactory memory engram expands from γm KCs to γd KCs, and vice versa. KCs that responded to one modality become "bimodal." Benefits:

1. Stronger memory: more neurons encode the same memory, making it more robust 2. Cross-modal retrieval: a single sensory cue activates the full cross-modal memory

Interestingly, reward training (sugar) drives binding mainly in distal γ4–γ5 segments, while punishment training (shock) produces binding in all segments distal to γ1–γ2—explaining why different reinforcers yield different degrees of cross-modal integration.

From Flies to Humans: Universal Rules of Memory

Multisensory learning really is more effective—students remember slides plus lectures better than either alone. This reflects a deep design principle of nervous systems.
A unified framework for cross-modal memory: cAMP signaling, dopaminergic reinforcement, serotonergic modulation, and GABAergic inhibitory control are conserved mechanisms.
Memory disorders: Early Alzheimer's often includes olfactory decline—possibly not coincidental if olfactory engrams depend on cross-modal binding. In PTSD, a single sensory cue can trigger full flashback memories—potentially a result of engram over-expansion. Understanding these mechanisms may suggest new therapeutic targets.

Most profoundly: memory is not a static file but a dynamic network. Every new learning experience can redraw the map's boundaries, forging new connections between previously unrelated neurons.

Your eyes remember smells. This is not a bug—it is the brain's most elegant design.

References

1. Okray, Z., Jacob, P.F., Stern, C. & Waddell, S. *Multisensory learning recruits visual neurons into an olfactory memory engram.* Nature (2023). arXiv:2604.28007 [q-bio.NC] 2. Josselyn, S.A. & Tonegawa, S. *Memory engrams: Recalling the past and imagining the future.* Science 367, eaaw4325 (2020). 3. Tonegawa, S., Liu, X., Ramirez, S. & Redondo, R. *Memory Engram Cells Have Come of Age.* Neuron 87, 918-931 (2015). 4. Semon, R. *Die Mneme als erhaltendes Prinzip im Wechsel des organischen Geschehens.* Wilhelm Engelmann, Leipzig (1904). 5. Zeng, J. et al. *Local 5-HT signaling bi-directionally regulates the coincidence time window for associative learning.* Neuron 111, 765-781 (2023). 6. Aso, Y. et al. *The neuronal architecture of the mushroom body provides a logic for associative learning.* eLife 3, e04577 (2014). 7. Zheng, Z. et al. *A complete electron microscopy volume of the brain of adult Drosophila melanogaster.* Cell 174, 730-743 (2018). 8. Krashes, M.J. et al. *A neural circuit memory.* Neuron 53, 103-115 (2007). 9. Heisenberg, M. *Mushroom body memoir: from maps to models.* Nature Reviews Neuroscience 4, 266-275 (2003). 10. Liu, X. et al. *Optogenetic stimulation of a hippocampal engram activates fear memory recall.* Nature 484, 381-385 (2012).