Hidden Algorithm of Pitch Perception: Audition Shares Motion-Detection Logic with Vision

> Core finding: Your brain decides whether a sound is rising or falling not by tracking the fundamental frequency (F0), but via a spectrotemporal correlation computation analogous to insect visual motion detection. A Yale team, publishing in *Nature Human Behaviour*, demonstrates that the human auditory system uses the same algorithmic logic as the visual system to detect "frequency motion"—including an auditory counterpart of the famous reverse-phi illusion.

Paper Details

| Field | Content | |-------|---------| | Title | Humans can use positive and negative spectrotemporal correlations to detect rising and falling pitch | | Authors | Parisa A. Vaziri, Samuel D. McDougle, Damon A. Clark | | Institution | Yale University | | Journal | Nature Human Behaviour | | DOI | 10.1038/s41562-025-02371-7 | | Sample | 125 participants + 5 fMRI participants |

Background: A Century-Old Puzzle

The traditional model holds that pitch perception relies on tracking the fundamental frequency (F0) over time. But many natural sounds—inharmonic percussion, noise, friction—lack a clear F0, and humans still perceive pitch changes in them. Recent work (e.g., McPherson & McDermott) suggested a lower-level mechanism: detecting *how* frequency content changes over time, rather than identifying specific frequencies.

Methods: Borrowing from Vision Science

The team adapted the correlation-based motion stimulus paradigm from vision research. They generated random spectrotemporal envelopes, then added a shifted copy of the envelope (time shift: 1/6 s; frequency shift: 1/15 octave, up or down; correlation: positive or negative). Crucially, all stimuli were inharmonic with no trackable F0—listeners could not follow any "tone" to judge direction.

Key Findings

• Positive correlations: Upward frequency shifts with positive correlation were heard as rising pitch; downward shifts as falling—far above chance, without any F0.
• Negative correlations (auditory reverse-phi): Upward shifts with negative correlation were heard as *falling*, and vice versa—the perceived direction inverted, exactly paralleling the visual reverse-phi illusion.
• Coherence manipulation: Mixing correlated and random noise showed perception approaching chance (0.5) as coherence decreased; (↑+) and (↓−) psychometric curves were indistinguishable, ruling out explanations based on tracking spectral patterns.

Neural Mechanisms

• Tuning curves: Peak sensitivity at ~40 ms temporal delay and ~1/15 octave (~4.7%) frequency offset—hallmarks of a motion detector rather than simple frequency discrimination.
• fMRI evidence: Responses to rising and falling stimuli could be cancelled by opponent (rising+falling) stimuli, analogous to motion opponency in visual cortex. This supports a "pitch direction opponency" hypothesis with opposing neural populations.
• Computational model: The classic Hassenstein-Reichardt correlator—developed in 1956 for insect visual motion—accurately explains the human auditory data. Positive correlations yield positive output ("rising"); negative correlations flip the sign of the cross-correlation, predicting the reversed percept.

Ecological Validity

Analysis of real English and Mandarin speech shows that intonation and tonal pitch changes naturally produce both positive and negative spectrotemporal correlations. Sensitivity to both provides redundancy and robustness—especially important for tonal languages like Mandarin, where pitch direction carries lexical meaning.

Implications

• Tonal language & speech recognition: Systems should incorporate spectrotemporal correlation features, not just F0 tracking.
• Auditory prostheses: Cochlear implants and hearing aids could enhance correlation coding rather than only spectral energy.
• Neuromorphic computing: The same correlational circuit architecture used for visual motion (e.g., event cameras) could process auditory pitch, enabling unified multimodal sensing hardware.

Open Questions

1. How do F0 tracking and correlation detection relate—complementary, hierarchical, or competitive? 2. How does the ~40 ms optimal delay reconcile with syllable durations (~100–300 ms)? 3. Did mammalian audition independently evolve the Hassenstein-Reichardt structure, or share an ancient neural architecture? 4. Where does the conversion from physical frequency to subjective pitch occur?

Outlook

The deepest implication is that perception may fundamentally be about *detecting how the world changes*, not identifying what it is. Correlation detection plus opponent mechanisms may be a universal computational primitive deployed across sensory modalities—suggesting that "auditory" and "visual" cortex labels may reflect inputs rather than fundamentally different circuits.

References

• Vaziri, P. A., McDougle, S. D., & Clark, D. A. (2026). *Humans can use positive and negative spectrotemporal correlations to detect rising and falling pitch*. Nature Human Behaviour. DOI: 10.1038/s41562-025-02371-7
• Anstis, S. M., & Rogers, B. J. (1975). Illusory reversal of visual depth and movement during changes of contrast. *Vision Research*, 15(8-9), 957-961.
• Hassenstein, B., & Reichardt, W. (1956). *Zeitschrift für Naturforschung B*, 11(9-10), 513-524.
• Clark, D. A., et al. (2011). Defining the computational structure of the motion detector in Drosophila. *Neuron*, 70(6), 1165-1177.
• McPherson, M. J., & McDermott, J. H. (2023). *Journal of Neuroscience*, 43(12), 2214-2229.
• Salazar-Gatzimas, E., et al. (2016). *Nature Neuroscience*, 19(10), 1312-1319.