The McGurk effect is a perceptual illusion in which what you see a person’s mouth doing changes what you hear them say. When an audio recording of one syllable is paired with video of a mouth producing a different syllable, most people hear a third syllable that matches neither the sound nor the lip movements alone. The classic example: audio of “ba” dubbed over video of someone saying “ga” typically produces the perception of “da.”1Frontiers in Psychology (via Europe PMC). What is the McGurk effect? First described in 1976, the effect has become one of the most studied demonstrations in perception science, and it reveals something fundamental about how the brain processes speech.
What Your Brain Is Actually Doing
When you listen to someone talk, your brain is not just decoding sound waves. It is simultaneously reading lip movements, jaw position, and tongue placement, then blending all of that with the audio signal to arrive at a single percept. Under normal conditions this integration is seamless and helpful: seeing someone’s mouth move makes it easier to understand them, especially in a noisy room. The McGurk effect exposes this process by deliberately putting the audio and visual signals in conflict. Because the brain cannot reconcile “ba” sounds with “ga” lip movements, it settles on a compromise, often “da,” which splits the difference in terms of where in the mouth the consonant is produced.
This is not a conscious decision. You cannot simply choose to hear the audio correctly once you are watching the face. Close your eyes and the illusion vanishes instantly; the audio snaps back to “ba.” Open them again and “da” returns. That automatic, involuntary quality is what makes the McGurk effect such a useful window into how the brain merges sensory channels.
Where It Happens in the Brain
The left superior temporal sulcus, a groove running along the side of the brain roughly behind the ear, is the region most consistently linked to the McGurk effect. Brain imaging studies show that the strength of activity in this region correlates with how susceptible a person is to the illusion: a weak response in the left STS means a person is less likely to perceive the fused syllable, while a strong response means they are more likely to perceive it.2PubMed Central. A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion When researchers used transcranial magnetic stimulation to temporarily disrupt the STS, the McGurk illusion weakened significantly, but only in a narrow window from about 100 milliseconds before the sound onset to 100 milliseconds after it. Stimulating a control brain region had no effect.3PubMed Central. fMRI-Guided transcranial magnetic stimulation reveals that the superior temporal sulcus is a cortical locus of the McGurk effect
An earlier hypothesis suggested that the motor system, specifically regions involved in planning mouth and tongue movements, might drive the effect by simulating the speaker’s articulations. Research on this has gone in both directions. One study found that disrupting premotor cortex impaired the ability to distinguish speech sounds in noise, suggesting the motor system plays some role in speech perception broadly.4PubMed Central. The essential role of premotor cortex in speech perception But when researchers tested whether silently mouthing syllables (which ties up the motor speech system) reduced the McGurk effect, it did not. And imaging data confirmed that motor speech regions did not show the audio-visual integration profile seen in the STS.5PubMed Central. Audiovisual speech integration does not rely on the motor system: evidence from articulatory suppression, the McGurk effect, and fMRI The current consensus points to the STS as the main site where auditory and visual speech signals are merged, though the motor system may still contribute to speech perception in other ways.
Why Some People Experience It More Than Others
Not everyone perceives the McGurk effect equally. In a typical experiment, some participants report the illusion on nearly every trial while others rarely experience it. The single best predictor of susceptibility turns out to be lipreading skill, specifically the ability to extract fine-grained information about where in the mouth a consonant is being produced. Measures of attention, processing speed, and working memory did not predict susceptibility.6PLoS ONE. What accounts for individual differences in susceptibility to the McGurk effect?
Eye gaze plays a role, too. People who frequently experience the McGurk illusion tend to spend more time fixating on the speaker’s mouth. In one study, strong perceivers spent about 41% of their fixation time on the mouth compared to 23% for weak perceivers, and each 2% increase in mouth-looking time corresponded to roughly a 10% increase in how often the illusion was perceived.7PubMed Central. A link between individual differences in multisensory speech perception and eye movements That said, the illusion is remarkably robust even under unfavorable viewing conditions. Experiments that forced participants to look away from the mouth found that the effect persisted until gaze was displaced more than 10 to 20 degrees from the mouth and did not become negligible until about 60 degrees away.8PubMed. Gaze behavior in audiovisual speech perception: the influence of ocular fixations on the McGurk effect So while looking at the mouth strengthens the illusion, you do not need to stare directly at the lips to experience it.
How Noise Changes the Balance
In everyday life, background noise degrades the audio signal and makes your brain lean more heavily on visual cues. This same principle shapes the McGurk effect. Adding noise to the audio increases the rate of McGurk illusions, because the brain compensates by weighting the visual channel more. Blurring the video, on the other hand, decreases the illusion. Participants also shift their gaze in response: noisy audio leads to more mouth fixation, while degraded video leads to less.9PubMed Central. Audio-visual integration in noise: Influence of auditory and visual stimulus degradation on eye movements and perception of the McGurk effect The brain is constantly reweighting its sensory inputs based on which channel is more reliable at any given moment.
One subtlety researchers have flagged: being good at the McGurk illusion does not necessarily mean you are better at understanding real speech in noise. The McGurk effect involves conflicting cues between the eyes and ears, which introduces an additional source of variability that does not apply when audio and video match.10PubMed Central. Weak observer-level correlation and strong stimulus-level correlation between the McGurk effect and audiovisual speech-in-noise: A causal inference explanation In real conversation, audio and video agree, so being highly susceptible to the McGurk illusion does not straightforwardly translate to better listening skills in a crowded restaurant.
The Timing Does Not Have to Be Perfect
You might expect that the audio and video need to be precisely synchronized for the illusion to work, but the brain gives itself a generous window. Experiments have shown that the McGurk effect persists even when the audio lags behind the video by up to 180 milliseconds.11PubMed. Temporal constraints on the McGurk effect This tolerance for asynchrony makes sense from a biological standpoint: light travels faster than sound, so in real life the visual signal from a distant speaker will always arrive slightly before the audio. The brain has evolved to handle that offset rather than reject the visual information as irrelevant.
How the Effect Develops with Age
Children are less susceptible to the McGurk effect than adults. In studies comparing age groups, children between 3 and 9 years old reported more correct auditory responses and fewer fused (illusory) percepts than adults. This gap narrowed with age, and by 10 to 12 the children’s responses were statistically indistinguishable from adult levels. Correct auditory responses dropped by about 1.6% per year of age, suggesting a gradual shift from auditory dominance in childhood toward the heavier visual weighting seen in adults.12Scientific Reports. The threshold for the McGurk effect in audio-visual noise decreases with development
Even infants show some sensitivity to audiovisual speech. Babies as young as five months perceive the McGurk effect when watching a speaker whose face matches their own racial background, but not when watching a speaker of a different race.13PubMed Central. The other-race effect on the McGurk effect in infancy This mirrors the well-documented narrowing of face perception in infancy, where babies gradually specialize in the faces they see most often.
At the other end of the age spectrum, older adults tend to be more susceptible to the McGurk effect than younger adults. A recent meta-analysis of nine studies found a small-to-medium effect, with older adults more likely to perceive the illusion.14PubMed. A Systematic Review and Multilevel Meta-Analysis of the McGurk Effect in Older and Younger Adults One likely explanation is that age-related hearing decline prompts the brain to rely more heavily on visual speech cues, amplifying the illusion.
Language Matters
The McGurk effect is not equally strong across all languages. Japanese listeners hearing Japanese syllables, for instance, show a weaker effect than English listeners hearing English, particularly when the audio signal is highly intelligible.15PubMed. McGurk effect in non-English listeners: few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility The differences likely reflect how much a given language relies on visible mouth distinctions to differentiate sounds. Languages whose key consonant contrasts happen deep in the mouth or throat, where lip movements are less informative, give the visual channel less to work with. The brain adjusts its weighting accordingly.
Autism and Schizophrenia
The McGurk effect has drawn considerable attention in clinical research because altered multisensory integration can ripple outward into social communication. A meta-analysis of nine studies found that people with autism spectrum disorder show a substantially weaker McGurk effect than typically developing controls.16PubMed. McGurk Effect by Individuals with Autism Spectrum Disorder and Typically Developing Controls: A Systematic Review and Meta-analysis The reduced illusion does not appear to stem from an inability to lipread. In one study, individuals with high autistic traits performed normally on lipreading tasks and on audiovisual congruent speech, yet still showed a weaker McGurk effect when the audio and video conflicted.17PubMed Central. Intact lip-reading but weaker McGurk effect in individuals with high autistic traits This pattern suggests the issue is specifically in the integration step, not in either sensory channel alone. And there is a functional connection: among children with and without autism, those who perceived the McGurk illusion more frequently tended to have lower levels of autistic features and stronger communication skills.18PubMed Central. Relations Between the McGurk Effect, Social and Communication Skill, and Autistic Features in Children with and without Autism
Schizophrenia also shows reduced McGurk susceptibility. Individuals with schizophrenia experience the illusion less frequently than controls despite performing comparably on non-illusion audiovisual trials.19PubMed Central. Eluding the illusion? Schizophrenia, dopamine and the McGurk effect This has been interpreted in terms of altered probabilistic inference: the brain in schizophrenia may weight prior expectations differently, making it less likely to merge conflicting sensory signals into a single fused percept. The pattern holds in adolescents with schizophrenia as well, who showed significantly fewer McGurk responses than age-matched healthy peers.20PubMed. Differences in audiovisual integration, as measured by McGurk phenomenon, among adult and adolescent patients with schizophrenia and age-matched healthy control groups
Cochlear Implants and the Visual Shift
People who lost their hearing after learning to speak and later received cochlear implants present an interesting case. These individuals do experience the McGurk effect, but with a notable difference: they show a stronger visual bias than people with normal hearing. Where a normally hearing person tends to balance auditory and visual cues fairly evenly, cochlear implant users lean more heavily toward the visual signal during incongruent speech.21PubMed. McGurk effects in cochlear-implanted deaf subjects This makes sense given that during the period of deafness, these individuals depended entirely on visual communication. Even after the implant restores some auditory input, the brain’s weighting does not fully snap back to the hearing-typical balance. Researchers have used this population to develop better tools for measuring audiovisual integration, confirming the prediction of stronger integration in cochlear implant users through probabilistic modeling.22PubMed. McGurk stimuli for the investigation of multisensory integration in cochlear implant users: The Oldenburg Audio Visual Speech Stimuli (OLAVS)
The Illusion Can Permanently Change How You Hear
One of the more striking recent findings is that the McGurk effect is not just a momentary trick. Experiencing the illusion can actually recalibrate how you perceive speech sounds afterward, even without any visual input. In one experiment, people who perceived the fused “da” during a McGurk trial were significantly more likely to mishear a subsequent audio-only “ba” as “da.” The probability of this misperception jumped by about 80% compared to trials that followed non-McGurk stimuli.23Scientific Reports. McGurk illusion recalibrates subsequent auditory perception This rapid recalibration has been replicated and quantified: sensitivity to the difference between “ba” and “da” decreased after a McGurk illusion compared to after hearing either syllable on its own.24PubMed Central. Rapid recalibration of speech perception after experiencing the McGurk illusion
Even more remarkable, repeated exposure to McGurk stimuli over two weeks induced changes in auditory perception that lasted a year or longer in some participants. The audio component of the McGurk stimulus began evoking the fused percept even when presented alone, without any visual speech. This long-lasting shift was specific to the particular speaker and syllable used during exposure, and it occurred only in people who actually experienced the illusion, confirming that the recalibration was driven by the conflict between what the brain predicted and what it heard.25PubMed Central. Repeatedly experiencing the McGurk effect induces long-lasting changes in auditory speech perception The brain does not just passively blend senses in the moment. It uses the outcome of that blending to update its expectations for future sounds.
How Researchers Model the Effect
Computational models have been developed to explain not just that the McGurk effect happens, but why it happens for some audio-visual pairings and not others. A causal inference model of multisensory speech perception starts by asking a basic question: did the audio and the video come from the same source? If the brain decides they did, it merges them. If it decides they did not, it keeps them separate. When applied to data from 60 participants perceiving both McGurk and non-McGurk incongruent stimuli, this model successfully predicted which pairings would produce illusions and which would not. A version of the model without causal inference failed at both tasks.26PLOS Computational Biology. A Causal Inference Model Explains Perception of the McGurk Effect and Other Incongruent Audiovisual Speech
Bayesian models take this further by breaking the process into two stages: binding, which is influenced by prior knowledge about which sounds and sights tend to co-occur, and fusion, which is influenced by how reliable each sensory signal is. These models account for both the McGurk illusion and the everyday benefit of congruent audiovisual speech, treating them as two outcomes of the same computational process.27PubMed Central. Bayesian binding and fusion models explain illusion and enhancement effects in audiovisual speech perception The practical takeaway from this modeling work is that the McGurk effect is not a quirky failure of perception. It is a natural consequence of a system that is designed to combine all available information, and that usually works extremely well.
Touch Influences Speech Perception Too
The principle behind the McGurk effect extends beyond vision. Researchers have shown that touch can alter speech perception in a strikingly similar way. Some speech sounds produce tiny puffs of air when spoken. The English “p” in “pa,” for example, creates a small burst of aspiration that “ba” does not. When experimenters applied inaudible air puffs to participants’ skin at the moment they heard a syllable, the puff made them more likely to hear an aspirated sound, causing them to mishear “ba” as “pa.”28PubMed Central. Aero-tactile integration in speech perception Speech sounds also activate somatosensory brain areas, the regions that process touch and body sensation, suggesting that the brain’s representation of speech is deeply multisensory rather than strictly auditory.29PubMed Central. Towards a somatosensory theory of speech perception
This broader picture helps explain why the McGurk effect exists at all. Speech perception evolved not as a purely auditory process but as a system that exploits every available cue, from lip shape to air flow, to figure out what someone is saying. The McGurk effect is simply what happens when that system receives contradictory inputs it was never designed to encounter.
Shared Roots Across Species
Humans are not the only species that combine faces and voices. Monkeys in noisy environments also use visual information from faces to improve their detection of vocalizations, and their integration behavior parallels that of humans performing the same task.30PubMed Central. Monkeys and Humans Share a Common Computation for Face/Voice Integration This suggests that audiovisual integration in communication did not arise with human language but builds on a much older capacity for combining sensory signals. The McGurk effect, then, is a modern laboratory artifact of a system that predates speech itself.