We humans are laboring under the sometimes false impression that our sensory organs, and the brain integrative centers that interpret the input they provide, are reliable. We hope that they are reliable most of the time -- after all, science itself would be a massive self-delusion if the percent of wrong data our senses provide was much larger than 1%. The error, of course, is in assuming that because our sensory organs and brain are reliable much of the time, that they are reliable all of the time.
The most common example of sensory misinterpretation is the optical illusion. There are a number of weird, and largely unexplained, optical illusions at this page (I use several of the ones on this site in my Brain & Senses class when we discuss the visual integrative systems in the brain). And while these illusions are charming and fascinating, there's a lesser-known one that I want to consider today.
Called the McGurk effect, this phenomenon is a type of brain confusion that occurs when what your ears are telling you and what your eyes are telling you are at odds. Think, for example, of how much easier it is to understand someone's speech in a crowded, noisy pub if you're looking at him while he talks. (The same thing in part explains why it's so easy to mishear someone on the telephone, when you have no visual cues to support what you're hearing.)
What if, however, what you're hearing and what you're seeing don't line up? Common sense might dictate that since what we're talking about is the interpretation of sounds, that hearing would win -- that if your ears told you that you were hearing one phoneme, and your eyes told you you were hearing another, your brain would give more credence to your ears.
This, in fact, isn't what happens, and thus the McGurk effect. Watch the following video (here) if you don't believe me. In it, psychologist Dr. Lawrence Rosenblum forces your brain into a perceptual no-win situation by saying the syllable "ba" while overlapping it with a video of him saying "va." Amazingly enough, the brain hears "va."
And the effect is quite robust -- you can't make it go away once you've understood what's going on, the way you can with many simple optical illusions. It's instantaneous and quite unambiguous. While watching the clip, in the segment where he's saying "ba" and we're seeing "va" over and over, I tried shutting my eyes and blocking the visual input for every other syllable. And what I heard was... "ba va ba va ba va." As soon as my eyes were open, my visual cortex overrode my auditory cortex, even as my prefrontal cortex was shouting at it, "Hey! You! You're being tricked! Don't believe it!"
You might think that rationalists like myself would be dismayed at this, relying as we do on our brains' ability to distinguish fact from fiction, perception from illusion. My reaction is quite the opposite. Our brains are generally so good at picking up, sorting out, and making sense of the chaotic mishmash of sensory input we get bombarded with that the few instances that they don't work stand out. Optical illusions, and such sensory-clash phenomena as the McGurk effect, want explanation precisely because our brains are amazingly good at detecting, interpreting, and storing information.
And that's probably why we find them so fascinating. It's definitely why I've watched the McGurk effect video clip three times, and each time I try unsuccessfully to get the effect to vanish with a variety of tricks -- deliberately blurring my vision, concentrating on his forehead instead of his mouth, and so on. It's also why I'll be paying closer attention to looking at my friends' faces next time I'm chatting with them in a crowded pub.