Trompe l'oeil

I have a fascination for optical illusions.

Not only are they cool, they often point out some profound information about how we process sensory input.  Take the famous two-and-a-half pronged fork:

The problem here is that we're trying to interpret a two-dimensional drawing as if it were a three-dimensional object, and the two parts of the drawing aren't compatible under that interpretation.  Worse, when you try to force your brain to make sense of it -- following the drawing from the bottom left to the top right, and trying to figure out when the object goes from three prongs to two -- you fail utterly.

Neil deGrasse Tyson used optical illusions as an example of why we should be slow to accept eyewitness testimony.  "We all love optical illusions," he said. "But that's not what they should call them.  They should call them 'brain failures.'  Because that's what they are.  A clever drawing, and your brain can't handle it."

(If you have some time, check out this cool compendium of optical illusions collected by Michael Bach, which is even more awesome because he took the time to explain why each one happens, at least where an explanation is known.)

It's even more disorienting when an illusion occurs because of two senses conflicting.  Which was the subject of a paper out of Caltech, "What You Saw Is What You Will Hear: Two New Illusions With Audiovisual Postdictive Effects," by Noelle R. B. Stiles, Monica Li, Carmel A. Levitan, Yukiyasu Kamitani, and Shinsuke Shimojo.  What they did is an elegant experiment to show two things -- how sound can interfere with visual processing, and how a stimulus can influence our perception of an event, even if the stimulus occurs after the event did!

Sounds like the future affecting the past, doesn't it?  It turns out the answer is both simpler and more humbling; it's another example of a brain failure.

Here's how they did the experiment.

In the first trial, they played a beep three times, 58 milliseconds apart.  The first and third beeps were accompanied by a flash of light.  Most people thought there were three flashes -- a middle one coincident with the second beep.

The second setup was, in a way, opposite to the first.  They showed three flashes of light, on the right, middle, and left of the computer screen.  Only the first and third were accompanied by a beep.  Almost everyone didn't see -- or, more accurately, didn't register -- the middle flash, and thought there were only two lights.

Sorry, I had to.

"The significance of this study is twofold," said study co-author Shinsuke Shimojo.  "First, it generalizes postdiction as a key process in perceptual processing for both a single sense and multiple senses.  Postdiction may sound mysterious, but it is not—one must consider how long it takes the brain to process earlier visual stimuli, during which time subsequent stimuli from a different sense can affect or modulate the first.  The second significance is that these illusions are among the very rare cases where sound affects vision, not vice versa, indicating dynamic aspects of neural processing that occur across space and time.  These new illusions will enable researchers to identify optimal parameters for multisensory integration, which is necessary for both the design of ideal sensory aids and optimal training for low-vision individuals."

All cool stuff, and more information about how the mysterious organ in our skull works.  Of course, this makes me wonder what we imagine we see because our brain anticipates that it will there, or perhaps miss because it anticipates that something out of of place shouldn't be there.  To end with another quote from Tyson: "Our brains are unreliable as signal-processing devices.  We're confident about what we see, hear, and remember, when in fact we should not be."

****************************************