The visual time machine

I don't know if you've ever considered what I'm about to describe; I know I had to have it pointed out to me.

Let's say you're walking down a long hallway, where there are other people, doorways, windows, pieces of art on the wall -- lots of stuff to look at.  As you walk, you move your head and your eyes to check out the surroundings, and also so you don't run into anyone.  Now, let's say that at the same time, you have a miniature videocamera attached to your forehead, so that it's recording the scene using the exact same perspective and movements as you.

Now, consider the difference between what you saw while walking, and what you'd see if you looked at the video of the same walk down the hall.

The recorded video would have incorporated every jolt from your feet striking the ground, every jerky movement of your head.  The visual field would bounce all over the place.  You know that show, Finding Bigfoot?  The one that's been going on for ten years, wherein despite the name, they have found exactly zero Bigfoots?  They're always showing video footage taken with hand-held video recorders, as the crew of the show run about in the woods excitedly not finding any Bigfoots, and those videos look like someone strapped the camera to a kangaroo on speed.  The movie The Blair Witch Project was filmed to look like it had been taken with a hand-held recorder, and they succeeded -- to the point that some people find it unwatchable, and end up feeling queasy or headachy from the scene being jostled around continuously.

The question is, why don't we see exactly the same thing?  Unless we're rattled way harder than usual -- like riding too fast in a car over a rutted and potholed road -- we have no visual sense of the fact that just like the video recorder, the scene we're looking at is jittering around continuously.

One possible explanation that has been given is microsaccades -- continuous minuscule back-and-forth jerks of the eyes that everyone has (but are so fast that you need a slowed-down video recording to see them).  It's possible that the brain uses these quick-but-tiny shifts in the visual field to smooth out the input and erase the sense that what you're seeing is bouncing around.

As an aside, there's another curious feature of microsaccades; they can be used to detect when someone's not paying attention.  I read about funny bit of research a few years ago, but unfortunately I can't find a link referencing it -- if anyone knows the source, please post a link in the comments.  The gist was that they took volunteers and attached head-mounted cameras to them, but the cameras weren't looking at the surroundings -- the lens was pointed backwards at the volunteers' eyes.  The instructions were that the volunteers were supposed to chat with the bartender, and not look around at anything or anyone else.  

Then, during the middle of the experiment, an attractive person of the volunteer's preferred gender walked in and sat down a few barstools over.  

The volunteers all did what they were told -- none of them turned and looked toward the eye candy parked only a few feet away.  But their microsaccades reacted big time.  The little jitters in the eye suddenly all were aimed in the same direction -- toward the hot-looking person near them.  It's like the brain is saying, "No, I can't look, I told the researchers I wouldn't," while the microsaccades are saying "LOOK AT THAT SEXY PERSON!  LOOK!  I KNOW YOU WANT TO!"

[Image licensed under the Creative Commons Laurinemily at English Wikipedia, Hazel-green eye 2, CC BY-SA 2.5]

In any case, some research came out last week, by Mauro Manassi (University of Aberdeen) and David Whitney (University of California - Berkeley), that suggests that there's another smoothing effect at work in addition to microsaccades.  What the researchers found was that there is a feature of our brain that does the same thing in time that the microsaccades do in space; they blur out little jolts by averaging the input.  In this case, your brain coalesces the images we've received during the last fifteen seconds, so any small vibrations get blended into a sense of a smooth, continuous visual field.

What the researchers did was to show volunteers a thirty-second video clip of a face that was slowly morphing in such a way that it appeared to change age.  The volunteers were then asked what age the individual was at the end of the clip.  Across the board, they underestimated the age of the face. On the other hand, given a still shot of the face as it was at the end resulted in fairly accurate assessment of the person's age.  But when watching the video, the answer they gave was consistently the apparent age of the individual not at the end, but the average over the previous fifteen seconds of the video.

The authors write:

In other words, the brain is like a time machine which keeps sending us back in time.  It’s like an app that consolidates our visual input every 15 seconds into one impression so that we can handle everyday life.  If our brains were always updating in real time, the world would feel like a chaotic place with constant fluctuations in light, shadow and movement.  We would feel like we were hallucinating all the time...  This idea... of mechanisms within the brain that continuously bias our visual perception towards our past visual experience is known as continuity fields.  Our visual system sometimes sacrifices accuracy for the sake of a smooth visual experience of the world around us.  This can explain why, for example, when watching a film we don’t notice subtle changes that occur over time, such as the difference between actors and their stunt doubles.

So once again, our sensory-perceptive systems (1) are way more complex than we thought, and (2) are recording the perceptions we have in such a way that they're not necessarily completely accurate, but the most useful.  "I saw it with my own eyes!" really doesn't mean very much.  As my neuroscience professor told us many years ago, "Your senses don't have to reflect reality; they just have to work well enough that you can find food, avoid being killed, and find a mate."

And if that means losing some visual accuracy in favor of the world not looking like hand-held video footage from Finding Bigfoot, I'm okay with that.

*******************************

It's obvious to regular readers of Skeptophilia that I'm fascinated with geology and paleontology.  That's why this week's book-of-the-week is brand new: Thomas Halliday's Otherlands: A Journey Through Extinct Worlds.

Halliday takes us to sixteen different bygone worlds -- each one represented by a fossil site, from our ancestral australopithecenes in what is now Tanzania to the Precambrian Ediacaran seas, filled with animals that are nothing short of bizarre.  (One, in fact, is so weird-looking it was christened Hallucigenia.)  Halliday doesn't just tell us about the fossils, though; he recreates in words what the place would have looked like back when those animals and plants were alive, giving a rich perspective on just how much the Earth has changed over its history -- and how fragile the web of life is.

It's a beautiful and eye-opening book -- if you love thinking about prehistory, you need a copy of Otherlands.

[Note: if you purchase this book using the image/link below, part of the proceeds goes to support Skeptophilia!]

In the blink of an eye

One of the things I love about science is how it provides answers to questions that are so ordinary that few of us appreciate how strange they are.

I remember how surprised I was when I first heard a question about our vision that had honestly never occurred to me.  You know how images jump around when you're filming with a hand-held videocamera?  Even steady-handed people make videos that are seriously nausea-inducing, and when the idea is to make it look like it's filmed by amateurs -- such as in the movie The Blair Witch Project -- the result looks like it was produced by strapping a camera to the head of a kangaroo on crack.

What's a little puzzling is why the world doesn't appear to jump around like that all the time.  I mean, think about it; if you walk down the hall holding a videocamera on your shoulder, and watch the video and compare it to the way the hall looked while you were walking, you'll see the image bouncing all over the place on the video, but won't have experienced that with your eyes.  Why is that?

The answer certainly isn't obvious.  One guess scientists have is that we stabilize the images we see, and compensate for small movements of our head, by using microsaccades -- tiny, involuntary, constant jitters of the eyes.  The thought is that those little back-and-forth movements allow your brain to smooth out the image, keeping us from seeing the world as jumping around every time we move.

Another question about visual perception that I had never thought about was the subject of some recent research out of New York University and the University Medical Center of Göttingen that was just published last week in Current Biology.  Why don't you have the perception of the world going dark for a moment when you blink?  After all, most of us blink about once every five seconds, and we don't have the sense of a strobe effect.  In fact, most of us are unaware of any change in perception whatsoever.

[Image licensed under the Creative Commons Mcorrens, Iris of the Human Eye, CC BY-SA 3.0]

By studying patients who had lesions in the cerebrum, and comparing them to patients with intact brains, the scientists were not only able to answer this question, but to pinpoint exactly where this phenomenon happens -- the dorsomedial prefrontal cortex, a part of the brain immediately behind the forehead.  What they found was that individuals with an intact dmPFC store a perceptual memory of what they've just seen, and use that to form the perception they're currently seeing, so the time during which there's no light falling on the retina -- when you blink -- doesn't even register.  On the other hand, a patient with a lesion in the dmPFC lost that ability, and didn't store immediate perceptual memories.  The result?  Every time she blinked, it was like a shutter closed on the world.

"We were able to show that the prefrontal cortex plays an important role in perception and in context-dependent behavior," said neuroscientist Caspar Schwiedrzik, who was lead author of the study.  "Our research shows that the medial prefrontal cortex calibrates current visual information with previously obtained information and thus enables us to perceive the world with more stability, even when we briefly close our eyes to blink...  This is not only true for blinking but also for higher cognitive functions.  Even when we see a facial expression, this information influences the perception of the expression on the next face that we look at."

All of which highlights that all of our perceptual and integrative processes are way more sophisticated than they seem at first.  It also indicates something that's a little scary; that what we're perceiving is partly what's really out there, and partly what our brain is telling us it thinks is out there.  Which is right more often than not, of course.  If that weren't true, natural selection would have finished us off a long time ago.  But that fraction of the times that it's wrong, it can create some seriously weird sensations -- or make us question things that we'd always taken for granted.

*****************************

This week's recommendation is a classic.

When I was a junior in college, I took a class called Seminar, which had a new focus/topic each semester.  That semester's course was a survey of the Book Gödel, Escher, Bach: An Eternal Golden Braid by Douglas Hofstadter.  Hofstadter does a masterful job of tying together three disparate realms -- number theory, the art of M. C. Escher, and the contrapuntal music of J. S. Bach.

It makes for a fascinating journey.  I'll warn you that the sections in the last third of the book that are about number theory and the work of mathematician Kurt Gödel get to be some rough going, and despite my pretty solid background in math, I found them a struggle to understand in places.  But the difficulties are well worth it.  Pick up a copy of what my classmates and I came to refer to lovingly as GEB, and fasten your seatbelt for a hell of a ride.

[If you purchase the book from Amazon using the image/link below, part of the proceeds goes to supporting Skeptophilia!]