In the blink of an eye

One of the things I love about science is how it provides answers to questions that are so ordinary that few of us appreciate how strange they are.

I remember how surprised I was when I first heard a question about our vision that had honestly never occurred to me.  You know how images jump around when you're filming with a hand-held videocamera?  Even steady-handed people make videos that are seriously nausea-inducing, and when the idea is to make it look like it's filmed by amateurs -- such as in the movie The Blair Witch Project -- the result looks like it was produced by strapping a camera to the head of a kangaroo on crack.

What's a little puzzling is why the world doesn't appear to jump around like that all the time.  I mean, think about it; if you walk down the hall holding a videocamera on your shoulder, and watch the video and compare it to the way the hall looked while you were walking, you'll see the image bouncing all over the place on the video, but won't have experienced that with your eyes.  Why is that?

The answer certainly isn't obvious.  One guess scientists have is that we stabilize the images we see, and compensate for small movements of our head, by using microsaccades -- tiny, involuntary, constant jitters of the eyes.  The thought is that those little back-and-forth movements allow your brain to smooth out the image, keeping us from seeing the world as jumping around every time we move.

Another question about visual perception that I had never thought about was the subject of some research out of New York University and the University Medical Center of Göttingen that was published in the journal Current Biology.  Why don't you have the perception of the world going dark for a moment when you blink?  After all, most of us blink about once every five seconds, and we don't have the sense of a strobe effect.  In fact, most of us are unaware of any change in perception whatsoever.

[Image licensed under the Creative Commons Mcorrens, Iris of the Human Eye, CC BY-SA 3.0]

By studying patients who had lesions in the cerebrum, and comparing them to patients with intact brains, the scientists were not only able to answer this question, but to pinpoint exactly where this phenomenon happens -- the dorsomedial prefrontal cortex, a part of the brain immediately behind the forehead.  What they found was that individuals with an intact dmPFC store a perceptual memory of what they've just seen, and use that to form the perception they're currently seeing, so the time during which there's no light falling on the retina -- when you blink -- doesn't even register.  On the other hand, a patient with a lesion in the dmPFC lost that ability, and didn't store immediate perceptual memories.  The result?  Every time she blinked, it was like a shutter closed on the world.

"We were able to show that the prefrontal cortex plays an important role in perception and in context-dependent behavior," said neuroscientist Caspar Schwiedrzik, who was lead author of the study.  "Our research shows that the medial prefrontal cortex calibrates current visual information with previously obtained information and thus enables us to perceive the world with more stability, even when we briefly close our eyes to blink...  This is not only true for blinking but also for higher cognitive functions.  Even when we see a facial expression, this information influences the perception of the expression on the next face that we look at."

All of which highlights that all of our perceptual and integrative processes are way more sophisticated than they seem at first.  It also indicates something that's a little scary; that what we're perceiving is partly what's really out there, and partly what our brain is telling us it thinks is out there.  Which is right more often than not, of course.  If that weren't true, natural selection would have finished us off a long time ago.  But that fraction of the times that it's wrong, it can create some seriously weird sensations -- or make us question things that we'd always taken for granted.

****************************************

The visual time machine

I don't know if you've ever considered what I'm about to describe; I know I had to have it pointed out to me.

Let's say you're walking down a long hallway, where there are other people, doorways, windows, pieces of art on the wall -- lots of stuff to look at.  As you walk, you move your head and your eyes to check out the surroundings, and also so you don't run into anyone.  Now, let's say that at the same time, you have a miniature videocamera attached to your forehead, so that it's recording the scene using the exact same perspective and movements as you.

Now, consider the difference between what you saw while walking, and what you'd see if you looked at the video of the same walk down the hall.

The recorded video would have incorporated every jolt from your feet striking the ground, every jerky movement of your head.  The visual field would bounce all over the place.  You know that show, Finding Bigfoot?  The one that's been going on for ten years, wherein despite the name, they have found exactly zero Bigfoots?  They're always showing video footage taken with hand-held video recorders, as the crew of the show run about in the woods excitedly not finding any Bigfoots, and those videos look like someone strapped the camera to a kangaroo on speed.  The movie The Blair Witch Project was filmed to look like it had been taken with a hand-held recorder, and they succeeded -- to the point that some people find it unwatchable, and end up feeling queasy or headachy from the scene being jostled around continuously.

The question is, why don't we see exactly the same thing?  Unless we're rattled way harder than usual -- like riding too fast in a car over a rutted and potholed road -- we have no visual sense of the fact that just like the video recorder, the scene we're looking at is jittering around continuously.

One possible explanation that has been given is microsaccades -- continuous minuscule back-and-forth jerks of the eyes that everyone has (but are so fast that you need a slowed-down video recording to see them).  It's possible that the brain uses these quick-but-tiny shifts in the visual field to smooth out the input and erase the sense that what you're seeing is bouncing around.

As an aside, there's another curious feature of microsaccades; they can be used to detect when someone's not paying attention.  I read about funny bit of research a few years ago, but unfortunately I can't find a link referencing it -- if anyone knows the source, please post a link in the comments.  The gist was that they took volunteers and attached head-mounted cameras to them, but the cameras weren't looking at the surroundings -- the lens was pointed backwards at the volunteers' eyes.  The instructions were that the volunteers were supposed to chat with the bartender, and not look around at anything or anyone else.  

Then, during the middle of the experiment, an attractive person of the volunteer's preferred gender walked in and sat down a few barstools over.  

The volunteers all did what they were told -- none of them turned and looked toward the eye candy parked only a few feet away.  But their microsaccades reacted big time.  The little jitters in the eye suddenly all were aimed in the same direction -- toward the hot-looking person near them.  It's like the brain is saying, "No, I can't look, I told the researchers I wouldn't," while the microsaccades are saying "LOOK AT THAT SEXY PERSON!  LOOK!  I KNOW YOU WANT TO!"

[Image licensed under the Creative Commons Laurinemily at English Wikipedia, Hazel-green eye 2, CC BY-SA 2.5]

In any case, some research came out last week, by Mauro Manassi (University of Aberdeen) and David Whitney (University of California - Berkeley), that suggests that there's another smoothing effect at work in addition to microsaccades.  What the researchers found was that there is a feature of our brain that does the same thing in time that the microsaccades do in space; they blur out little jolts by averaging the input.  In this case, your brain coalesces the images we've received during the last fifteen seconds, so any small vibrations get blended into a sense of a smooth, continuous visual field.

What the researchers did was to show volunteers a thirty-second video clip of a face that was slowly morphing in such a way that it appeared to change age.  The volunteers were then asked what age the individual was at the end of the clip.  Across the board, they underestimated the age of the face. On the other hand, given a still shot of the face as it was at the end resulted in fairly accurate assessment of the person's age.  But when watching the video, the answer they gave was consistently the apparent age of the individual not at the end, but the average over the previous fifteen seconds of the video.

The authors write:

In other words, the brain is like a time machine which keeps sending us back in time.  It’s like an app that consolidates our visual input every 15 seconds into one impression so that we can handle everyday life.  If our brains were always updating in real time, the world would feel like a chaotic place with constant fluctuations in light, shadow and movement.  We would feel like we were hallucinating all the time...  This idea... of mechanisms within the brain that continuously bias our visual perception towards our past visual experience is known as continuity fields.  Our visual system sometimes sacrifices accuracy for the sake of a smooth visual experience of the world around us.  This can explain why, for example, when watching a film we don’t notice subtle changes that occur over time, such as the difference between actors and their stunt doubles.

So once again, our sensory-perceptive systems (1) are way more complex than we thought, and (2) are recording the perceptions we have in such a way that they're not necessarily completely accurate, but the most useful.  "I saw it with my own eyes!" really doesn't mean very much.  As my neuroscience professor told us many years ago, "Your senses don't have to reflect reality; they just have to work well enough that you can find food, avoid being killed, and find a mate."

And if that means losing some visual accuracy in favor of the world not looking like hand-held video footage from Finding Bigfoot, I'm okay with that.

*******************************

It's obvious to regular readers of Skeptophilia that I'm fascinated with geology and paleontology.  That's why this week's book-of-the-week is brand new: Thomas Halliday's Otherlands: A Journey Through Extinct Worlds.

Halliday takes us to sixteen different bygone worlds -- each one represented by a fossil site, from our ancestral australopithecenes in what is now Tanzania to the Precambrian Ediacaran seas, filled with animals that are nothing short of bizarre.  (One, in fact, is so weird-looking it was christened Hallucigenia.)  Halliday doesn't just tell us about the fossils, though; he recreates in words what the place would have looked like back when those animals and plants were alive, giving a rich perspective on just how much the Earth has changed over its history -- and how fragile the web of life is.

It's a beautiful and eye-opening book -- if you love thinking about prehistory, you need a copy of Otherlands.

[Note: if you purchase this book using the image/link below, part of the proceeds goes to support Skeptophilia!]

Windows of the soul

I find the science of personality fascinating, not least because we're only in the earliest stages of any kind of comprehensive understanding.  What makes human beings act the way they do is certainly some kind of mix between genetics (nature) and environment (nurture), but in what proportion, and to what extent the two influence each other, are still mysteries.

The situation isn't helped much by "type tests," the best-known of which is the Myers-Briggs Type Indicator.  Research has conclusively demonstrated that the MBTI is unreliable -- the same person at different times can end up with entirely different results -- meaning that as a psychiatric tool, it's fairly useless.

Of course, maybe I'm only saying that because I'm an INTP.

So it's not surprising that psychologists and neuroscientists are eager to find more reliable ways to measure personality, especially with respect to the "Big Five" traits (neuroticism, extraversion, agreeableness, conscientiousness, and openness to experience).  And they may have gotten an unexpected leg up with some recent research indicating that where you fall along those five spectra can be given away by your involuntary eye movements.

In their paper, "Eye Movements During Everyday Behavior Predict Personality Traits," which appeared earlier this year in Frontiers in Human Neuroscience, researchers Sabrina Hoppe (of the University of Stuttgart), Tobias Loetscher (of the University of South Australia), Stephanie A. Morey (of Flinders University), and Andreas Bulling (of the Max Planck Institute) found something fascinating; when a headset kept track of the eye movements of forty-two test subjects as they went about their daily business, an artificial-intelligence program could determine where they ranked on various personality scales with astonishing accuracy.

The authors write:

One key contribution of our work is to demonstrate, for the first time, that an individual's level of neuroticism, extraversion, agreeableness, conscientiousness, and perceptual curiosity can be predicted only from eye movements recorded during an everyday task.  This finding is important for bridging between tightly controlled laboratory studies and the study of natural eye movements in unconstrained real-world environments...  The proposed machine learning approach was particularly successful in predicting levels of agreeableness, conscientiousness, extraversion, and perceptual curiosity.  It therefore corroborates previous laboratory-based studies that have shown a link between personality traits and eye movement characteristics. 

[O]ur work... shed[s] additional light on the close link between personality traits and an individual's eye movements.  Thanks to the machine learning approach, we could automatically analyze a large set of eye movement characteristics and rank them by their importance for personality trait prediction.  Going beyond characteristics investigated in earlier works, this approach also allowed us to identify new links between previously under-investigated eye movement characteristics and personality traits.  This was possible because, unlike classical analysis approaches, the proposed machine learning method does not rely on a priori hypotheses regarding the importance of individual eye movement characteristics... 

[I]mproved theoretical understanding will assist the emerging interdisciplinary research field of social signal processing, toward development of systems that can recognize and interpret human social signals. 

Such knowledge of human non-verbal behavior might also be transferred to socially interactive robots, designed to exhibit human-like behavior.  These systems might ultimately interact with humans in a more natural and socially acceptable way, thereby becoming more efficient and flexible.

Which is absolutely fascinating, and of course raises the question of why my neuroticism and introversion would affect the tiny, involuntary movements of my eyes, but answering that was beyond the scope of this study.  Of course, we already knew that the small, back-and-forth movements of the eyes called microsaccades can give you information about what a person is thinking.  A highly amusing experiment a few years ago monitored test subjects in a crowded pub with head-mounted cameras.  The only instructions were that the person should try to keep focused on the person next to them, with whom they were having a discussion.

Well, a few minutes in, the researchers sent in a gorgeous, scantily-clad individual of the test subject's preferred gender to walk by, and even when the subject made a heroic effort not to look, the microsaccades gave him/her away.  While they were focused on the person they were talking to, their microsaccades were going, "Oh dear god that person is drop-dead sexy look that way look that way LOOK THAT WAY."

So the Hoppe et al. study is a fascinating refinement of what our eyes give away about us.  It does make me wonder, however, how this could be used to reveal information the individual would prefer not to reveal.  If an AI program can successfully determine a person's personality from nothing more than eye movements, it's another potential blow to privacy -- however astonishing the results may be to people who, like me, are fascinated with the intricacies of the human brain.

***************************

In writing Apocalyptic Planet, science writer Craig Childs visited some of the Earth's most inhospitable places.  The Greenland Ice Cap.  A new lava flow in Hawaii.  Uncharted class-5 rapids in the Salween River of Tibet.  The westernmost tip of Alaska.  The lifeless "dune seas" of northern Mexico.  The salt pans in the Atacama Desert of Chile, where it hasn't rained in recorded history.

In each place, he not only uses lush, lyrical prose to describe his surroundings, but uses his experiences to reflect upon the history of the Earth.  How conditions like these -- glaciations, extreme drought, massive volcanic eruptions, meteorite collisions, catastrophic floods -- have triggered mass extinctions, reworking not only the physical face of the planet but the living things that dwell on it.  It's a disturbing read at times, not least because Childs's gift for vivid writing makes you feel like you're there, suffering what he suffered to research the book, but because we are almost certainly looking at the future.  His main tenet is that such cataclysms have happened many times before, and will happen again.

It's only a matter of time.

[If you purchase the book from Amazon using the image/link below, part of the proceeds goes to supporting Skeptophilia!]

In the blink of an eye

One of the things I love about science is how it provides answers to questions that are so ordinary that few of us appreciate how strange they are.

I remember how surprised I was when I first heard a question about our vision that had honestly never occurred to me.  You know how images jump around when you're filming with a hand-held videocamera?  Even steady-handed people make videos that are seriously nausea-inducing, and when the idea is to make it look like it's filmed by amateurs -- such as in the movie The Blair Witch Project -- the result looks like it was produced by strapping a camera to the head of a kangaroo on crack.

What's a little puzzling is why the world doesn't appear to jump around like that all the time.  I mean, think about it; if you walk down the hall holding a videocamera on your shoulder, and watch the video and compare it to the way the hall looked while you were walking, you'll see the image bouncing all over the place on the video, but won't have experienced that with your eyes.  Why is that?

The answer certainly isn't obvious.  One guess scientists have is that we stabilize the images we see, and compensate for small movements of our head, by using microsaccades -- tiny, involuntary, constant jitters of the eyes.  The thought is that those little back-and-forth movements allow your brain to smooth out the image, keeping us from seeing the world as jumping around every time we move.

Another question about visual perception that I had never thought about was the subject of some recent research out of New York University and the University Medical Center of Göttingen that was just published last week in Current Biology.  Why don't you have the perception of the world going dark for a moment when you blink?  After all, most of us blink about once every five seconds, and we don't have the sense of a strobe effect.  In fact, most of us are unaware of any change in perception whatsoever.

[Image licensed under the Creative Commons Mcorrens, Iris of the Human Eye, CC BY-SA 3.0]

By studying patients who had lesions in the cerebrum, and comparing them to patients with intact brains, the scientists were not only able to answer this question, but to pinpoint exactly where this phenomenon happens -- the dorsomedial prefrontal cortex, a part of the brain immediately behind the forehead.  What they found was that individuals with an intact dmPFC store a perceptual memory of what they've just seen, and use that to form the perception they're currently seeing, so the time during which there's no light falling on the retina -- when you blink -- doesn't even register.  On the other hand, a patient with a lesion in the dmPFC lost that ability, and didn't store immediate perceptual memories.  The result?  Every time she blinked, it was like a shutter closed on the world.

"We were able to show that the prefrontal cortex plays an important role in perception and in context-dependent behavior," said neuroscientist Caspar Schwiedrzik, who was lead author of the study.  "Our research shows that the medial prefrontal cortex calibrates current visual information with previously obtained information and thus enables us to perceive the world with more stability, even when we briefly close our eyes to blink...  This is not only true for blinking but also for higher cognitive functions.  Even when we see a facial expression, this information influences the perception of the expression on the next face that we look at."

All of which highlights that all of our perceptual and integrative processes are way more sophisticated than they seem at first.  It also indicates something that's a little scary; that what we're perceiving is partly what's really out there, and partly what our brain is telling us it thinks is out there.  Which is right more often than not, of course.  If that weren't true, natural selection would have finished us off a long time ago.  But that fraction of the times that it's wrong, it can create some seriously weird sensations -- or make us question things that we'd always taken for granted.

*****************************

This week's recommendation is a classic.

When I was a junior in college, I took a class called Seminar, which had a new focus/topic each semester.  That semester's course was a survey of the Book Gödel, Escher, Bach: An Eternal Golden Braid by Douglas Hofstadter.  Hofstadter does a masterful job of tying together three disparate realms -- number theory, the art of M. C. Escher, and the contrapuntal music of J. S. Bach.

It makes for a fascinating journey.  I'll warn you that the sections in the last third of the book that are about number theory and the work of mathematician Kurt Gödel get to be some rough going, and despite my pretty solid background in math, I found them a struggle to understand in places.  But the difficulties are well worth it.  Pick up a copy of what my classmates and I came to refer to lovingly as GEB, and fasten your seatbelt for a hell of a ride.

[If you purchase the book from Amazon using the image/link below, part of the proceeds goes to supporting Skeptophilia!]

The voice of truth

Having been a blogger for seven years -- the fact of which I find a little astonishing -- I am well aware of the difficulty of coming across with the right emotional tone in writing.

Especially given the fraught nature of many of the topics I address, I'm sure that my words sometimes elicit strong emotions.  (Cf. the post I did a couple of days ago on hate mail.)  In some cases, the ire is probably justified; perhaps I stepped on your toes about some dearly-held belief of yours, which is bound to raise people's hackles.

On the other hand, I am often afraid that what I'm saying will be misconstrued, not because of the words themselves, but because of the inherent deficiency of the written word in representing the writer's motivations and emotional content accurately.  It's why emails so often generate misunderstandings; it's also why people often feel freer to be nasty online than face-to-face.  When we lack the visual cues of people's facial expressions and body language, we not only are more prone to misinterpreting what people's words mean, we sometimes feel less inhibited about saying things we'd never dream of saying if the person was standing right in front of us.

But apparently you don't even need to see the person's face to diminish this tendency.  A recent experiment by Juliana Schroeder of the Haas School of Business at UC-Berkeley, and Michael Kardas and Nicholas Epley of the University of Chicago, has shown that all you need to add is a voice.

In "The Humanizing Voice: Speech Reveals, and Text Conceals, a More Thoughtful Mind in the Midst of Disagreement," that appeared at PubMed a couple of weeks ago, the researchers showed that you're more receptive to viewpoints you disagree with, and less judgmental about the people stating them, if you hear those statements spoken rather than simply reading them in text form.

The authors write:

A person's speech communicates his or her thoughts and feelings.  We predicted that beyond conveying the contents of a person's mind, a person's speech also conveys mental capacity, such that hearing a person explain his or her beliefs makes the person seem more mentally capable-and therefore seem to possess more uniquely human mental traits-than reading the same content.  We expected this effect to emerge when people are perceived as relatively mindless, such as when they disagree with the evaluator's own beliefs.  Three experiments involving polarizing attitudinal issues and political opinions supported these hypotheses.  A fourth experiment identified paralinguistic cues in the human voice that convey basic mental capacities.  These results suggest that the medium through which people communicate may systematically influence the impressions they form of each other.  The tendency to denigrate the minds of the opposition may be tempered by giving them, quite literally, a voice.

Which is fascinating, if a little unsurprising.  After all, we are social primates, and we evolved in a context of living in groups in which communication was always face-to-face.  We're exquisitely sensitive to subtleties of expression (nicknamed microexpressions), often on a completely subconscious level.  Experiments have shown that we use minor cues such as pupil dilation size to make judgments about attractiveness, and the imperceptibly tiny back-and-forth movements of the eye called microsaccades can give you information about emotional state and what you're paying attention to (even if you're trying to hide that fact).

[image courtesy of photographer Lydia Icerko and the Wikimedia Commons]

And as far as voices go, small differences of inflection can provide huge cues as to what the speaker's intent was.  Consider the following phrase: "She gave the money to him."  Now speak the words aloud, but the first time put the emphasis on the word "she," then on "gave," then on "money," then on "him."

Each one has a different implication, doesn't it?

So if we're reading what someone's written, we're losing access to the cues that might tell us such important information as what the person's motivations and emotional state was when they wrote it.  It's no wonder this leads to frequent misjudgments.  We're trying to parse a person's words based on incomplete data.

This should make us a little more cautious about deciding that we know what people mean when we read an email -- or a blog post.  Clear communication is one thing, and (being a writer) I'm all for that.  But no matter how clear we are, we're never going to be able to communicate emotional depth via the written word as well as we can in person.

So if you think your favorite blogger is being an asshole sometimes, you might want to give him the benefit of the doubt.