It's a bird, it's a plane... no, it's both

One topic I've come back to over and over again here at Skeptophilia is how flawed our sensory/perceptive apparatus is.  Oh, it works well enough; most of the time, we perceive the external world with sufficient clarity not to walk into walls or get run over by oncoming trains.  But our impression that we experience the world as it is -- that our overall ambient sense of everything around us, what the brilliant neurophysiologist David Eagleman calls our umwelt, is a crystal-clear reflection of the real universe -- simply is false.

All it takes is messing about with optical illusions to convince yourself how easy our brains and sensory organs are to fool.  For example, in the following drawing, which is darker; square A or square B?

They're exactly the same.  Don't believe me?  Here's the same drawing, with a pair of gray lines superimposed on it:

Because your brain decided that B was in the shadow and A wasn't, then it concluded that A had to be intrinsically darker.  What baffles me still about this illusion is that even once you know how the trick works, it's impossible to see it any other way.

As astronomer Neil deGrasse Tyson put it, "Our brains are rife with ways of getting it wrong.  You know optical illusions?  That's not what they should call them.  They should call them brain failures.  Because that's what they are.  A few cleverly drawn lines, and your brain can't handle it."

Well, we just got another neat hole shot in our confidence that what we're experiencing is irrefutable concrete reality with a study that appeared in the journal Psychological Science this week.  What the researchers did was attempt to confound the senses of sight and hearing by showing test subjects a photograph of one object morphing into another -- say, a bird into an airplane.  During the time they studied the photograph, they were exposed to a selection from a list of sounds, two of which were relevant (birdsong and the noise of a jet engine) and a number of which were irrelevant distractors (like a hammer striking a nail).

They were then told to use a sliding scale to estimate where in the transformation of bird-into-airplane the image was (e.g. seventy percent bird, thirty percent airplane).  What the researchers found was that people were strongly biased by what they were hearing; birdsong biased the test subjects to overestimate the birdiness of the photograph, and reverse happened with the sound of a jet engine.  The irrelevant noises didn't effect choice (and thus, when exposed to the irrelevant noises, their visual perceptions of the image were more accurate).

"When sounds are related to pertinent visual features, those visual features are prioritized and processed more quickly compared to when sounds are unrelated to the visual features," said Jamal Williams, of the University of California - San Diego, who led the study, in an interview with Science Daily.  "So, if you heard the sound of a birdsong, anything bird-like is given prioritized access to visual perception.  We found that this prioritization is not purely facilitatory and that your perception of the visual object is actually more bird-like than if you had heard the sound of an airplane flying overhead."

I guess it could be worse; at least hearing birdsong didn't make you see a bird that wasn't there.  But it does once again make me wonder how eyewitness testimony is still considered to carry the most weight in a court of law when experiment after experiment has demonstrated not only how incomplete and easily biased our perceptions are, but how flawed our memories are.

Something to keep in mind next time you are tempted to say "I know it happened that way, I saw it with my own eyes."

****************************************

Sounding off

Ever have the experience of getting into a car, closing the door, and accidentally shutting the seatbelt in the door?

What's interesting about this is that most of the time, we immediately realize it's happened, reopen the door, and pull the belt out.  It's barely even a conscious thought.  The sound is wrong, and that registers instantly.  We recognize when something "sounds off" about noises we're familiar with -- when latches don't seat properly, when the freezer door hasn't completely closed, even things like the difference between a batter's solid hit and a tip during a baseball game.

Turns out, scientists at New York University have just figured out that there's a brain structure that's devoted to that exact phenomenon.

A research team led by neuroscientist David Schneider trained mice to learn to associate a particular sound with pushing a lever for a treat.  After learning the sound, it became as habituated in their brains as our own expectation of what the car door closing is supposed to sound like.  If after that the tone was varied even a little, or the timing between the lever push and the sound was changed, a part of the mouse's brain began to fire rapidly.

The activated part of the brain is a cluster of neurons in the auditory cortex, but I think of it as the "What The Hell Just Happened?" module.

"We listen to the sounds our movements produce to determine whether or not we made a mistake," Schneider said.  "This is most obvious for a musician or when speaking, but our brains are actually doing this all the time, such as when a golfer listens for the sound of her club making contact with the ball.  Our brains are always registering whether a sound matches or deviates from expectations.  In our study, we discovered that the brain is able to make precise predictions about when a sound is supposed to happen and what it should sound like...  Because these were some of the same neurons that would have been active if the sound had actually been played, it was as if the brain was recalling a memory of the sound that it thought it was going to hear."

As a musician, I find myself wondering if this is why I had such a hard time unlearning my tendency to make a face when I hit a wrong note, when I first started performing on stage.  My bandmates said (rightly) that if it's not a real howler, most mistakes will just zoom right past the audience unnoticed -- unless the musician clues them in by wincing.  (My bandmate Kathy also added that if it is a real howler, just play it that way again the next time that bit of the tune comes around, and the audience will think it's a deliberate "blue note" and be really impressed about how avant-garde we are.) 

My band Crooked Sixpence, with whom I played for an awesome ten years -- l. to r., Kathy Selby (fiddle), me (flute), John Wobus (keyboard)

I found it a hard response to quell, though.  My awareness of having hit a wrong note was so instantaneous that it's almost like my ears are connected directly to my facial wince-muscles, bypassing my brain entirely.  I did eventually get better, both in the sense of making fewer mistakes and also responding less when I did hit a clam, but it definitely took a while for the flinch response to calm down.

It's interesting to speculate on why we have this sense, and evidently share it with other mammals.  The obvious explanation is that a spike of awareness about something sounding off could be a good clue to the presence of danger -- the time-honored trope in horror movies of one character saying something doesn't seem quite right.  (That character, however, is usually the first one to get eaten by the monster, so the response may be of dubious evolutionary utility, at least in horror movies.)

I find it endlessly fascinating how our brains have evolved independent little subunits for dealing with contingencies like this.  Our sensory processing systems are incredibly fine-tuned, and they can alert us to changes in our surroundings so quickly it hardly involves conscious thought.

Think about that the next time your car door doesn't close completely.

****************************************

A chat at the pub

When I'm out in a crowded bar, I struggle with something that I think a lot of us do -- trying to isolate the voice of the person I'm talking to from all of the background noise.

I can do it, but it's a struggle.  When I'm tired, or have had one too many pints of beer, I find that my ability to hear what my friend is saying suddenly disappears, as if someone had flipped off a switch.  His voice is swallowed up by a cacophony of random noise in which I literally can't isolate a single word.

Usually my indication that it's time to call it a night.

[Image is in the Public Domain]

It's an interesting question, though, how we manage to do this at all.  Think about it; the person you're listening to is probably closer to you than the other people in the pub, but the others might well be louder.  Add to that the cacophony of glasses clinking and music blaring and whatever else might be going on around you, and the likelihood is that your friend's overall vocal volume is probably about the same as anyone or anything else picked up by your ears.

Yet most of us can isolate that one voice and hear it distinctly, and tune out all of the other voices and ambient noise.  So how do you do this?

Scientists at Columbia University got a glimpse of how our brains might accomplish this amazing task in a set of experiments described in a paper that appeared in the journal Neuron this week.  In "Hierarchical Encoding of Attended Auditory Objects in Multi-talker Speech Perception," by James O’Sullivan, Jose Herrero, Elliot Smith, Catherine Schevon, Guy M. McKhann, Sameer A. Sheth, Ashesh D. Mehta, and Nima Mesgarani, we find out that one part of the brain -- the superior temporal gyrus (STG) -- seems to be capable of boosting the gain of a sound we want to pay attention to, and to do so virtually instantaneously.

The auditory input we receive is a complex combination of acoustic vibrations in the air received all at the same time, so sorting them out is no mean feat.  (Witness how long it's taken to develop good vocal transcription software -- which, even now, is fairly slow and inaccurate.)  Yet your brain can do it flawlessly (well, for most of us, most of the time).  What O'Sullivan et al. found was that once received by the auditory cortex, the neural signals are passed through two regions -- first the Heschl's gyrus (HG), and then the STG.  The HG seems to create a multi-dimensional neural representation of what you're hearing, but doesn't really pick out one set of sounds as being more important than another.  The STG, though, is able to sort through that tapestry of electrical signals and amplify the ones it decides are more important.

"We’ve long known that areas of auditory cortex are arranged in a hierarchy, with increasingly complex decoding occurring at each stage, but we haven’t observed how the voice of a particular speaker is processed along this path," said study lead author James O’Sullivan in a press release.  "To understand this process, we needed to record the neural activity from the brain directly...  We found that that it’s possible to amplify one speaker’s voice or the other by correctly weighting the output signal coming from HG.  Based on our recordings, it’s plausible that the STG region performs that weighting."

The research has a lot of potential applications, not only for computerized vocal recognition, but for guiding the creation of devices to help the hearing impaired.  It's long been an issue that traditional hearing aids amplify everything equally, so a hearing-impaired individual in a noisy environment has to turn up the volume to hear what (s)he wants to listen to, but this can make the ambient background noise deafeningly loud.  If software can be developed that emulates what the STG does, it might create a much more natural-sounding and comfortable experience.

All of which is fascinating, isn't it?  The more we learn about our own brains, the more astonishing they seem.  Abilities we take entirely for granted are being accomplished by incredibly complex arrays and responses in that 1.3-kilogram "meat machine" sitting inside our skulls, often using mechanisms that still amaze me even after thirty-odd years of studying neuroscience.  

And it leaves me wondering what we'll find out about our own nervous systems in the next thirty years.

**************************************

In keeping with Monday's post, this week's Skeptophilia book recommendation is about one of the most enigmatic figures in mathematics; the Indian prodigy Srinivasa Ramanujan.  Ramanujan was remarkable not only for his adeptness in handling numbers, but for his insight; one of his most famous moments was the discovery of "taxicab numbers" (I'll leave you to read the book to find out why they're called that), which are numbers that are expressible as the sum of two cubes, two different ways.

For example, 1,729 is the sum of 1 cubed and 12 cubed; it's also the sum of 9 cubed and 10 cubed.

What's fascinating about Ramanujan is that when he discovered this, it just leapt out at him.  He looked at 1,729 and immediately recognized that it had this odd property.  When he shared it with a friend, he was kind of amazed that the friend didn't jump to the same realization.

"How did you know that?" the friend asked.

Ramanujan shrugged.  "It was obvious."

The Man Who Knew Infinity by Robert Kanigel is the story of Ramanujan, whose life ended from tuberculosis at the young age of 32.  It's a brilliant, intriguing, and deeply perplexing book, looking at the mind of a savant -- someone who is so much better than most of us at a particular subject that it's hard even to conceive.  But Kanigel doesn't just hold up Ramanujan as some kind of odd specimen; he looks at the human side of a man whose phenomenal abilities put him in a class by himself.

[Note: if you purchase this book using the image/link below, part of the proceeds goes to support Skeptophilia!]

Noise alert

The week after I retired I made the mistake of saying to my wife, "I don't know what I'm going to do with all of my free time!"

Two days later we found out we had to have major foundation work done on our house.  I do mean major; erosion and settling on one corner was causing the slab to twist, and if we didn't do something, we were going to have our slab -- and almost certainly our walls -- crack catastrophically.

So yeah.  Me and my big mouth.  It's times like this I have a hard time maintaining my status as Non-Superstitious Guy.

The foundation work required that we more or less gut our formerly-finished basement.  We were already planning on redoing it, just not this completely or this precipitously.  It could be a nice space -- it's got a walk-out (we're built on a hill, which is part of what caused the problem in the first place) and with some messing about it could be a den or even a rental apartment, now that we're empty nesters and it's just me and Carol in this big house.

Me and my son working on demolition.  You can probably see the amazing family resemblance between us.

In any case, this all comes up because of a paper that appeared last week in Nature Communications  about why we perceive some sounds as unpleasant (such as shop vacs, reciprocating saws, dehumidifiers, and air filters -- all of which we had going at once down there).  And it turns out that it's not just the volume (amplitude) of the sound waves.

In "The Rough Sound of Salience Enhances Aversion Through Neural Synchronisation," by Luc H. Arnal, Andreas Kleinschmidt, Laurent Spinelli, Anne-Lise Giraud, and Pierre Mégevand of the University of Geneva, we find that the degree of perceived unpleasantness of a sound has to do with repeated peaks in "fast repetitive modulations" in the sound.  Put simply, there are two kinds of frequency most sounds have: the fundamental frequency of the tone, which we perceive as its pitch; and the rise and fall of overall loudness.  And what the researchers discovered is when that second frequency is between 30 and 150 hertz, we find it really unpleasant.  (One hertz is one vibration per second; so even 30 hertz is fast enough that we're not consciously aware of it as a repetitive noise.)

Apparently sounds in that range cause our neurons to synchronize at that frequency, heightening awareness and making them difficult to ignore.  The researchers suspect that it may be an evolved response because those sorts of noises may signal danger, but that's speculation at this point.

The authors write:

Fast repetitive modulations produce “temporally salient” flickering percepts (e.g. strobe lights, vibrators, and alarm sounds), which efficiently capture attention, generally induce rough and unpleasant sensations, and elicit avoidance.  Despite the high ecological relevance of such flickering stimuli, there is to our knowledge no existing operational definition of temporal salience and only limited experimental work accounting for the intriguing aversive sensation such auditory textures produce and the reactions they trigger.  Here, we introduce and explore the notion of temporal salience and investigate its behavioural and neural underpinnings.  Of note, although salience may not systematically result in aversive percept, we argue that in this specific context, temporal salience—owing to the imperative effect of exogenously saturating perceptual systems in time—constitutes a valid proxy of aversion.  Therefore, we hypothesise that providing fast, but still discretisable and perceptible, temporally salient acoustic cues should enhance neural processing and ensuing aversive sensation.

This discovery led to some surprising connections.  "These sounds solicit the amygdala, hippocampus and insula in particular, all areas related to salience, aversion and pain.  This explains why participants experienced them as being unbearable," said Luc Arnal, who was the paper's lead author.   "This is the first time that sounds between 40 and 80 hertz have been shown to mobilise these neural networks, although the frequencies have been used for a long time in alarm systems...  We now understand at last why the brain can't ignore these sounds.  Something particular happens at these frequencies, and there are also many illnesses that show atypical brain responses to sounds at 40 Hz.  These include Alzheimer's, autism and schizophrenia."

Which is unexpected and startling.  What is happening in the brain at those frequencies -- and how does it connect with overall mental functioning?  Does schizophrenia (for example) involve some sort of "brain noise" that is at a frequency that the sufferer can't ignore?

In any case, it's a fascinating piece of research, and on a more banal level explains why I find that shop vac so damned annoying.  At least we've got the demolition done, so I won't have any more huge messes to clean up.

Unless the universe is listening and causes some catastrophic upheaval in another part of our house.  You never know.  Just because I'm not superstitious doesn't mean I can't jinx myself.

********************************

This week's Skeptophilia book recommendation is by the team of Mark Carwardine and the brilliant author of The Hitchhiker's Guide to the Galaxy, the late Douglas Adams.  Called Last Chance to See, it's about a round-the-world trip the two took to see the last populations of some of the world's most severely endangered animals, including the Rodrigues Fruit Bat, the Mountain Gorilla, the Aye-Aye, and the Komodo Dragon.  It's fascinating, entertaining, and sad, as Adams and Carwardine take an unflinching look at the devastation being wrought on the world's ecosystems by humans.

But it should be required reading for anyone interested in ecology, the environment, and the animal kingdom. Lucid, often funny, always eye-opening, Last Chance to See will give you a lens into the plight of some of the world's rarest species -- before they're gone forever.

[Note: if you purchase this book using the image/link below, part of the proceeds goes to support Skeptophilia!]

Giving weight to illusion

The idea that our sensory processing apparatus and our brains are unreliable has been something I've come back to again and again here at Skeptophilia.  "I saw it with my own eyes" is simply not enough evidence by which to make any kind of sound scientific judgment.

Not only are we likely to get things wrong just because the equipment is faulty, our prior ideas can predispose us to get things wrong in a particular way.  Think of it as a sort of built-in confirmation bias; our brains are set up in such a fashion that when we've already decided what's going to happen, it's much more likely that's what we'll perceive.

This latter problem was demonstrated in an elegant, if disturbing, fashion in a paper released last week in Science called "Pavlovian Conditioning–Induced Hallucinations Result From Overweighting of Perceptual Priors," by Albert R. Powers, Christoph Mathys, and Philip R. Corlett, of the Yale School of Medicine, the International School for Advanced Studies (Trieste, Italy), and the University of Zurich, respectively.  Their research springboarded from previous studies wherein individuals who had been trained to associate a tone with an image were more likely to continue "hearing" the tone when shown the image with no accompanying tone than were members of a control group.

[image courtesy of the Wikimedia Commons]

What Corlett's team did was to divide participants into four groups: normal, healthy individuals; self-described psychics; individuals with psychosis who did not report hearing voices; and individuals with schizophrenia who reported hearing voices.  The researchers trained all test subjects to associate a checkerboard image with a one second long, one kilohertz tone.  They not only recorded data on which participants continued to "hear" an illusory tone when shown the checkerboard in silence, they also used a protocol (how hard they pushed the button when they heard the tone, whether real or imagined) to gauge their confidence in what they were experiencing, and they looked at neuronal activity in the brain using an fMRI machine.

The results were intriguing, to say the least.  Both the schizophrenics and the self-described psychics were five times as likely to report hearing a tone when none existed than either the control group of healthy individuals or the psychotic individuals who did not hear voices.  Not only that, the schizophrenics and the psychics were 28% more confident in their perceptions when they did hear a tone that wasn't there than were the other two groups when they made a similar mistake.

Further, the schizophrenics and psychics showed abnormal neuronal activity in two regions of the brain; the parts of the cerebrum involved in creating our internal representation of reality showed strikingly different firing patterns, and the cerebellum -- the part of the brain involved in planning and coordinating our motor responses to stimuli -- showed much lower than normal neuronal activity.

"The findings confirm that, when it comes to how we perceive the world, our ideas and beliefs can easily overpower our senses," said Albert Powers, one of the paper's authors.  Which is about as succinct a cautionary statement about trusting our judgments as I can imagine.

While the researchers specifically tested the likelihood of experiencing auditory hallucinations, I find myself wondering if this study might not have wider applications.  How do our prior perceptions bias us in general?  I know I have frequently been baffled, especially in these fractious times, how two people can see the same event and come to strikingly opposite conclusions about it.  At times, I have found myself asking, "Are we even talking about the same thing, here?"  But if our preconceived notions about the world can bias us strongly enough to hear sounds that aren't there, why should any other perception be immune to the same effect?

This possibility drives me to a disturbing conclusion.  How do you convince people that what they're perceiving is not real, if that conclusion is contrary to what their senses and their brains are telling them?

I think the key, here, is always to keep focused on the statement, "... but I might be wrong."  A lot of our faulty judgments are caused not only by our coming to the wrong conclusion, but our stubborn certainty that we are, in fact, right.  A willingness to revise our beliefs -- failing that, at least to consider the possibility that our beliefs are incorrect -- is absolutely critical.

Otherwise, we're at the mercy of sensory apparatus that are easily fooled, and a brain that bases what it perceives as much on what it already thought to be true as on the actual data it's presented with.

Which seems to me to be awfully shaky ground.