Remembrance of things past

Almost all of us implicitly trust our own memories.

Experiment after experiment, however, has shown that this trust is misplaced.  Even if you leave out people with obvious memory deficits -- victims of dementia, for example -- the rest of us give far too much credence to our brain's version of the past.  In truth, what we remember is a conglomerate of what actually did happen, what we were told happened, what we imagine happened based upon the emotions associated with the event, and pure (if inadvertent) fabrication.  And the scariest part is that absent hard evidence (a video, for example), there's no way to tell which parts are what.

It all feels true.

If you don't believe this, consider what happened to cognitive researcher Elizabeth Loftus, of the University of California - Irvine, whose experiments establishing the unreliability of memory are described in neuroscientist David Eagleman's wonderful book The Brain: The Story of You:

We're all susceptible to this memory manipulation -- even Loftus herself.  As it turned out, when Elizabeth was a child, her mother had drowned in a swimming pool.  Years later, a conversation with a relative brought out an extraordinary fact: that Elizabeth had been the one to find her mother's body in the pool.  That news came as a shock to her; she hadn't known that, and in fact didn't believe it.  But, she describes, "I went home from that birthday and I started to think: maybe I did.  I started to think about other things that I did remember -- like when the firemen came, they gave me oxygen.  Maybe I needed the oxygen because I was so upset I found the body?"  Soon, she could visualize her mother in the swimming pool.

But then, her relative called to say he had made a mistake.  It wasn't the young Elizabeth after all who had found the body.  It had been Elizabeth's aunt.  And that's how Loftus had the experience what it was like to possess her own false memory, richly detailed and deeply felt.

So it's not like false memories seem vague or tentative.  They're so vivid you can't tell them from real ones.

Which brings us to the strange story of an arcade video game called "Polybius."

In the early 1980s, a rumor began to circulate that there was an arcade game that combined some very frightening effects.  Its visuals and sounds were dark, surreal, and suggestive.  Children who played the game sometimes had seizures or hallucinations, and afterwards experienced periods of amnesia and night terrors.  Worse, there was something about it that was strangely addictive.  People who played it more than two or three times were likely to become obsessed by it, and keep coming back over and over.  Some, they said, finally could think of nothing else and went incurably mad.  Some committed suicide.

Some simply... vanished.

The FBI launched an investigation, removing Polybius from arcades wherever they could find it.  The "Men in Black" got involved, and there were reports of mysterious strangers showing up and demanding that arcade owners provide lists of the names (or at least initials) of high scorers in the game.  Those unfortunates were rounded up for psychological testing -- and some of them never returned, either.

There are webpages and subreddits devoted to people's memories of Polybius, their experiences of playing it, and scary things that happened subsequently.  There's just one problem with all this, and you've guessed it:

Polybius never existed.  Despite many, many people searching, there has never been a single Polybius cabinet found, nor even a photograph from the time period showing one.  Oh, sure, we have mock-ups people made long after the fact:

[Image licensed under the Creative Commons DocAtRS, Polybius Arcade 1 cropped, CC BY-SA 3.0]

But hard evidence of the real deal?  Zero.  Nada.  Zip.  Zilch.

So what happened here?

Part of it, of course, was a deliberate hoax; an "urban legend."  Part of it was confabulation of memory with a real event, when an arcade in Portland, Oregon removed a game that had triggered a couple of kids to have a seizure.  There was also an incident in 1981 where the FBI raided arcades that had converted game stations into illegal gambling machines.  There was a 1980 New York Times article citing research (later largely called into question) that playing violent video games predisposes kids to commit violence themselves.  And in 1982, there was a widely-reported incident that a teenager had died while playing the game Berzerk in a Calumet City, Illinois arcade -- the story was true, but his heart failure was caused by a physical defect, and had nothing to do with playing the game.

Put all that together, and there are still people now -- forty-some-odd years later -- who are certain they remember Polybius, and what it was like to play it.

It's another example of the "Mandela Effect," isn't it?  This phenomenon got its name from certain people's memories that Nelson Mandela died in jail -- when in fact, the reality is that he survived, eventually became president of South Africa, and died peacefully in his home in Johannesburg in 2013.  Other examples are that the "Berenstain Bears" -- the annoyingly moralistic cartoon characters who preach such eternal truths as Be Nice To Your Siblings Even When You Feel Like Punching The Shit Out Of Them and Your Parents Are Always Right About Everything and Pay Attention In School Or Else You Are Bad -- were originally the Berenstein Bears (with an "e," not an "a"), that the Fruit of the Loom logo originally had a cornucopia (not just a bunch of fruit), and (I shit you not) that Sri Lanka and New Zealand "should be" in different places.

Almost no one who experiences the Mandela Effect, though, laughs it off and says, "Wow, memory sure is unreliable, isn't it?"  Those memories feel completely real, just as real as memories of stuff you know occurred, that you have incontrovertible hard evidence for.  The idea that you could be so certain of something that never happened is profoundly disconcerting, to the extent that people have looked for some explanation, any explanation, for how their memories ended up with information that is demonstrably false.  Some have even cited the "Many-Worlds" Model of quantum mechanics, and posited that there really is a timeline where Mandela died in prison, the cartoon bears were the "Berenstein Bears," Fruit of the Loom had a cornucopia in its logo, and Sri Lanka and New Zealand were somewhere other than where they now are.  It's just that we've side-slipped into a parallel universe, bringing along our memories of the one where we started -- where all those things were dramatically different.

That's how certain people are that their memories are flawless.  They'd rather believe that the entire universe bifurcated than that they're simply remembering wrong. 

How many times have you been in an argument with a friend, relative, or significant other, and one of you has said, "I know what happened!  I was there!", often with a self-righteous tone that how dare anyone question that they might be recalling things incorrectly?  Well, the truth is that none of us are remembering things correctly; what remains in our mind is a partial record, colored by emotions and second-hand contamination and imagination, blended so well there's no way to tease apart the accurate parts from the inaccurate.  What our memories for sure are not is a factual, blow-by-blow account, a mental video of the past that misses nothing and mistakes nothing.

I know this is kind of a terrifying thing.  Our memories are a huge part of our sense of self; if you want a brilliant (fictional) example of the chaos that happens when our memories become unmoored from reality, watch the fantastic movie Memento, in which the main character (played to perfection by Guy Pearce) has anterograde amnesia, a cognitive disorder where he can't form any new short-term memories.  To compensate for this, he takes Polaroid photographs of stuff he thinks is important, and if it's really important he tattoos it onto his skin.  But then the problem is, how does he know the contents of the photos and tattoos are true?  He has no touchstone for what truth about the past actually is.

Although Pearce's character has an extreme form of this problem, in reality, all of us have the same issue.  Those neural firings in the memory centers of our brain are all we have left of the past -- that, and certain fragmentary records, objects, and writings.  

So, how accurate is our view of the past?

No way to tell.  Better than zero, but certainly far less than one hundred percent.

And there's not even any need for a cursed arcade game to screw around with your perception.  We're built like this -- like it or not.

****************************************

Mental maps

Picture a place you know well.  Your house, your apartment, a park, a church, a school.  You can probably imagine it, remember what it's like to wander around in it, maybe even visualize it to a high level of detail.

Now, let's change the perspective to one you probably have never taken.  Would you be able to draw a map of the layout -- as seen from above?  An aerial view?

Here's a harder task.  In a large room, there are various obstacles, all fairly big and obvious.  Tables, chairs, sofas, the usual things you might find in a living room or den.  You're standing in one corner, and from that perspective are allowed to study it for as long as you like.

Once you were done, could you walk from that corner to the diagonally opposite one without running into anything -- while blindfolded?

Both of these tasks require the use of a part of your brain called the hippocampus.  The name of the structure comes from the Greek word ἱππόκαμπος -- literally, "seahorse" -- because of its shape.  The hippocampus has a role in memory formation, conflict avoidance... and spatial navigation.

Like the other structures in the brain, the hippocampus seems to be better developed in some people than others.  My wife, for example, has something I can only describe as an internal GPS.  To my knowledge, she has never been lost.  When we took a trip to Spain and Portugal a few years ago, we rented a car in Madrid and she studied a map -- once.  After that, she navigated us all over the Iberian Peninsula with only very infrequent checks to make sure we were taking the correct turns, which because of her navigational skills, we always were.

I, on the other hand, get lost walking around a tree.

[Image licensed under the Creative Commons Edward Betts, Bloomsbury - map 1, CC BY-SA 2.0]

The topic comes up because of a paper I came across in the journal Cell that showed something absolutely fascinating.  It's called "Targeted Activation of Hippocampal Place Cells Drives Memory-Guided Spatial Behavior," and was written by a team led by Nick T. M. Robinson of University College London.  But to understand what they did, you have to know about something called optogenetics.

Back in 2002, a pair of geneticists, Boris Zemelman and Gero Miesenböck, developed an amazing technique.  They genetically modified mammalian nerve tissue to express a protein called rhodopsin, which is one of the light-sensitive chemicals in the retina of your eye.  By hitching the rhodopsin to ion-sensitive gateway channels in the neural membrane, they created neurons that literally could be turned on and off using a beam of light.

Because the brain is encased in bone, animals that express this gene don't respond any time the lights are on; you have to shine light directly on the neurons that contain rhodopsin.  This involves inserting fiber optics into the brain of the animal -- but once you do that, you have a set of neurons that fire when you shine a light down the fibers.  Result: remote-control mice.

Okay, if you think that's cool, wait till you hear what Robinson et al. did.

So you create some transgenic mice that express rhodopsin in the hippocampus.  Fit them out with fiber optics.  Then let the mice learn how to run a maze for a reward, in this case sugar water in a feeder bottle.  Watch through an fMRI and note which hippocampal neurons are firing when they learn -- and especially when they recall -- the layout of the maze.

Then take the same mice, put them in a different maze.  But switch the lights on in their brain to activate the neurons you saw firing when they were recalling the map of the first maze.

The result is that the mice picture the first maze, and try to run that pattern even though they can see that they are now in a different maze.  The light activation has switched on a memory of the layout of the maze they'd learned that then overrode all the other sensory information they had access to.

It's as if you moved from Tokyo to London, and then tried to use your knowledge of the roads of Tokyo to find your way from St. Paul's Cathedral to the Victoria & Albert Museum.

This is pretty astonishing from a number of standpoints.  First, the idea that you can switch a memory on and off like that is somewhere between fascinating and freaky.  Second, that the neural firing pattern is so specific -- that pattern corresponds to that map, and no other.  And third, that the activation of the map made the mice doubt the information coming from their own eyes.

So once again, we have evidence of how plastic our brains are, and how easy they are to fool.  What you're experiencing right now is being expressed in your brain as a series of neural firings; in a way, the neural firing pattern is the experience.  If you change the pattern artificially, you experience something different.

More disturbing still is that our sense of self is also deeply tied to our neural links (some would say that our sense of self is nothing more than neural links; to me, the jury's still out on where consciousness comes from, so I'm hesitant to go that far).  So not only what you perceive, but who you are can change if you alter the pattern of neural activation.

We're remarkable, complex, amazing, and fragile beasts, aren't we?

So that's today's contribution from the Not Science Fiction department.  I'm wondering if I might be able to get one of those fiber optics things to activate my hippocampus.  Sounds pretty extreme, but I am really tired of getting lost all the time.  There are trees everywhere around here.

****************************************

The persistence of memory

A paper published this week in the journal Nature: Scientific Reports provided some interesting insights into how our memories of our own past might work -- but also raised a couple of troubling questions in my mind.

It's called "Illusory Ownership of One's Younger Face Facilitates Access to Childhood Episodic Autobiographical Memories," and was the work of Utkarsh Gupta, Peter Bright, Alex Clarke, Waheeb Zafar, Pilar Recarte-Perez and Jane E. Aspell, of Anglia Ruskin University.  Here's their description of what they did:

Our autobiographical memories reflect our personal experiences at specific times in our lives.  All life events are experienced while we inhabit our body, raising the question of whether a representation of our bodily self is inherent in our memories.  Here we explored this possibility by investigating if the retrieval of childhood autobiographical memories would be influenced by a body illusion that gives participants the experience of ownership for a ‘child version’ of their own face.  Fifty neurologically healthy adults were tested in an online enfacement illusion study.  Feelings of ownership and agency for the face were greater during conditions with visuo-motor synchrony than asynchronous conditions.  Critically, participants who enfaced (embodied) their child-like face recollected more childhood episodic memory details than those who enfaced their adult face.  No effects on autobiographical semantic memory recollection were found.  This finding indicates that there is an interaction between the bodily self and autobiographical memory, showing that temporary changes to the representation and experience of the bodily self impacts access to memory.

Which is fascinating.  Given the sensation of inhabiting our own (younger) body, we seem to unlock stored memories we previously could not access.  It makes me wonder what's up there in our memory centers, you know?  Assuming your brain is physiologically normal and uninjured, do you really have a record of everything that's happened to you in there somewhere, just waiting for the right trigger to release it?

"Our findings suggest that the bodily self and autobiographical memory are linked, as temporary changes to bodily experience can facilitate access to remote autobiographical memories," said study senior author Jane Aspell, in an interview with Science Daily.  "These results are really exciting and suggest that further, more sophisticated body illusions could be used to unlock memories from different stages of our lives -- perhaps even from early infancy.  In the future it may even be possible to adapt the illusion to create interventions that might aid memory recall in people with memory impairments."

Here's the thing, though.

How do they know the memories these volunteers reported are real?

[Image is in the Public Domain]

Let me give you an example from my own childhood.

When I was about four, my parents and I moved from a house in South Charleston, West Virginia to one in nearby Saint Albans.  My dad worked at the Marine Corps Recruiting and Training Station at the time, and the move was basically to a nicer neighborhood.  We'd lived in a rental next door to a big house I remember as "the green house" -- it was a blocky rectangular thing, two story, painted light green, where a family with two older boys (at a guess, perhaps seven and nine) lived.

Well, on moving day, my parents were loading the last stuff in the car, and had told me to entertain myself for a half-hour or so while they were finishing up.  I wandered into the yard in front of the green house, and the two boys who lived there asked me if I wanted to play.  I said "sure," and we went inside, then upstairs -- where they thought it'd be funny to trap me, and convince me my parents were going to leave without me.

I looked down from the window, screaming and trying to alert my mother, but she didn't hear me.  I was terrified of being left behind (not, realistically, that this would ever have happened).  Eventually the two boys relented and let me go, and I rejoined my parents -- me still tearful and freaking out about my near miss, they wondering what the hell had upset me.

Here's the kicker, though:  I have no idea if this actually happened.

I asked my mother about it some years later, and she had no memory of it -- she didn't recall my disappearing, even for a short time, on the day we moved, nor returning upset and scared.  "Why would I have told you to run off and play when we were about to leave?" she asked, which I had to admit was a good question.  I have zero other memories of the two boys next door (other than that they existed), and to my knowledge I never went inside their house, nor was invited by them to play, on any other occasion.  I've always been prone to vivid dreams; I remember being somewhat older, perhaps eight or nine, and having flying dreams so realistic that upon awakening I was halfway convinced they'd really happened.  I might be recalling an unusually detailed (and terrifying) dream; or maybe there were two neighbor boys who thought it'd be funny to scare the living shit out of a gullible little kid.

The problem is, there's no way to tell which is the truth.

So I have no doubt that the Gupta et al. study triggered the release of something in the minds of the volunteers, but I think it's a stretch to conclude that what they accessed were real and accurate memories.  I've seen plenty of evidence -- both from scientific studies and the experiences of me and my friends -- indicating that our memories are plastic, malleable, easily warped, and inaccurate.  We all too readily conflate our recollections of what actually happened with (1) what we think happened, (2) what we were told happened, and (3) outright mental fabrications.  A famous -- if unsettling -- study from Portsmouth University in 2008 looked at people's memories of the 2005 terrorist bombing of a double-decker bus in London, and found that many people recalled intricate and vivid detail from CCTV footage of the explosion, and made statements like, "The bus had just stopped to let people off when two women and a man got on" and "He placed a bag by his side, the woman sat down and as the bus left, there was an explosion" and "There was a severed leg on the floor" and "The bus had stopped at a traffic light when there was a bright light, a loud bang and the top flew off."

The problem?  There is no CCTV footage of the explosion.  None.  Presented with that fact, people were astonished.  That couldn't be true, they said; they knew they'd seen it, they could still picture it, still recall how upset they'd been watching it.  One person, told that no video of the event existed, accused the researchers of lying.

So there you have it.  Another reason not to trust your own recollections of past events, and a caution not to get your hopes up about accessing them by visualizing yourself as a child.  Me, I'd just as soon not remember a lot of that stuff.  Even if I was never kidnapped by the neighbors when I was four, I didn't exactly have a happy childhood.  I'd just as soon remain in the present, thank you very much.

****************************************

Signal out of noise

A paper this week out of the University of Washington describes research suggesting that intelligence is positively correlated with the ability to discern what someone is saying in a noisy room.

This was a little distressing to me, because I am terrible at this particular skill.  When I'm in a bar or other loud, chaotic environment, I can often pick out a few words, but understanding entire sentences is tricky.  I also run out of steam really quickly -- I can focus for a while, but suddenly the whole thing descends into a wall of noise.

The evidence, though, seems strong.  "The relationship between cognitive ability and speech-perception performance transcended diagnostic categories," said Bonnie Lau, lead author on the paper.  "That finding was consistent across all three groups studied [an autistic group, a group who had fetal alcohol syndrome, and a neurotypical control group]."

So.  Yeah.  Not a favorable result for yours truly.  I mean, I get why it makes sense; focusing on one conversation when there are others going on is a complex task.  "You have to segregate the streams of speech," Lau explained.  "You have to figure out and selectively attend to the person that you're interested in, and part of that is suppressing the competing noise characteristics.  Then you have to comprehend from a linguistic standpoint, coding each phoneme, discerning syllables and words.  There are semantic and social skills, too -- we're smiling, we're nodding.  All these factors increase the cognitive load of communicating when it is noisy."

While I'm not seriously concerned that about the implications regarding my own intelligence, it does make me wonder about sensory synthesis and interpretation in general.  A related phenomenon I've noticed is that if there is a song playing while there's noise going on -- in a restaurant, or on earphones at the gym -- I often have no idea what the song is, can't understand a single word or pick up the beat or figure out the music, until something clues me in to what the song is.  Then, all of a sudden, I find I'm able to hear it clearly.

A while back, some neuroscientists at the University of California - Berkeley elucidated what's happening in the brain that causes this oddity in auditory perception, and it provides an interesting contrast to this week's study.  A paper in Nature: Communications in 2016, by Christopher R. Holdgraf, Wendy de Heer, Brian Pasley, Jochem Rieger, Nathan Crone, Jack J. Lin, Robert T. Knight, and Frédéric E. Theunissen, considered how the perception of garbled speech changes when subjects are told what's being said -- and found through a technique called spectrotemporal receptive field mapping that the brain is able to retune itself in less than a second.

The authors write:

Experience shapes our perception of the world on a moment-to-moment basis.  This robust perceptual effect of experience parallels a change in the neural representation of stimulus features, though the nature of this representation and its plasticity are not well-understood.  Spectrotemporal receptive field (STRF) mapping describes the neural response to acoustic features, and has been used to study contextual effects on auditory receptive fields in animal models.  We performed a STRF plasticity analysis on electrophysiological data from recordings obtained directly from the human auditory cortex. Here, we report rapid, automatic plasticity of the spectrotemporal response of recorded neural ensembles, driven by previous experience with acoustic and linguistic information, and with a neurophysiological effect in the sub-second range.  This plasticity reflects increased sensitivity to spectrotemporal features, enhancing the extraction of more speech-like features from a degraded stimulus and providing the physiological basis for the observed ‘perceptual enhancement’ in understanding speech.

What astonishes me about this is how quickly the brain is able to accomplish this -- although that is certainly matched by my own experience of suddenly being able to hear lyrics of a song once I recognize what's playing.  As James Anderson put it, writing about the research in ReliaWire, "The findings... confirm hypotheses that neurons in the auditory cortex that pick out aspects of sound associated with language, the components of pitch, amplitude and timing that distinguish words or smaller sound bits called phonemes, continually tune themselves to pull meaning out of a noisy environment."

A related phenomenon is visual priming, which occurs when people are presented with a seemingly meaningless pattern of dots and blotches, such as the following:

Once you're told that the image is a cow, it's easy enough to find -- and after that, impossible to unsee.

"Something is changing in the auditory cortex to emphasize anything that might be speech-like, and increasing the gain for those features, so that I actually hear that sound in the noise," said study co-author Frédéric Theunissen.  "It’s not like I am generating those words in my head. I really have the feeling of hearing the words in the noise with this pop-out phenomenon.  It is such a mystery."

Apparently, once the set of possibilities of what you're hearing (or seeing) is narrowed, your brain is much better at extracting meaning from noise.  "Your brain tries to get around the problem of too much information by making assumptions about the world," co-author Christopher Holdgraf said.  "It says, ‘I am going to restrict the many possible things I could pull out from an auditory stimulus so that I don’t have to do a lot of processing.’  By doing that, it is faster and expends less energy."

It makes me wonder about the University of Washington finding, though, if there might be an association between poor auditory discernment and attention-related disorders like ADHD.  My own experience is that I can focus on what's being said in a noisy environment, it's just exhausting.  Perhaps -- like with the song phenomenon, and things like visual priming -- chaotic brains like mine simply can't throw away extraneous information fast enough to retune.  Eventually, it just gives up, and the whole world turns into white noise.

In any case, there's another fascinating, and mind-boggling, piece of how our brains make sense of the world.  It's wonderful that evolution could shape such an amazingly adaptive device, although the survival advantage is obvious.  The faster you are at pulling a signal out of the noise, the more likely you are to make the right decisions about what it is that you're perceiving -- whether it's you talking to a friend in a crowded bar or a proto-hominid on the African savanna trying to figure out if that odd shape in the grass is a predator lying in wait.  

Even if it means that I personally would probably have been a lion's afternoon snack.

****************************************

Facing facts

"I'm sorry, but I have no idea who you are."

I can't tell you how many times I've had to utter that sentence.  Regular readers of Skeptophilia know why; I have a peculiar disability called prosopagnosia, or "face blindness."  I have a nearly complete inability to recognize faces, even of people I've known for some time.

Well, that's not exactly true.  I recognize people differently than other people do.  I remember the people I know as lists of features.  I know my wife has curly brown hair and freckles and an infectious smile, but I honestly have no mental image of her.  I can't picture my own face, although -- like with my wife -- I could list some of my features.

That system doesn't have a high success rate, however, and a lot of the time I have no idea who the people around me are, especially in a place where there are few clues from context.  I have pretty serious social anxiety, and my condition makes it worse, having put me in the following actual situations:

• introducing myself twice to the same person at a party
• getting a big, enthusiastic hug and an "it's been so long!" from someone in our local gym, and never figuring out who I was talking to
• having two of my students switch seats and not realizing it for three weeks, until finally they 'fessed up
• going to see a movie, and not knowing until the credits rolled that the main characters were played by Kenneth Branagh, Penelope Cruz, Judi Dench, Derek Jacobi, Michelle Pfeiffer, Daisy Ridley, and Johnny Depp
• countless incidents of my fishing for clues ("so, how's your, um... spouse, parents, kids, pets, job..."), sometimes fruitlessly

My anxiety has made me really good at paying attention to, and recalling, other cues like voice, manner of dress, posture, walk, hair style, and so on.  But when one or more of those change -- such as with the student I had one year who cut her hair really short during the summer, and whom I didn't recognize when she showed up in one of my classes on the first day of school the following year -- it doesn't always work.

One up side to the whole thing is that I do get asked some funny questions.  One student asked me if when I looked at people, their faces were invisible.  Another asked me if when I look in the bathroom mirror in the morning, I don't know that's me.  (It's a pretty shrewd guess that it is me, since there's generally no one else in there at the time.)

But at least it's not as bad as the dumb questions that my former students who are identical triplets sometimes get.  One of them was once asked by a friend how she kept track of which triplet she was.

No, I'm not kidding.  Neither, apparently, was the person who asked the question.

[Image licensed under the Creative Commons Randallbritten, FaceMachine screenshots collage, CC BY-SA 3.0]

In any case, all of this comes up because of some research that came out in the journal Cortex that tried to parse what's happening (or what's not happening) in the brains of people like me.  Some level of prosopagnosia affects about one person in fifty; some of them lose their facial recognition ability because of a stroke or other damage to the fusiform gyrus, the part of the brain that seems to be a dedicated face-memory module.  Others, like me, were born this way.  Interestingly, a lot of people who have lifelong prosopagnosia take a while to figure it out; for years, I just thought I was unobservant, forgetful, or a little daft.  (All three of those might be true as well, of course.)  It was only after I had enough embarrassing incidents occur, and -- most importantly -- saw an eye-opening piece about face blindness by Leslie Stahl on 60 Minutes, that I realized what was going on.

In any case, the paper in Cortex looked at trying to figure out why people who are face-blind often do just fine on visual perception tests, then fail utterly when it comes to remembering photographs of faces.  The researchers specifically tried to parse whether the difference was coming from an inability to connect context cues to the face you're seeing (e.g., looking at someone and thinking, "She's the woman who was behind the counter at the library last week") versus simple familiarity (the more nebulous and context-free feeling of "I've seen that person before").  They showed each test subject (some of whom weren't face-blind) a series of 120 faces, then a second series of 60 faces where some of them were new and some of them were in the previous series.  The researchers were not only looking for whether the subjects could correctly pick out the old faces, but how confident they were in their answers -- the surmise being that low confidence on correct answers was an indicator of relying on familiarity rather than context memory.

The prosopagnosics in the test group not only were bad at identifying which faces were old and which ones they'd seen before; but their confidence was really low, even on the ones they got right.  Normally-sighted people showed a great deal more certainty in their answers.  What occurs to me, though, is that knowing they're face-blind would skew the results, in that we prosopagnosics are always doubtful we're recalling correctly.  So these data could be a result of living with the condition, not some kind of underlying mechanism at work.  I almost never greet someone first, because even if I think I might know them, I'm never certain.  A lot of people think I'm aloof because of this, but the reality is that I honestly don't know which of the people I'm seeing are friends and which are total strangers.

One thing about the researchers' conclusion does ring true, however.  The subconscious "feeling of familiarity" is definitely involved.  My experience of face blindness isn't that I feel like I'm surrounded by strangers; it's more that everyone looks vaguely familiar.  The problem is, that feeling is no stronger when I see a close friend than when I see someone I've never met before, so the intensity of that sense -- what apparently most people rely on -- doesn't help me.

So that's the view of the world through the eyes of someone who more often than not doesn't know who he's looking at.  Fortunately for me, (1) at this point in my life I'm unembarrassed by my condition, and (2) most of the people in my little village know I'm face-blind and will say, "Hi, Gordon, it's Steve..." when they walk up, and spare me the awkwardness of fishing for clues.  (Nota bene: This only works if it actually is Steve.  Otherwise it would be even more awkward.)  But hopefully some good will come from this research, because face blindness is kind of a pain in the ass.

"Our results underscore that prosopagnosia is a far more complex disorder that is driven by more than deficits in visual perception," said study first author Anna Stumps, a researcher in the Boston Attention Learning Laboratory at VA Boston.  "This finding can help inform the design of new training approaches for people with face blindness."

Which would be really, really nice.

****************************************

Remembrance of things past

"The human brain is rife with all sorts of ways of getting it wrong."

This quote is from a talk by eminent astrophysicist Neil deGrasse Tyson, and is just about spot on.  Oh, sure, our brains work well enough, most of the time; but how many times have you heard people say things like "I remember that like it was yesterday!" or "Of course it happened that way, I saw it with my own eyes"?

Anyone who knows something about neuroscience should immediately turn their skepto-sensors up to 11 as soon as they hear either of those phrases.

fMRI scan of a human brain during working memory tasks [Image is in the Public Domain courtesy of the Walter Reed National Military Medical Center]

Our memories and sensory-perceptual systems are selective, inaccurate, heavily dependent on what we're doing at the time, and affected by whether we're tired or distracted or overworked or (even mildly) inebriated.  Sure, what you remember might have happened that way, but -- well, let's just say it's not as much of a given as we'd like to think.  An experiment back in 2005 out of the University of Portsmouth looked memories of the Tavistock Square (London) bus bombing, and found that a full forty percent of the people questioned had "memories" of the event that were demonstrably false -- including a number of people who said they recalled details from CCTV footage of the explosion, down to what people were wearing, who showed up to help the injured, when police arrived, and so on.

Oddly enough, there is no CCTV footage of the explosion.  It doesn't exist and has never existed.

Funny thing that eyewitness testimony is considered some of the most reliable evidence in courts of law, isn't it?

There are a number of ways our brains can steer us wrong, and the worst part of it all is that they leave us simultaneously convinced that we're remembering things with cut-crystal clarity.  Here are a few interesting memory glitches that commonly occur in otherwise mentally healthy people, that you might not have heard of:

• Cryptomnesia.  Cryptomnesia occurs when something from the past recurs in your brain, or arises in your external environment, and you're unaware that you've already experienced it.  This has resulted in several probably unjustified accusations of plagiarism; the author in question undoubtedly saw the text they were accused of plagiarizing some time earlier, but honestly didn't remember they'd read it and thought that what they'd come up with was entirely original.  It can also result in some funnier situations -- while the members of Aerosmith were taking a break from recording their album Done With Mirrors, they had a radio going, and the song "You See Me Crying" came on.  Steven Tyler said he thought that was a pretty cool song, and maybe they should record a cover of it.  Joe Perry turned to him in incredulity and said, "That's us, you fuckhead."

• Semantic satiation.  This is when a word you know suddenly looks unfamiliar to you, often because you've seen it repeatedly over a fairly short time.  Psychologist Chris Moulin of Leeds University did an experiment where he had test subjects write the word door over and over, and found that after a minute of this 68% of the subjects began to feel distinctly uneasy, with a number of them saying they were doubting that "door" was a real word.  I remember being in high school writing an exam in an English class, and staring at the word were for some time because I was convinced that it was spelled wrong (but couldn't, of course, remember how it was "actually" spelled).

• Confabulation.  This is the recollection of events that never happened -- along with a certainty that you're remembering correctly.  (The people who claimed false memories of the Tavistock Square bombing were suffering from confabulation.)  The problem with this is twofold; the more often you think about the false memory or tell your friends and family about it, the more sure you are of it; and often, even when presented with concrete evidence that you're recalling incorrectly, somehow you still can't quite believe it.  A friend of mine tells the story of trying to help her teenage son find his car keys, and that she was absolutely certain that she'd seen them that day lying on a blue surface -- a chair, tablecloth, book, she wasn't sure which, but it was definitely blue.  They turned the house upside down, looking at every blue object they could find, and no luck.  Finally he decided to walk down to the bus stop and take the bus instead, and went to the garage to get his stuff out of the car -- and the keys were hanging from the ignition, where he'd left them the previous evening.  "Even after telling me this," my friend said, "I couldn't accept it.  I'd seen those keys sitting on a blue surface earlier that day, and remembered it as clearly as if they were in front of my face."

• Declinism.  This is the tendency to remember the past as more positive than it actually was, and is responsible both for the "kids these days!" thing and "Make America Great Again."  There's a strong tendency for us to recall our own past as rosy and pleasant as compared to the shitshow we're currently immersed in, irrespective of the fact that violence, bigotry, crime, and general human ugliness are hardly new inventions.  (A darker aspect of this is that some of us -- including a great many MAGA types -- are actively longing to return to the time when straight White Christian men were in charge of everything; whether this is itself a mental aberration I'll leave you to decide.)  A more benign example is what I've noticed about travel -- that after you're home, the bad memories of discomfort and inconveniences and delays and questionable food fade quickly, leaving behind only the happy feeling of how much you enjoyed the experience.

• The illusion of explanatory depth.  This is a dangerous one; it's the certainty that you understand deeply how something works, when in reality you don't.  This effect was first noted back in 2002 by psychologists Leonid Rozenblit and Frank Keil, who took test subjects and asked them to rank from zero to ten their understanding of how common devices worked, including zippers, bicycles, electric motors, toasters, and microwave ovens, and found that hardly anyone gave themselves a score lower than five on anything.  Interestingly, the effect vanished when Rozenblit and Keil asked the volunteers actually to explain how the devices worked; after trying to describe in writing how a zipper works, for example, most of test subjects sheepishly realized they actually had no idea.  This suggests an interesting strategy for dealing with self-styled experts on topics like climate change -- don't argue, ask questions, and let them demonstrate their ignorance on their own.

• Presque vu.  Better known as the "tip-of-the-tongue" phenomenon -- the French name means "almost seen" -- this is when you know you know something, but simply can't recall it.  It's usually accompanied by a highly frustrating sense that it's right there, just beyond reach.  Back in the days before The Google, I spent an annoyingly long time trying to recall the name of the Third Musketeer (Athos, Porthos, and... who???).  I knew the memory was in there somewhere, but I couldn't access it.  It was only after I gave up and said "to hell with it" that -- seemingly out of nowhere -- the answer (Aramis) popped into my head.  Interestingly, neuroscientists are still baffled as to why this happens, and why turning your attention to something else often makes the memory reappear.

So be a little careful about how vehemently you argue with someone over whether your recollection of the past or theirs is correct.  Your version might be right, or theirs -- or it could easily be that both of you are remembering things incompletely or incorrectly.  I'll end with a further quote from Neil deGrasse Tyson: "We tend to have great confidence in our own brains, when in fact we should not.  It's not that eyewitness testimony by experts or people in uniform is better than that of the rest of us; it's all bad....  It's why we scientists put great faith in our instruments.  They don't care if they've had their morning coffee, or whether they got into an argument with their spouse -- they get it right every time."

****************************************

Face forward

Life with prosopagnosia is peculiar sometimes.

Better known as "face blindness," it's a partial or complete inability to recognize people's faces.  I'm not sure where I fall on the spectrum -- I'm certainly nowhere as bad as neuroscientist and author Oliver Sacks, who didn't recognize his own face in the mirror.  Me, I'm hampered by it, but have learned to compensate by being very sensitive to people's voices and how they move.  (I've noticed that I'm often more certain who someone is if I see them walking away than I am if they're standing right in front of me.)

Still, it results in some odd situations sometimes.  I volunteer once a week as a book sorter at our local Friends of the Library book sale, and there's this one guy named Rich who is absolutely a fixture -- he always seems to be there.  I've seen him and spoken with him at least a hundred times.  Well, a couple of weeks ago, I was working, and there was this guy who was behind the counter, messing with stuff.  I was about to ask who he was and what he was doing, when he said something, and I realized it was Rich -- who had shaved off his facial hair.

Until he opened his mouth, I honestly had no idea I'd ever seen him in my life.

Then, a couple of nights ago, my wife and I were watching the Doctor Who Christmas episode "Joy to the World," and afterward got to see a thirty-second teaser trailer for season two, which is being released next spring.  Well, in season one, there was this mysterious recurring character named Mrs. Flood (played by British actress Anita Dobson) whose role we have yet to figure out, and who has the Who fandom in quite the tizzy.  And in the trailer, there's a quick clip of an old woman in formal attire watching a theater performance through opera glasses, and until another fan said, "What did you think about the appearance of Mrs. Flood in the trailer?" I had no clue -- not the least suspicion -- that it was her.

So it's kind of inconvenient, sometimes.  When people post still shots from movies or television shows on social media, I usually not only don't know who the actors are, I have no idea what film it's from (unless there's an obvious clue from the setting).  And as I've related before, there are times when even my voice-recognition strategy hasn't worked, and I've had entire conversations with people and then left still not knowing who it was I'd been talking to.

The reason the topic comes up (again) is some research out of Toyohashi University of Technology that was the subject of a paper in the Journal of Vision last week.  The researchers were trying to figure out if humans have a better innate ability to filter out extraneous visual distractions when it comes to facial recognition than they do for recognizing other objects.  Using a technique called "continuous flash suppression" (CFS), they presented volunteers with fast-moving high-contrast images in one eye, and a target image in the other, then using an fMRI measured how long it took the brain's visual recognition centers to "break through" the distraction and recognize the target image.

If the target image was a face -- or "face-like" -- that breakthrough happened much faster than it did with any other sort of image.  And, interestingly, the breakthrough time was significantly slowed for faces that were upside-down.

We're wired, apparently, to recognize right-side-up human faces faster than just about anything else.

"Our study shows that even vague, face-like images can trigger subconscious processing in the brain, demonstrating how deeply rooted facial recognition is in our visual system," said Makoto Michael Martinsen, who co-authored the study.  "This ability likely evolved to help us prioritize faces, which are critical for social interaction, even when visual information is scarce...  [However] we didn’t consider factors like emotion or attractiveness, which can affect facial perception...  Despite this, our study highlights the brain’s incredible ability to extract important information from minimal cues, especially when it comes to faces.  It emphasizes the importance of facial features in both conscious and subconscious perception and raises interesting questions about how this mechanism evolved."

Naturally, I found myself wondering how face-blind people like myself would do in this task.  After all, it's not that we can't tell something is a face; it's that the visual information in a face doesn't trigger the same instantaneous recall it does in other people.  When I do recognize someone visually, it's more that I remember a list of their features -- he's the guy with square plastic frame glasses and curly gray hair, she's the woman with a round face and dark brown eyes who favors brightly-colored jewelry.  This, of course, only takes me so far.  When someone changes their appearance -- like Rich shaving off his beard and mustache -- it confounds me completely.

So I'm curious whether I'd be like the rest of the test subjects and have faster recognition times for faces than for non-face objects, or if perhaps my peculiar wiring means my brain weights all visual stimuli equally.  I'd be happy to volunteer to go to Japan to participate, if anyone wants to find out the answer badly enough to spot me for a plane ticket.

No?  Oh, well, perhaps that'll be the next phase of Martinsen et al.'s research.  I'm willing to wait.

Until then -- if I know you, and happen to run into you in the local café, keep in mind I may have no idea who you are.  It helps if you start the conversation with, "I'm _____" -- I'm not embarrassed by my odd neurological condition, and it's better than spending the day wondering who the person was who came up and gave me a hug and asked about my wife and kids and dogs and whatnot.

****************************************

Talking in your sleep

A little over a year ago, I decided to do something I've always wanted to do -- learn Japanese.

I've had a fascination with Japan since I was a kid.  My dad lived there for a while during the 1950s, and while he was there collected Japanese art and old vinyl records of Japanese folk and pop music, so I grew up surrounded by reminders of the culture.  As a result, I've always wanted to learn more about the country and its people and history, and -- one day, perhaps -- visit.

So in September of 2023 I signed up for Duolingo, and began to inch my way through learning the language.

[Image is in the Public Domain]

It's a challenge, to say the least.  Japanese usually shows up on lists of "the five most difficult languages to learn."  Not only are there the three different scripts you have to master in order to be literate, the grammatical structure is really different from English.  The trickiest part, at least thus far, is managing particles -- little words that follow nouns and indicate how they're being used in the sentence.  They're a bit like English prepositions, but there's a subtlety to them that is hard to grok.  Here's a simple example:

Watashi wa gozen juuji ni tokoshan de ane aimasu.

(I) (particle indicating the subject of the sentence) (A.M.) (ten o'clock) (particle indicating movement or time) (library) (particle indicating where something is happening) (my sister) (am meeting with) = "I am meeting my sister at ten A.M. at the library."

Get the particles wrong, and the sentence ends up somewhere between grammatically incorrect and completely incomprehensible.

So I'm coming along.  Slowly.  I have a reasonably good affinity for languages -- I grew up bilingual (English/French) and have a master's degree in linguistics -- but the hardest part for me is simply remembering the vocabulary.  The grammar patterns take some getting used to, but once I see how they work, they tend to stick.  The vocabulary, though?  Over and over again I'll run into a word, and I'm certain I've seen it before and at one point knew what it meant, and it will not come back to mind.  So I look it up...

... and then go, "Oh, of course.  Duh.  I knew that."

But according to a study this week out of the University of South Australia, apparently what I'm doing wrong is simple: I need more sleep.

Researchers in the Department of Neuroscience took 35 native English speakers and taught them "Mini-Pinyin" -- an invented pseudolanguage that has Mandarin Chinese vocabulary but English sentence structure.  (None of them had prior experience with Mandarin.)  They were sorted into two groups; the first learned the language in the morning and returned twelve hours later to be tested, and the second learned it in the evening, slept overnight in the lab, and were tested the following morning.

The second group did dramatically better than the first.  Significantly, during sleep their brains showed a higher-than-average level of brain wave patterns called slow oscillations and sleep spindles, that are thought to be connected with memory consolidation -- uploading short-term memories from the hippocampus into long-term storage in the cerebral cortex.  Your brain, in effect, talks in its sleep, routing information from one location to another.

"This coupling likely reflects the transfer of learned information from the hippocampus to the cortex, enhancing long-term memory storage," said Zachariah Cross, who co-authored the study.  "Post-sleep neural activity showed unique patterns of theta oscillations associated with cognitive control and memory consolidation, suggesting a strong link between sleep-induced brainwave co-ordination and learning outcomes."

So if you're taking a language class, or if -- like me -- you're just learning another language for your own entertainment, you're likely to have more success in retention if you study in the evening, and get a good night's sleep before you're called upon to use what you've learned.

Of course, many of us could use more sleep for a variety of other reasons.  Insomnia is a bear, and poor sleep is linked with a whole host of health-related woes.  But a nice benefit of dedicating yourself to getting better sleep duration and quality is an improvement in memory.

And hopefully for me, better scores on my Duolingo lessons.

****************************************

Memory boost

About two months ago I signed up with Duolingo to study Japanese.

I've been fascinated with Japan and the Japanese culture pretty much all my life, but I'm a total novice with the language, so I started out from "complete beginner" status.  I'm doing okay so far, although the fact that it's got three writing systems is a challenge, to put it mildly.  Like most Japanese programs, it's beginning with the hiragana system -- a syllabic script that allows you to work out the pronunciation of words -- but I've already seen a bit of katakana (used primarily for words borrowed from other languages) and even a couple of kanji (the ideographic script, where a character represents an entire word or concept).

[Image licensed under the Creative Commons 663highland, 140405 Tsu Castle Tsu MIe pref Japan01s, CC BY-SA 3.0]

While Duolingo focuses on getting you listening to spoken Japanese right away, my linguistics training has me already looking for patterns -- such as the fact that wa after a noun seems to act as a subject marker, and ka at the end of a sentence turns it into a question.  I'm still perplexed by some of the pronunciation patterns -- why, for example, vowel sounds sometimes don't get pronounced.  The first case of this I noticed is that the family name of the brilliant author Akutagawa Ryūnosuke is pronounced /ak'tagawa/ -- the /u/ in the second syllable virtually disappears.  I hear it happening fairly commonly in spoken Japanese, but I haven't been able to deduce what the pattern is.  (If there is one.  If there's one thing my linguistics studies have taught me, it's that all languages have quirks.  Try explaining to someone new to English why, for instance, the -ough combination in cough, rough, through, bough, and thorough are all pronounced differently.) 

Still and all, I'm coming along.  I've learned some useful phrases like "Sushi and water, please" (Sushi to mizu, kudasai) and "Excuse me, where is the train station?" (Sumimasen, eki wa doko desu ka?), as well as less useful ones like "Naomi Yamaguchi is cute" (Yamaguchi Naomi-san wa kawaii desu), which is only critical to know if you have a cute friend who happens to be named Naomi Yamaguchi.

The memorization, however, is often taxing to my 63-year-old brain.  Good for it, I have no doubt -- a recent study found that being bi- or multi-lingual can delay the onset of dementia by four years or more -- but it definitely is a challenge.  I go through my hiragana flash cards at least once a day, and have copious notes for what words mean and for any grammatical oddness I happen to notice.  Just the sheer amount of memorization, though, is kind of daunting.

Maybe what I should do is find a way to change the context in which I have to remember particular words, phrases, or characters.  That seems to be the upshot of a study I ran into a couple of days ago in Proceedings of the National Academy of Sciences, about a study by a group from Temple University and the University of Pittsburgh about how to improve retention.

I'm sure all of us have experienced the effects of cramming for a test -- studying like hell the night before, and then you do okay on the test but a week later barely remember any of it.  This practice does two things wrong; not only stuffing all the studying into a single session, but doing it all the same way.

What this study showed was two factors that significantly improved long-term memory.  One was spacing out study sessions -- doing shorter sessions more often definitely helped.  I'm already approaching Duolingo this way, usually doing a lesson or two over my morning coffee, then hitting it again for a few more after dinner.  But the other interesting variable they looked at was that test subjects' memories improved substantially when the context was changed -- when, for example, you're trying to remember as much as you can of what a specific person is wearing, but instead of being shown the same photograph over and over, you're given photographs of the person wearing the same clothes but in a different setting each time.

"We were able to ask how memory is impacted both by what is being learned -- whether that is an exact repetition or instead, contains variations or changes -- as well as when it is learned over repeated study opportunities," said Emily Cowan, lead author of the study.  "In other words... we could examine how having material that more closely resembles our experiences of repetition in the real world -- where some aspects stay the same but others differ -- impacts memory if you are exposed to that information in quick succession versus over longer intervals, from seconds to minutes, or hours to days."

I can say that this is one of the things Duolingo does right.  Words are repeated, but in different combinations and in different ways -- spoken, spelled out using the English transliteration, or in hiragana only.  Rather than always seeing the same word in the same context, there's a balance between the repetition we all need when learning a new language and pushing your brain to generalize to slightly different usages or contexts.

So all things considered, Duolingo had it figured out even before the latest research came out.  I'm hoping it pays off, because my son and I would like to take a trip to Japan at some point and be able to get along, even if we don't meet anyone cute named Naomi Yamaguchi.  But I should wind this up, so for now I'll say ja ane, mata ashita (goodbye, see you tomorrow).

****************************************