Falling into the uncanny valley

As we get closer and closer to something that is unequivocally an artificial intelligence, engineers have tackled another aspect of this; how do you create something that not only acts (and interacts) intelligently, but looks human?

It's a harder question than it appears at first.  We're all familiar with depictions of robots from movies and television -- from ones that made no real attempt to mimic the human face in anything more than the most superficial features (such as the robots in I, Robot and the droids in Star Wars) to ones where the producers effectively cheated by having actual human actors simply try to act robotic (the most famous, and in my opinion the best, was Commander Data in Star Trek: The Next Generation).  The problem is, we are so attuned to the movement of faces that we can be thrown off, even repulsed, by something so minor that we can't quite put our finger on what exactly is wrong.

This phenomenon was noted a long time ago -- first back in 1970, when roboticist Masahiro Mori coined the name "uncanny valley" to describe the phenomenon.  His contention, which has been borne out by research, is that we generally do not have a strong negative reaction to clearly non-human faces (such as teddy bears, the animated characters in most kids' cartoons, and the aforementioned non-human-looking robots).  But as you get closer to accurately representing a human face, something fascinating happens.  We suddenly start being repelled -- the sense is that the face looks human, but there's something "off."  This has been a problem not only in robotics but in CGI; in fact, one of the first and best-known cases of an accidental descent into the uncanny valley was the train conductor in the CGI movie The Polar Express, where a character who was supposed to be friendly and sympathetic ended up scaring the shit out of the kids for no very obvious reason.

As I noted earlier, the difficulty is that we evolved to extract a huge amount of information from extremely subtle movements of the human face.  Think of what can be communicated by tiny gestures like a slight lift of a eyebrow or the momentary quirking upward of the corner of the mouth.  Mimicking that well enough to look authentic has turned out to be as challenging as the complementary problem of creating AI that can act human in other ways, such as conversation, responses to questions, and the incorporation of emotion, layers of meaning, and humor.

The latest attempt to create a face with human expressivity comes out of Columbia University, and was the subject of a paper in arXiv this week called "Smile Like You Mean It: Animatronic Robotic Face with Learned Models," by Boyuan Chen, Yuhang Hu, Lianfeng Li, Sara Cummings, and Hod Lipson.  They call their robot EVA:

The authors write:

Ability to generate intelligent and generalizable facial expressions is essential for building human-like social robots.  At present, progress in this field is hindered by the fact that each facial expression needs to be programmed by humans.  In order to adapt robot behavior in real time to different situations that arise when interacting with human subjects, robots need to be able to train themselves without requiring human labels, as well as make fast action decisions and generalize the acquired knowledge to diverse and new contexts.  We addressed this challenge by designing a physical animatronic robotic face with soft skin and by developing a vision-based self-supervised learning framework for facial mimicry.  Our algorithm does not require any knowledge of the robot's kinematic model, camera calibration or predefined expression set.  By decomposing the learning process into a generative model and an inverse model, our framework can be trained using a single motor dataset.

Now, let me say up front that I'm extremely impressed by the skill of the roboticists who tackled this project, and I can't even begin to understand how they managed it.  But the result falls, in my opinion, into the deepest part of the uncanny valley.  Take a look:

The tiny motors that control the movement of EVA's face are amazingly sophisticated, but the expressions they generate are just... off.  It's not the blue skin, for what it's worth.  It's something about the look in the eyes and the rest of the face being mismatched or out-of-sync.  As a result, EVA doesn't appear friendly to me.

To me, EVA looks like she's plotting something, like possibly the subjugation of humanity.

So as amazing as it is that we now have a robot who can mimic human expressions without those expressions being pre-programmed, we have a long way to go before we'll see an authentically human-looking artificial face.  It's a bit of a different angle on the Turing test, isn't it?  But instead of the interactions having to fool a human judge, here the appearance has to fool one.

And I wonder if that, in the long haul, might turn out to be even harder to do.

***********************************

Saber-toothed tigers.  Giant ground sloths.  Mastodons and woolly mammoths.  Enormous birds like the elephant bird and the moa.  North American camels, hippos, and rhinos.  Glyptodons, an armadillo relative as big as a Volkswagen Beetle with an enormous spiked club on the end of their tail.

What do they all have in common?  Besides being huge and cool?

They all went extinct, and all around the same time -- around 14,000 years ago.  Remnant populations persisted a while longer in some cases (there was a small herd of woolly mammoths on Wrangel Island in the Aleutians only four thousand years ago, for example), but these animals went from being the major fauna of North America, South America, Eurasia, and Australia to being completely gone in an astonishingly short time.

What caused their demise?

This week's Skeptophilia book of the week is The End of the Megafauna: The Fate of the World's Hugest, Fiercest, and Strangest Animals, by Ross MacPhee, which considers the question, and looks at various scenarios -- human overhunting, introduced disease, climatic shifts, catastrophes like meteor strikes or nearby supernova explosions.  Seeing how fast things can change is sobering, especially given that we are currently in the Sixth Great Extinction -- a recent paper said that current extinction rates are about the same as they were during the height of the Cretaceous-Tertiary Extinction 66 million years ago, which wiped out all the non-avian dinosaurs and a great many other species at the same time.  

Along the way we get to see beautiful depictions of these bizarre animals by artist Peter Schouten, giving us a glimpse of what this continent's wildlife would have looked like only fifteen thousand years ago.  It's a fascinating glimpse into a lost world, and an object lesson to the people currently creating our global environmental policy -- we're no more immune to the consequences of environmental devastation as the ground sloths and glyptodons were.

[Note: if you purchase this book using the image/link below, part of the proceeds goes to support Skeptophilia!] 

Uncanniness in the brain

When The Polar Express hit the theaters in 2004, it had a rather unexpected effect on some young movie-goers.

The train conductor, who (like several other characters) was voiced by Tom Hanks and was supposed to be viewed in a positive light, freaked a lot of kids right the hell out.  It was difficult for them to say exactly why.  He was "creepy" and "sinister" and "scary" -- even though nothing he explicitly did was any of those things.

The best guess we have about why people had this reaction is a phenomenon first described in the 1970s by Japanese robotics professor Masahiro Mori.  Called the uncanny valley, Mori's discovery came out of studies of people's responses to human-like robots (studies that were later repeated with CGI figures like the conductor in Express).  What Mori (and others) found was that faces intended to represent humans but in fact very dissimilar to an actual human face -- think, for example, of Dora the Explorer -- are perceived positively.  Take Dora's face and make it more human-like, and the positive response continues to rise -- for a while.  When you get close to a real human face, people's reactions take a sudden nosedive.  Eventually, of course, when you arrive at an actual face, it's again perceived positively.

That dip in the middle, with faces that are almost human but not quite human enough, is what Mori called "the uncanny valley."

The explanation many psychologists give is that a face being very human-like but having something non-human about the expression can be a sign of psychopathy -- the emotionless, "mask-like" demeanor of true psychopaths has been well documented.  (This probably also explains the antipathy many people have to clowns.)  In the case of the unfortunate train conductor, in 2004 CGI was well-enough developed to give him almost human facial features, expressions, and movements, but still just a half a bubble off from those of a real human face, and that was enough to land him squarely in the uncanny valley -- and to seriously freak out a lot of young movie-goers.

This all comes up because of a study that appeared this week in The Journal of Neuroscience, by Fabian Grabenhorst (Cambridge University) and Astrid Rosenthal-von der Pütten (University of Aachen), called "Neural Mechanisms for Accepting and Rejecting Artificial Social Partners in the Uncanny Valley."  And what the researchers have done is to identify the neural underpinning of our perception of the uncanny valley -- and to narrow it down to one spot in the brain, the ventromedial prefrontal cortex, which is part of our facial-recognition module.  Confronted with a face that shows something amiss, the VMPFC then triggers a reaction in the amygdala, the brain's center of fear, anxiety, perception of danger, and avoidance.

The authors write:

Using functional MRI, we investigated neural activity when subjects evaluated artificial agents and made decisions about them.  Across two experimental tasks, the ventromedial prefrontal cortex (VMPFC) encoded an explicit representation of subjects' UV reactions.  Specifically, VMPFC signaled the subjective likability of artificial agents as a nonlinear function of human-likeness, with selective low likability for highly humanlike agents.  In exploratory across-subject analyses, these effects explained individual differences in psychophysical evaluations and preference choices...  A distinct amygdala signal predicted rejection of artificial agents.  Our data suggest that human reactions toward artificial agents are governed by a neural mechanism that generates a selective, nonlinear valuation in response to a specific feature combination (human-likeness in nonhuman agents).  Thus, a basic principle known from sensory coding—neural feature selectivity from linear-nonlinear transformation—may also underlie human responses to artificial social partners.

The coolest part of this is that what once was simply a qualitative observation of human behavior can now be shown to have an observable and quantifiable neurological cause.

"It is useful to understand where this repulsive effect may be generated and take into account that part of the future users might dislike very humanlike robots," said Astrid Rosenthal-von der Pütten, who co-authored the study, in an interview in Inverse.  "To me that underlines that there is no ‘one robot that fits all users’ because some users might actually like robots that give other people goosebumps or chills."

This puts me in mind of Data from Star Trek: The Next Generation.  He never struck me as creepy -- although to be fair, he was being played by an actual human, so he could only go so far in appearing as an artificial life form using makeup and mannerisms.  It must be said that I did have a bit more of a shuddery reaction to Data's daughter Lal in the episode "The Offspring," probably because the actress who played her (Hallie Todd) was so insanely good at making her movements and expressions jerky and machine-like.  (I have to admit to bawling at the end of the episode, though.  You'd have to have a heart of stone not to.)

So we've taken a further step in elucidating the neurological basis of some of our most basic responses.  All of which goes back to what my friend Rita Calvo, professor emeritus of human genetics at Cornell University, said to me years ago: "If I was going into science now, I would go into neurophysiology.  We're at the same point in our understanding of the brain now that we were in our understanding of the gene in 1910 -- we knew genes existed, we had some guesses about how they worked and their connection to macroscopic features, and that was about all.  The twentieth century was the century of the gene; the twenty-first will be the century of the brain."

*********************************

This week's Skeptophilia book recommendation is about a subject near and dear to me: sleep.

I say this not only because I like to sleep, but for two other reasons; being a chronic insomniac, I usually don't get enough sleep, and being an aficionado of neuroscience, I've always been fascinated by the role of sleep and dreaming in mental health.  And for the most up-to-date analysis of what we know about this ubiquitous activity -- found in just about every animal studied -- go no further than Matthew Walker's brilliant book Why We Sleep: Unlocking the Power of Sleep and Dreams.

Walker, who is a professor of neuroscience at the University of California - Berkeley, tells us about what we've found out, and what we still have to learn, about the sleep cycle, and (more alarmingly) the toll that sleep deprivation is taking on our culture.  It's an eye-opening read (pun intended) -- and should be required reading for anyone interested in the intricacies of our brain and behavior.

[Note: if you purchase this book using the image/link below, part of the proceeds goes to support Skeptophilia!]

The valley of the shadow of uncanniness

Today in the news is a story about the creation of a robot named "Kaspar" at the University of Hertfordshire, whose purpose is to help autistic children relate to people better.

Kaspar is programmed not only to respond to speech, but to react when hugged or hurt.  He is capable of demonstrating a number of facial expressions, helping autistic individuals learn to connect expressions with emotions in others.  The program has tremendous potential, says Dr. Abigael San, a London clinical psychologist and spokesperson for the British Psychological Society.  "Autistic children like things that are made up of different parts, like a robot," she said, "so they may process what the robot does more easily than a real person."

I think this is awesome -- autism is a tremendously difficult disorder to deal with, much less to treat, and conventional therapies can take years and result in highly varied outcomes.  Anything that is developed to help streamline the treatment process is all to the good.

I am equally intrigued, however, by my reaction to photographs of Kaspar.  (You can see a photograph here.) 

On looking at the picture, I had to suppress a shudder.  Kaspar, to me, looks creepy, and I don't think it's just associations with dolls like Chucky that made me react that way.  To me, Kaspar lies squarely in the Uncanny Valley.

The concept of the Uncanny Valley was first formalized by Japanese roboticist Masahiro Mori in 1970, and it has to do with our reaction to non-human faces.  A toy, doll, or robot with a very inhuman face is considered somewhere in the middle on the creepiness scale (think of the Transformers, the Iron Giant, or Sonny in I, Robot).  As its features become more human, it generally becomes less creepy looking -- think of a stuffed toy, or a well-made doll.  Then, at some point, there's a spike on the creepiness axis -- it's just too close to being like a human for comfort, but not close enough to be actually human -- and we tend to rank those faces as scarier than the purely non-human ones.  This is the "Uncanny Valley."

This concept has been used to explain why a lot of people had visceral negative reactions to the protagonists in the movies The Polar Express and Beowulf.  There was something a little too still, a little too unnatural, a little too much like something nonhuman pretending to be human, about the CGI faces of the characters.  The character Data in Star Trek: The Next Generation, however, seems to be on the uphill side of the Uncanny Valley; since he was played by a human actor, he had enough human-like characteristics that his android features were intriguing rather than disturbing.

It is an open question as to why the Uncanny Valley exists.  It's been explained through mechanisms of mate selection (we are programmed to find attractive faces that respond in a thoroughly normal, human way, and to be repelled by human-like faces which do not, because normal responses are a sign of genetic soundness), fear of death or disease (the face of a corpse resides somewhere in the Uncanny Valley, as do the faces of individuals with some mental and physical disorders), or a simple violation of what it means to be human.  A robot that is too close but not close enough to mimicking human behavior gets caught both ways -- it seems not to be a machine trying to appear human, but a human with abnormal appearance and reactions.

Don't get me wrong; I'm thrilled that Kaspar has been created.  And given that a hallmark of autism is the inability to make judgments about body and facial language, I doubt an Uncanny Valley exists for autistic kids (or, perhaps, it is configured differently -- I don't think the question has been researched).  But in most people, facial recognition is a very fundamental thing.  It's hard-wired into our brains, at a very young age -- one of the first things a newborn baby does is fix onto its mother's face.  We're extraordinarily good at recognizing faces, and face-like patterns (thus the phenomenon of pareidolia, or the detection of faces in wood grain, clouds, and grilled cheese sandwiches, about which I have blogged before).

It's just that the faces need to be either very much like human faces, or sufficiently far away, or they result in a strong aversive reaction.  All of which makes me wonder who first came up with the concept of "clown."