
Rana El Kaliouby shares her work creating an “emotion engine” that will help computers and devices understand how we are feeling — and react accordingly. Photo: Marla Aufmuth/TED
Most human beings can tell the difference between a smirk and a smile. Our computers and devices, however? Not so much.
Fifteen years ago, Rana el Kaliouby left her home in Egypt to study computer science at Cambridge. With her husband at home, she found herself homesick. “I realized I was spending more hours with my laptop than any other human,” she says. “Despite this intimacy, my laptop had no idea how I was feeling — whether I was happy or having a bad day.”
This gave her an idea. “Technology has lots of IQ, but not EQ,” she says. “What if our technology could sense our emotions? What if our devices could sense how we feel and react accordingly just the way an emotionally intelligent friend would?”
El Kaliouby started investigating this as a research project at MIT, then founded a company, Affectiva, to make the tech available to developers. She asks for a volunteer so she can show us how Affectiva’s “emotion engine” works. On an iPad, the engine senses the bounds of our volunteer’s face, and constellation of points appear at the corners of her eyes, the outline of her lips and around her nose. As our volunteer smiles, frowns and acts surprised, bars that measure joy, disgust, valence and engagement get longer and shorter, flipping between green (for positive emotions) and red (for negative emotions).
Affectiva, says el Kaliouby, has collected the world’s largest emotion database: 12 billion emotion data points collected from 2.9 million face videos from volunteers in 75 countries.
“It blows me away that we can now quantify something as personal as our emotions and that we can do it at scale,” she says. “So what have we learned?”
She walks us through a few key learnings.
- There are gender differences. Women tend to be more expressive than men; they smile more often and longer than men.
- There are cultural differences. While women in the US smile 40% more than men, the same difference wasn’t found in the UK.
- There are age differences. People who are 50 or older are 25% more emotive than younger people.
- We are expressive all the time. We show emotions “even when sitting in front of devices alone,” el Kaliouby says. “We express emotions when emailing, shopping online, even doing our taxes.”
So what are the implications of this? El Kaliouby imagines emotion-sensing glasses that could help those who are visually impaired or on the autism spectrum interpret other people’s emotions. She imagines teaching apps that could speed up when a student looks bored. She jokingly imagines a fridge that knows when its owner is stressed and autolocks to prevent emotional eating.
“When I was at Cambridge, what if I had access to a real-time emotion stream and could share that in a very real way with my family?” she asks.
“As more of our lives become digital, we are fighting a losing battle,” she continues. “What I’m trying to do instead is bring emotions into technology … We have a golden opportunity to reimagine how we connect with machines and therefore how we, as human beings, connect with one another.”
Comments (2)
colleen15327804 commented on Jun 21 2015
HI Rani,
I enjoyed your talk and wondered if you can comment on the findings/outcomes of your research AND your thoughts on how similar/dissimilar the human experiences with technology will be in the future based on the scenarios set in the 2013 film ‘Her’. (OS’s become able to read human emotions and become interacitve, more and more human enter ‘relationships’ with them).
I concur with your thoughts on there being many more positive applications for your research findings, rather than negative but particularly with children diagnosed with autism or Aspergers. As I work in this field and can see how beneficial it would be in a training tool, possible ipad based. …and an updated version of a polygraph test in criminal cases…??? …research with convicted criminals perhaps…???
The avenues are endless..
Well done!
Colleen
wwwpalfitness commented on May 28 2015
Interesting read