Affective Computing: From Laughter to IEEE

Rosalind W. Picard
This is an invited introduction to the first issue of the IEEE Transactions on Affective Computing, telling personal stories and sharing viewpoints of a pioneer and visionary of the field of Affective Computing. This article is not intended to be a thorough or a historical account of the development of the field, for the author is not a historian and cannot begin to properly credit the extraordinary efforts of hundreds of people who helped to cultivate and expand the rich and fertile landscape that extends before us now.

JODIE is a young woman I am talking with at a fascinating annual retreat organized by autistic people for autistic people and their friends. Like most people on the autism spectrum (and many neurotypicals, a term for people who don’t have a diagnosed developmental disorder), she struggles with stress when unpredictable things happen. Tonight we are looking at what happened to her emotional arousal as measured by a wristband that gathers three signals—skin conductance, motion, and temperature (Fig. 1). Jodie says she was upset to learn that the event she was supposed to speak at was delayed from 8:00 to 8:30 pm. She started pacing until her friend told her that was not helping and to stop. Many people don’t have an accurate read on what they are feeling (this is known as alexithymia) and while she thought pacing helped, she wasn’t certain, so she took his advice. She then started to make repetitive movements often seen in autism, commonly called “stimming,” and continued these until the event began at 8:30. In Fig. 1, we see her skin conductance on the top graph, going down when she was pacing, up when she was stimming, and hitting its highest peaks while she gives her presentation. The level also stays high afterward during other people’s presentations, when she stayed up front to handle problems everyone was having with the audio-visual technology.

Collecting data related to emotional arousal is not new: For example, skin conductance has been studied for more than a hundred years. What is new, however, is how technology can now measure, communicate, adapt to, be adapted by, and transform emotion. Powerful new things can be done with these abilities. For example, Jodie collected her emotional arousal data wearing a stretchy wristband, clicked to upload it into a mobile viewer, let her friend (who asked her to stop pacing) see the data, and the first words out of his mouth were, “I’m not going to tell you to stop pacing anymore.” The next morning I saw her pacing without his interference. The ability to communicate objective data related to her emotional arousal and activity—specifically her sympathetic nervous system activation, of which skin conductance is a sensitive measure, prompted a change in his behavior. Mind you, she had told him in the moment of stress that she thought pacing was helping, but this did not change his behavior. Information about emotions that is objective carries much more power than self-reported subjective feelings.

The convenience of a new affective computing technology can lead to new self-understanding, to improved communication between people, and to much more, including (if researchers make it so) to new technologies that reduce stress instead of increasing it. There’s a saying “if you can’t measure it you can’t manage it.” Measuring the frustration caused by a technology when it happens can enable engineers to pinpoint what causes it and work to prevent or reduce it. Technology can also be improved if it has an intelligent ability to respond to emotion, and technology can be improved by virtue of incorporating principles of emotion learned from biological systems. But there are many extraordinarily hard challenges to solve in order to bring about new benefits.

Attitudes toward affective computing, which I defined in 1995 as “computing that relates to, arises from, and deliberately influences emotion,” have changed so much in the last decade that it is now hard for some people to believe it used to be a ludicrous idea. In the 1990s, I had never heard of the shorthand “LOL” (Laugh out Loud), but it applied to this research. I beg the reader to let me indulge in some remembrances, starting in 1991, my first year on the MIT faculty.

Read full text