The Emotions of Kinect

The Kinect is officially a hit. Microsoft is selling the video game accessory as quickly as it can produce them, and hopes to sell five million shiny motion-sensors before Christmas. The conceit of the Kinect is simple: there are no buttons or controllers. Instead, the plastic cyclops tracks the movement of your body, seamlessly weaving your flesh into the game. When you leap, the avatar on the screen leaps. When you swing or kick or punch, so does the imaginary character inside the TV. In The New York Times, Seth Schiesel wonders how this new gaming technology will impact the design of video games:

About a year ago I found myself trying to figure out how video games relate to traditional, non-interactive forms of artistic expression. I was struggling to articulate a sense that playing a game could be similar to consuming a painting, the symphony or the ballet.

And then a friend gave me a copy of “Art as Experience,” John Dewey’s seminal 1934 work on the philosophy of art, and it became clear: art exists not as artifact but as we engage with it. As Dewey puts it: “The product of art — temple, painting, statue, poem — is not the work of art. The work takes place when a human being cooperates with the product so that the outcome is an experience that is enjoyed because of its liberating and ordered properties.”

With no media do humans cooperate so intimately as video games. This is precisely why games have been the most popular new mass entertainment of recent decades. And this is also why the emergence of more physically natural and socially meaningful ways of enjoying games is so rich with creative possibility.

The Dewey reference is apt. While I'm not sure the pragmatist philosopher would think much of most video games - he'd probably see them as an isolating force, keeping us away from Nature and each other - I do think the Kinect would please another founding father of pragmatism: William James.

To understand why James would have bought a Kinect from his local Best Buy, it's worth revisiting his classic 1884 article "What is an emotion?" In this controversial paper, James argued that all of our mental feelings actually begin in the body. Although our emotions feel metaphysical, they are rooted in the movements of our muscles and the palpitations of our flesh. Typical of his work, James' evidence consisted of vivid examples stolen straight from real life, such as a person encountering a bear in the woods.

"What kind of an emotion of fear would be left," James wondered, "if the feeling of quickened heart beats nor of shallow breathing, neither of trembling lips nor of weakened limbs, neither of goose bumps nor of visceral stirrings, were present?" James' answer was simple: without the body there would be no fear, for an emotion begins as the perception of a bodily change. When it comes to the drama of feelings, our flesh is the main stage.

For most of the 20th century, James' theory of bodily emotions was ignored. It just seemed too implausible. But in the early 1980s, the neuroscientist Antonio Damasio realized that James was mostly right: Many of our emotions are preceded by changes in our physical body. Damasio came to this conclusion after studying neurological patients who, after suffering damage in their orbitofrontal cortex or somatosensory cortex, were unable to experience any emotion at all. Why not? The tight connection between the mind and body had been broken. Even though these patients could still feel their flesh - they weren't paraplegic - they could no longer use their body to generate feelings. And if you can't produce the bodily symptoms of an emotion - the swelling tear ducts of sadness, or the elevated heart rate of fear - then you can't feel the emotion. As Damasio notes, "The mind is embodied, not just embrained."

How might such a neurological process unfold? Let's say we are playing a shooter on the Kinect. Unlike other game consoles, which leave us stranded on the couch, this console (like the Wii before it) actually makes us move. If we want to kill off the bad guys, we need to run around and break a sweat. We are no longer just twiddling our thumbs.

In order to prepare for all this combat, the brain automatically triggers a wave of changes in our "physical viscera," such as quickening the pulse, flooding the bloodstream with adrenaline, and contracting our intestines. While even stationary entertainment can lead to corporeal changes - that's why the heart rate quickens when watching a Hitchcock movie - the physical activity of the Kinect exaggerates these effects. Although we might look a little foolish flailing around the living room, the game has managed to excite our flesh, and that means our emotions aren't far behind. As a result, we are more scared by the possibility of virtual death (and more thrilled by the virtual victory) because our body is fully engaged with the game.

For decades, video game designers have been obsessed with visual realism, as if the eyeball was the key to our emotional brain. But accurate graphics have diminishing returns. At a certain point, we don't need more pixels - we need more physicality. And that's what's so exciting about the Kinect (and the Wii before that): For the first time, video games are able to deliver a visceral emotional experience, as our body is tricked into confusing fiction with reality.

PS. Just to preempt the nasty comments: Yes, yes, I know the Kinect is a deeply imperfect console, with all sorts of annoying limitations. What interests me are larger implications of involving the body in the game.