The promise of virtual reality is being able to leave the ordinary behind and travel to fantastic new worlds. Some researchers at Microsoft are taking the opposite approach: They want to bring the fantasy to your living room.
The new project, RoomAlive, uses projectors and Kinect cameras to turn your den into an interactive gaming environment. Your floor becomes staging grounds for robot battle. Your walls get adorned with booby traps. Whack-a-mole means giant mutant moles actually burrowing into your house.
The project builds on a previous Microsoft effort called IllumiRoom, which used projection mapping to let games seep out beyond TV sets onto the walls behind them. RoomAlive makes the concept even more immersive. The nine-person team behind it includes a pair of designers hailing from Disney's research lab, a virtual reality game designer from USC, and a handful of other experts in projection mapping and augmented reality.
The RoomAlive rig took four months to create. It relies on six ceiling-mounted projector-Kinect pairs. On the technical side, one of the big challenges was perspective. Eventually, the group figured out how to use the Kinect cameras to track the player's head in space, rendering game elements dynamically so that they look right next to the physical objects in the room.
The most challenging aspect of RoomAlive, however, was figuring out how to design for it. "As a game designer, you can’t just create your level from scratch," says Raj Sodhi, one of the researchers behind the project. "You have to create your gaming experience around the user’s living room, and not just their living room, but every living room." To help designers, the researchers created tools that automatically detect basic spatial elements, like walls and floors, and tag other physical surfaces as "horizontal" or "vertical" to help sort out what objects should be projected on them. "The designer can say, I want to create an enemy character that only appears on the floor, or appears behind a player when they aren’t looking," Sodhi says.
The RoomAlive team thinks the concept offers something fundamentally different from even today's most sophisticated motion-sensing games. "With most Kinect games, you stay in a small area between your couch and TV," Sodhi explains. "With RoomAlive, you really have to dart from one side of your room to the other, and you aren’t pretending to whack a mole on the screen, you are actually whacking a mole on your couch."
Another use case the team is excited about involves incorporating physical objects into game design. The researchers imagine something like a Dora the Explorer game where the system would track a kid's backpack around the room; once the youngster opens it, all sorts of virtual creatures and contents spill forth, courtesy of the projectors overhead.
The future of gaming probably won't involve outfitting your ceiling with a half-dozen projectors. But RoomAlive does encourage us to think about forms of entertainment that aren't limited to the confines of a flat-screen. Think about Joaquin Phoenix's gaming setup in the movie Her. Most of the time, his living room looks perfectly unremarkable, like a setting from a catalog for an up-scale furniture. But thanks to the magic of some next-gen projectors, it instantly transforms into a cave-like virtual environment, where bookshelves fade to the background and fantastic character moving freely around the room.
In both Her and RoomAlive, we see technology being used to merge our own world with some other virtual one. They're not about transporting us to fantastic lands so much as letting us meet them halfway.
