This article was taken from the January issue of Wired UK magazine. Be the first to read Wired's articles in print before they're posted online, and get your hands on loads of additional content by subscribing online
See also: Review - Avatar delivers in three spectacular dimensions
James Cameron is striding across a vast soundstage in Playa Vista, an ocean-side district of Los Angeles. This enormous, near-windowless building used to be part of Hughes Aircraft, turning out parts for fighter planes during World War II. Next door is the hangar where Howard Hughes built his "Spruce Goose", the gargantuan flying boat that took so long to construct that the war was over before the craft made its one and only flight. Now it's March 2008, ten years almost to the day since the Titanic director took the stage at the Oscars, shook a gold statuette over his head, and notoriously declared himself you-know-what. Like Hughes, Cameron has been toiling away for years on an epic project some feared would never reach completion: a $250 million spectacle called Avatar. Even by Hollywood standards,
Avatar is film-making on a colossal scale. A sci-fi fantasy about a paraplegic ex- Marine who goes on a virtual quest to another planet, it merges performance-capture (a souped-up version of motion-capture) with live action shot in 3D using cameras invented by Cameron. If all goes according to plan, on December 17 Avatar will dissolve the boundary between audience and screen, reality and illusion - and change the way we watch movies forever. "Every film Jim has made has soared past the envelope into areas nobody even imagined," says Jim Gianopulos, co-chair of Fox, the studio behind both Titanic and
Avatar. "It's not enough for him to tell a story that has never been told. He has to show it in a way that has never been seen."
Today, Cameron and his crew are prepping the soundstage to record performance-capture data for a scene with the film's two stars, Sam Worthington (last seen in Terminator Salvation) and Zoe Saldana (Uhura in JJ Abrams' Star Trek, and featured in last month's Wired). "Come on over here," he calls out with a wave. "I'll show you what this is."Wearing worn jeans and a New Zealand Stunt Guild T-shirt, Cameron holds up a small flat-panel screen tricked out with multiple handles and knobs. This is his virtual camera, he explains, a custom-designed viewing system that enables him to see not what's in front of him (a darkened soundstage) but the lush, computer-generated world that will appear in the film.
A few metres away, Worthington, who plays soldier-turned-humanoid- avatar Jake Sully, and Saldana, his alien love interest Neytiri, wear black body-suits dotted with roughly 80 metallic spots. Infrared cameras are strung across the ceiling to track these reflective markers, capturing the movements of the actors' bodies. These same cameras register markers on the frame of Cameron's screen as the director moves it about.
Looking into his display, Cameron doesn't see Worthington and Saldana on a soundstage. Instead, he sees Sully and Neytiri, each ten feet tall with blue skin, catlike features, and long tails. The background is not grey plywood risers but the deep rainforest of the planet Pandora, where most of this movie takes place. Cameron can view in real time what other directors have to wait months to see.
But that's not all he can do. The director calls up a scene recorded yesterday, and suddenly the image switches to a fan lizard, a Cameron-invented flying creature unique to Pandora. As the reptile comes to pixelated life, Cameron begins to gyre across the soundstage, tilting the screen. "I can zoom out a little bit. I can followit around the landscape as he's flying. Come underneath, come up on top - see how it flies." It's as if he were shooting with an actual camera - exactly the point. "I can't operate a camera with a fucking mouse," he says. "It's ridiculous. It's why CG camera movements look computer-generated."
In the scene that Cameron is about to shoot, Sully and Neytiri are leaping through the jungle chasing fan lizards. The script calls for the two of them to stare in wonderment at the reptiles, represented on the set by some dots on the ends of half-a-dozen skinny wooden poles that crew members are waving about. Setting aside his virtual camera, Cameron grabs a stick and joins the fun.
While Cameron and the actors cavort around the set, technicians sit at the far end of the room, pulling the digital strings. This is Avatar's "brain bar" - three tiers of computer monitors manned by 20 or so people: video specialists on the first level, mo-cap technicians from Giant Studios and real-time CG operators from Cameron's Lightstorm Entertainment on the second, and visual-effects supervisors and facial-capture experts from Peter Jackson's Weta Digital up top.
On the monitors, the brain-bar crew is watching various iterations of the fan lizard scene as it's being captured. One screen maps skeletal data points on a grid. Others toggle from Worthington and Saldana on the risers to their animated characters in the jungle. The brain bar was built to accommodate
Avatar's digital production pipeline. Several months before, Cameron wrapped the bulk of the film's hi-def 3D shots in New Zealand, most of them against a green screen. Now, thanks to hardware called a simulcam, the production team can mix performance-capture from Playa Vista, live-action footage from New Zealand, and CG backdrops to conjure up a real-time composite. It compresses into seconds a job that would normally take weeks. On the fly, the brain bar can transmute hundreds of gigabytes of raw data into a rough approximation of the fantastic universe that, for the past 15 years, has existed only in Cameron's mind. "You can call it what you want," says Glenn Derry, Avatar's virtual-production supervisor. "But it's just a big database."
For Cameron, the brain bar is the ultimate control mechanism. "It started with Jim going, 'What if I could do this?'" says Richie Baneham, the production's animation director. But beyond that, there was no master plan. "It just kind of evolved," Derry says. "We kept adding features and adding features, and then we got to the point where we couldn't keep track of them all. Now we bring in people from outside and they go, 'Whoa!'" The foundation of Derry's virtual-production system is Motion- Builder, a 3D character-animation program that's been used on everything from
Rock Band to Beowulf. Giant Studios was brought in for its real-time "solve" - that is, the ability to interpret performance-capture data in real time. They ended up with a highly evolved form of previsualisation, a computer-animation technique designed to show what a scene might look like with live-action sequences and CGI meshed together.
When Derry started on Avatar in 2005, Steven Spielberg and Industrial Light & Magic were pushing the limits on previsualisation for War of the Worlds. As Spielberg was shooting an action scene, he knew where the alien war machine would go when ILM rendered it months later. But he couldn't actually see the tripod crashing into cars and buildings during his shoot - which is what you'd get with Cameron's setup.
Meanwhile, Baneham had to figure out how to overcome the "uncanny valley", that unnerving gulf between human and not-quite- human that made the performance-capture figures in Robert Zemeckis's Beowulf and Polar Express feel creepy.
Zemeckis, Cameron says, "used the same marker system we use for the body and captured the facial performance with those mo-cap cameras.
Really a bonehead idea." Baneham's solution required the production crew to make a plaster cast of each performer's head, then build a custom harness to hold a tiny videocam that would be aimed back at the actor's face from just a few centimetres away. The wide-angle lenses would record every twitch, every blink, every frown.
But it would take more than that to make Avatar look real. Before he joined this production, Baneham led the effort to bring Gollum to life in The Lord of the Rings. At that time, when digital animators wanted to render a smile they would just dial the mouth a little wider, which invariably looked fake.
So Baneham and his team studied physiology: what muscles fire to produce a smile, and in what order? How much light does the skin absorb, and how much does it reflect? How deep is the pupil, the opening at the centre of the eye? The facial details that made Gollum so convincing were not provided by performance-capture but by painstaking, frame-by-frame CG work. Avatar's system streamlines that process: animators can work from the head-rig videos, which supply a complete visual record they can map to each character's face.
Even as Derry was reinventing virtual production and Baneham was rethinking facial animation, Cameron was throwing out the rules of shooting 3D. With camera designer Vincent Pace, he developed a system with twin lenses that could mimic human vision. Wearing polarised glasses, viewers would get the same 3D effect they do with CG films like last summer's Monsters vs Aliens or the forthcoming Toy Story 3. But shooting live-action 3D is more complicated than rendering computer animation in 3D.
The conventional method relies on a series of cumbersome mathematical formulae designed to preserve the "screen plane" - the surface on which the movie appears. In 2D that's the screen itself, but in 3D it's an imaginary point somewhere in front of you." The viewer doesn't think there's a screen plane," Cameron explains. "There's only a perceptual window, and that perceptual window exists somewhere around arm's length from us. That's why I say everything that's ever been written about stereography is completely wrong."
To Cameron, eliminating the screen plane is crucial. "The screen plane has always been this subconscious barrier," says Jon Landau, his long-time producer. By removing it, Cameron hopes to create an all-encompassing cinematic experience. That's the reason for the virtual camera and the simulcam, the hi-def head rigs, the 3D camera system: total immersion. "This is not just a movie," Landau says. "It's a world. The film industry has not created an original universe since Star Wars. When one comes along so seldom, you want to realise it to the fullest possible extent."
That's one reason to do all this - the other reason is more personal. "I've made a bunch of movies, won a bunch of awards, made a bunch of money," Cameron declares. "None of those are interesting to me. They never were. It was never about the awards, never about the money, only about the films. But what's challenging? I look for the new thing. And with this film, we've really loaded it up."
Want more Wired UK magazine? Get your copy every month by subscribing online today
This article was originally published by WIRED UK
