ILM’s Jeff White On THE AVENGERS, Fixing It In Post And The Shrinking FX Time Budget

Jeff White, Oscar nominated FX supervisor for THE AVENGERS, talks about making the Hulk and the realities of modern CGI as a tool and a crutch.

When the Oscar nominations came out nobody was surprised that ILM's team got a nod for their work on The Avengers, a big budget FX spectacle that used FX right. Between the digital recreation of New York, the great shots of the SHIELD Helicarrier, and the creation of a wonderful digital character with The Hulk, The Avengers boasts top-notch digital artistry.

Disney invited me over to the studio to talk with Jeff White, the visual effects supervisor on the film and one of the people who will be up on stage should the movie win. I think they wanted me to get all Oscar-y, but I kind of don't care about that stuff. I'd rather talk about the realities of visual FX in the modern world. I'm an agnostic when it comes to CGI - I believe it can be used well, but I also think it's overused. I think at its best, CGI is invisible, at its worst it's A Good Day To Die Hard. So that was what I focused on, and I was especially happy that Jeff seemed to like the questions about 'fixing it in post' and the ever-shrinking post-production schedule for these massive blockbusters.  

One of the things I liked in The Avengers is that the big battle is set in New York, and as a New Yorker I found the recreation didn't pull me out of the movie - it worked!

We certainly wanted it to get to the point where New Yorkers wouldn’t know it wasn’t shot there. There were a lot of practical reasons why they couldn’t have shot [in New York]. Closing the Park Avenue viaduct for three months wouldn’t have made New Yorkers happy. Then there was all the pyro. And flying Iron Man around - we couldn’t get helicopters that low to fly between buildings. I think there was an initial desire to shoot more in New York, but for practical reasons it had to be visual effects.

The nice thing is we had three days there. And on one of those days we had a car rig and drove around the areas we were creating digitally and shot it with the Alexa. We kept going back to it - what’s the exposure of the sky when you’re standing on the Park Avenue viaduct, stuff like that.

Maybe that’s the ultimate best use of CGI - stuff you don’t even notice.

It is. I think for Joss [Whedon] it was important this movie was set in the heart of Manhattan. Grand Central was a big part of that. Stark Tower being where the Met Life building is was a big part of that. We had, on several films leading up to it, developed the ability to create virtual environments. This stressed those abilities, because it was larger in scope, but it pushed us to take it farther.

There’s this thought that doing stuff digitally will make it cheaper. Is that true?

No. If you’re shooting in New York City you set your camera down and roll. There’s probably a lot of costs for getting your crew there and housing and permits and things. But if you’re doing it digitally, you’ve got to set your camera down, render your background, paint out all the stretching things, put back in the digital set dressing, replace every window with a digital window to get a proper reflection - it’s a lot of work to do it digitally. It’s much more about the fact that we had to shoot it that way.

I’m an old man, so I love stop motion effects.

I came up in stop motion - I worked at Will Vinton Studios! I started there straight out of college.

What’s interesting about that stuff is that it’s not photorealistic. It never aspired to that. Should photorealism be what we try to do with CGI, or is there another feeling of magic we should go for?

With stuff like New York it should be. You don’t want audiences to be bummed with guys on green screen - you can absolutely tell, and it takes you out of the movie. We want all of that to be photoreal. And the same with the Hulk, I think. He was so hard, there were so many problems to solve in terms of creating a photoreal human. We look at people every day, and you know right away when it doesn’t look right. It’s a steep ravine of not looking good, working on it, working on it, working on it, until it finally comes back to the other side of looking good.

How good does it have to look? When you look at the Hulk you know that’s not a guy in make-up, you know that it’s an effect. Where’s the line where it’s ‘good enough’?

What we wanted was to create a great character doing fun things, lots of great things Joss gave us to do. You want to see him rage out, to see him smash aliens and slam Loki back and forth. Those are such fun moments for us to work on in special effects. What we didn’t want was for you to go ‘Uch, that guy looks CGI.’ That disconnects you from the story and the dialogue and the action Joss set up. We don’t want it to get in the way.

Where did the smashing back and forth come from? Was that you guys or Joss?

No, that’s Joss. That was in there from the very first pre-vis. You look at it in pre-vis and wonder if it’s too cartoony, but he was absolutely right. Even when we were working on it we would laugh every time we saw it, even the fourth time. Then when I saw it in a theater with people that didn’t know it was coming, it was phenomenal. It’s so gratifying in visual effects to have a reaction like that to your work.

Why do you think the Hulk popped so much in this iteration for audiences? How much was the work you guys did and how much was Joss, how much was a marriage of the two?

It was a marriage of the two. Joss set up some great stuff for the Hulk, and casting Mark Ruffalo as Banner - people aren’t going to buy Hulk if they don’t buy Banner. Ruffalo was such a nice guy he was willing to do anything we needed to bring this character to life. I don’t know how much you read about our process, but we took life casts and dental molds and fingerprints and everything from him so we weren’t making it up. When you have to make stuff up in CG is when you have a hard time getting to the finish line. But when we start with something real, and keep matching to that it helps. When we were in there doing a close-up smile shot of Hulk’s face, we had a plate of Mark Ruffalo and studied that. “Oh look, there are spittle bubbles on the teeth!” Those little things are amazing to me.

The scene where he transforms right before he punches the Leviathan had to be done all CG, because it was shot normal speed but we needed it slomo. We did have a plate of Mark Ruffalo that we lined everything up to, and we spent all of our time on frame one, figuring out why it didn’t look like him, and it came down to stuff like angles of eyelashes.

There’s a phrase used a lot which is ‘We’ll fix it in post.’


How much does it piss off FX guys? Or do you like it?

We can fix so much stuff, but it’s great if everybody has a plan. What we can’t fix is a situation where you shot something on a sunny day but our background has the light coming from the other direction. As soon as you put those together, it looks terrible.

Do you see that happening a lot? Do you see directors just not paying enough attention on the day and bringing it to you to fix?

It happens sometimes. It’s not usually an intentional screw-up on their part, it’s just the realities of the production, trying to make the day. Sometimes it’s about trying to figure out what has to be done and what can be fixed later. In visual effects we can sometimes do things quickly that would be laborious and costly on set. That’s where we’re on set sitting next to the director, telling them what we can help with.

What advice would you give to a director who is making the leap from smaller movies to FX heavy movies, like Joss Whedon did?

That would be a great question for Joss, because he pulled it off amazingly well. He had directed Serenity, which was FX-laden, but not with the scope of this.

Decisiveness is huge. The thing is CG only looks good when you have time to iterate, and if you iterate in this direction, change your mind and decide to go a different way, you’re going to run out of time. You have to be decisive, and he was. Then you can spend your time iterating.

What I loved about working with Joss is that decisions weren’t arbitrary. They were thought out and everything had a reason. In this movie Iron Man needed something new and they put a jet pack on him - that wasn’t just because it would be cool, it’s because now he didn’t have to be doing what Joss called the Tinkerbell pose. Now all of a sudden the animators can get him in great cowboy poses, and he can be firing while flying.

He’s a great model for if you’re jumping into a big FX film. It’s a complicated, huge machine, and you have to surround yourself with good people. If you can be decisive, you can communicate what you need, you can do great work.

Kenneth Branagh said about working on Thor that the FX show up on your doorstep the morning of the premiere. Do you foresee a world where Hollywood figures it out and realizes the development time needs to be longer?

Once Pandora’s Box is open, they never go back. Once we accomplish Avengers on the post schedule we do, they’re never going to go to Avengers 2 and give us twice as much time. That requires us instead to innovate and figure out how we can roll more real-time technology into the pipeline. How can we use the GPUs so that we’re seeing 50% quality renders immediately and then waiting for the long renders, so that we’re not spending long cycles waiting for renders to come out.

It makes us keep innovating!