Tim Sweeney: belief

Transcript context

…Humans are by far the hardest part of computer graphics because millions of years of evolution have given us dedicated brain systems to detect patterns in faces and infer emotions and intent because cavemen had to, when they see a stranger, determine whether they were likely friendly or they might be trying to kill them. And so people in the world have extraordinarily detailed expectations of a face and we can notice imperfections, especially perfections arising from computer graphics limitations. Okay, one part is capturing humans and so [inaudible 00:00:33] really advanced, dedicated hardware that puts a human in a capture sphere with dozens of cameras in them taking high resolution, high frame rate video of them as they go through a range of motions. And then capturing the human face is complicated because the nuanced detail of our faces and how all the muscles and sinews and fat work together to give us different expressions. So it’s not only about the shape of a person’s face, but it’s also about the entire range of motion that they might go through. So that’s the data problem. There’s a lot of other problems with computer graphics. There’s technology for rendering hair, which is really hard. Because you can’t render every… Again, we know the laws of physics. It would be easy to just render every hair. It would just be a billion times too slow. So you need approximations that capture the net effect of hair on rendering and on pixels without calculating every single interaction of every light with every strand of hair. That’s one part of it. There’s detailed features for different parts of faces. There’s subsurface scattering because we think of humans as opaque, but really our skin, light travels through it. It’s not completely opaque, and the way in which light travels through skin has a huge impact on our appearance. And this is why there’s no way you can paint a mannequin to look realistic for a human. It’s just a solid surface and we’ll never have the sort of detail you see. That kind of blew my mind, thinking through that. I think I heard that sort of the oiliness of the skin creates very specific, nuanced, complex reflections and then some light is absorbed and travels through the skin and that creates textures that our human eye is able to perceive and it creates the thing that we consider human, whatever that is. All of that, while considering all the muscles involved in making the nuanced expression, just the subtle squinting of the eyes or the subtle formation of a smile, it’s the subtlety of human faces that you have to capture, like the difference between a real smile and a fake smile, but the way to show beginning of a formation of a smile that actually reveals a deep sadness, all of that, when I watch a human face, I can read that. I could see that you have to have the tools that, in real time, can render something like that, and that’s incredibly difficult.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.