Tim Sweeney: prediction

Transcript context

…Do you think it will be possible to generate, so use generative AI to create dynamic objects, like you mentioned trees, in the Unreal Engine world to create meshes and textures and empower the creator to create faster, use meta knobs like hyperparameters versus very nuanced, where you can control much faster the look of a face, the look of a tree, all that kind of stuff? I think that’s the central challenge of the next decade of game engines and AI for content creation of all sorts, because you have two very different models to the world that are emerging. There’s the scene graph, the technical term we use to describe the set of all of the objects in the world in a 3D world maintained by Unreal Engine or another engine. In the videos you saw, it’s the rocks and the trees and the snow and the bridge and the people and all of these things. Each one has enormous amounts of data attached to it. Some are like texture maps, some are sound files, some are animation files and enormous amounts of detail all stored there in that procedural and in this precise computer graphics representation that enables rendering it from any perspective with any settings and so on. It’s a completely general system that has complete context about the state of the world at any point. You can always precisely reproduce it. If you play the same scene 10 times in a row, it’s always the same. It’s never randomly changing. You’re like, “Oh no, why did this character’s face change midstream?” But it’s also rather limited because you have to build everything manually. That’s costly and it’s time-consuming. It requires expertise. Then you have this other model of the world, which is what AI sees or thinks. If we could peer into what’s really happening in its parameters, there’s something like the mushy connections of neurons in a brain. It has a vast amount of knowledge about the world and about graphics and about images and about people and about everything else. It’s stored in a human incomprehensible form, but it can be extracted through queries like asking it to produce an image from a prompt or a video from a prompt or whatever. But the huge problem with that is it’s very mushy data. We don’t know how to give it a command that will give us a precise result. If it produces one image one time and we change our prompt slightly, it might produce something completely different. We are unable to art-direct it. It’s this completely untamed tool. I think when we figure out more and more ways to merge these and connect these two together, you can imagine AI enhancing the process of content creation in a traditional scene representation. You can imagine the scene representation being shared with the AI so the AI not only sees a prompt, but also here’s a list of all of the objects in the world and the characteristics and so on. It can learn more about how those objects should move and interact. If you get a constant feedback cycle going back and forth between an engine and AI, then I think you can get the best of both worlds, stable scenes, but also the higher productivity of being able to get content out and the ability to select specific parts of it and art-direct those and to have those art-direction stick and be recognized as part of this permanent scene representation. I can’t wait until AI can operate not in the space of pixels, but in the space of scene graphs, creating objects in the scene graph, whether it’s like you mentioned, audio or any of the things that you mentioned about that empower the creator. That’s a super exciting future. I wonder if you could speak to a fear that people have on this topic of artists, engineers are losing their jobs, being replaced by AI. Are there words of hope that you could offer them?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.