Tim Sweeney: observation

Transcript context

…It’s hard, yeah. Programming on a single-threaded computer is hard enough, but it is completely predictable. If you have a language that’s deterministic and you’re on the same code, over and over, it’s always going to do exactly the same thing and there’s no unpredictability about what might happen, right? You’re reading and writing variables in some order and you’re always going to see it behave the same. The problem is when you introduce multiple threads or multiple nodes in a data center, all working together on a single problem, is that they each want to read and write different pieces of data, and change of the state of the world as they go. And still almost all concurrency in real-world programs today is achieved manually. Programmers are writing this code that might run in multiple threads very, very carefully so that they are negotiating among each thread to get access to data in a way that’s going to give them predictable results. And it’s incredibly hard. It’s so hard that we’ve in five generations of Unreal Engine, every single generation decided we’re not going to try to scale up all of our gameplay code to multiple threads manually. It’s just much, much, much too likely to go wrong, not only for ourselves, but for every partner company who licenses Unreal Engine and tries to use it for building a game. It’s just a massive foot gun. There’s a variety of solutions to concurrency that are all rather suboptimal. One attempted solution was like, just don’t try to solve this problem at all. Let’s break our program down into microservices. And almost all online websites of massive scale like amazon.com work with hundreds of microservices where different servers negotiate with each other by sending messages to each other. And by programmers writing those things very carefully, they eventually get to being able to take your orders and not make a mess of them reliably. But this is totally not scalable to the metaverse where you have millions of programmers who are mostly not going to be computer scientists. They’re mostly going to be hobbyists, and enthusiasts, and first time programmers doing stuff for fun. That’s never going to work for them because they’ll never be able to envision all of the different dependencies between different computations they’re running in parallel. rammers doing stuff for fun. That’s never going to work for them because they’ll never be able to envision all of the different dependencies between different computations they’re running in parallel. But it turns out that there was an amazing foundational work done in 1980s that was made very real by a paper on Haskell concurrency. Composable Memory Transactions is the name of the paper. And it describes the system for transactional updates to programs. And the idea of a transaction is a transaction is a block of code that does a bunch of operations on memory, it might read, it might write, it might process an order, it might accept an order or reject an order. It might transfer money between one bank account and another. It might make conditional decisions like, “Oh, you asked to transfer a hundred dollars from your account to this guy’s account. We’re going to see if you have a hundred dollars. If you don’t, we’re going to reject it. And if you have a hundred dollars, we’re going to take a hundred dollars out of your account and add it to this other guy’s account.” Without transactions, if everybody’s just randomly adding and subtracting each other’s bank balances, then you might have somebody read a bank balance, subtract a hundred dollars and write it out. But in the meantime, somebody has written something else in the meantime. So you might get inconsistent bank balances arising if you don’t have a way of ensuring that these all run in a specific order. So the idea of transactions is its way of dividing an entire program into updates, self-contained updates that do an arbitrary amount of computation but must run in a single threaded manner. And in the case of a game engine, that’s a gameplay object update. When you’re playing Fortnite, you see a gameplay object. Every other player is a gameplay object. Every enemy is a gameplay object. Every rocket, and projectile, and car, and thing you see moving around and interacting, it’s not just a fixed static part of the world. That’s a separate game object. And each of those objects is updated at a rate of one update per frame at 60 frames per second. And so then in the course of Fortnite Battle Royale gameplay, you have tens of thousands of object updates happening every frame with a hundred players. In a simulation with billions of players, you’d have a whole lot more than that.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.