Andrej Karpathy: belief

Transcript context

…I’m curious to bounce two other ways in which the analogy might be different. The reason I’m especially curious about this is because the question of how fast AI is deployed, how valuable it is when it’s early on is potentially the most important question in the world right now. If you’re trying to model what the year 2030 looks like, this is the question you ought to have some understanding of. Another thing you might think is one, you have this latency requirement with self-driving. I have no idea what the actual models are, but I assume it’s like tens of millions of parameters or something, which is not the necessary constraint for knowledge work with LLMs. Maybe it might be with computer use and stuff. But the other big one is, maybe more importantly, on this capex question. Yes, there is additional cost to serving up an additional copy of a model, but the opex of a session is quite low and you can amortize the cost of AI into the training run itself, depending on how inference scaling goes and stuff. But it’s certainly not as much as building a whole new car to serve another instance of a model. So the economics of deploying more widely are much more favorable. I think that’s right. If you’re sticking to the realm of bits, bits are a million times easier than anything that touches the physical world. I definitely grant that. Bits are completely changeable, arbitrarily reshuffleable at a very rapid speed. You would expect a much faster adaptation also in the industry and so on. What was the first one? The latency requirements and its implications for model size?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.