Dylan Patel: belief

Transcript context

…Can you educate me a little bit about the speed of things? So the speed of memory versus the speed of interconnect versus the speed of fiber between data centers. Are these orders of magnitude different? Can we at some point converge towards a place where it all just feels like one computer? No, I don’t think that’s possible. It’s only going to get harder to program, not easier. It’s only going to get more difficult and complicated and more layers. The general image that people like to have is this hierarchy of memory, so on-chip is really close, localized within the chip, you have registers and those are shared between some compute elements and then you’ll have caches which are shared between more compute elements. Then you have memory like HBM or DRAM like DDRR memory or whatever it is, and that’s shared between the whole chip. And then you can have pools of memory that are shared between many chips and then storage and you keep zoning out. The access latency across data centers, within the data center within a chip is different. So you’re always going to have different programming paradigms for this. It’s not going to be easy. Programming this stuff is going to be hard, maybe AI can help with programming this. But the way to think about it is that there is sort of the more elements you add to a task, you don’t get strong scaling. If I double the number of chips, I don’t get two exit performance. This is just a reality of computing because there’s inefficiencies.And there’s a lot of interesting work being done to make it not to make it more linear, whether it’s making the chips more networked together more tightly or cool programming models or cool algorithmic things that you can do on the model side. DeepSeek did some of these really cool innovations because they were limited on interconnect, but they still needed to parallelize. Everyone’s always doing stuff. Google’s got a bunch of work and everyone’s got a bunch of work about this. That stuff is super exciting on the model and workload and innovation side. Hardware, solid-state transformers are interesting. For the power side, all sorts of stuff on batteries and there’s all sorts of stuff on. I think if you look at every layer of the compute stack, whether it goes from lithography and etch all the way to fabrication, to optics, to networking, to power, to transformers, to cooling, to a networking, and you just go on up and up and up and up the stack, even air conditioners for data centers are innovating. Copper cables are innovating. You wouldn’t think it, but copper cables, there’s some innovations happening there with the density of how you can pack them and it’s like all of these layers of the stack, all the way up to the models, human progress is at a pace that’s never been seen before. I’m just imagining you sitting back in a layer somewhere with screens everywhere, just monitoring the supply chain where all these clusters, all the information you’re gathering, you’re incredible.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.