Dwarkesh Patel: evaluation

Transcript context

…Yes. It might not be directly. It might be through Zeiss having so many suppliers and XYZ company having so many suppliers. If you just think about it, you’re talking about two physically moving objects that are the size of a wafer, and it has to be accurate to the level of single-digit nanometers or even smaller because the entire system, the overlay, the layer-to-layer overlay variation, has to be on the order of 3 nanometers. If the overlay is 3 nms, that means each individual part, the accuracy of its physical movement has to be even less than that. It has to be sub-one nanometer in most cases, because the error of these things stack up. There’s no way to just snap your fingers and increase production. Things as simple as power. The US going from zero percent power growth to two percent power growth, even though China’s already at thirty, was so hard for America to do. And that’s a really simple supply chain with very few people in it who make difficult things. There are probably 100,000 electricians and people who work in the electricity supply chain, or more, in the US? When you look at ASML, they employ so few people. Carl Zeiss probably employs less than a thousand people working on this, and all of those people are super, super specialized. You can’t just train random people up for this in the snap of a finger. You can’t just get your entire supply chain to get galvanized. Nvidia’s had to do a lot to get the entire supply chain to even deliver the capacity they’re going to make this year. When you go talk to Anthropic, they’re like, “We’re short of TPUs, we’re short of training, and we’re short of GPUs.” When you go talk to OpenAI, they’re like, “We’re short of these things.” OpenAI and Anthropic know they need X. Nvidia is not quite as AGI-pilled. They’re building X - 1. You go down the supply chain, everyone’s doing X - 1. In some cases, they’re doing X ÷ 2, because they’re not AGI-pilled. You end up with this time lag for the whip to react. The AI-pilledness and the desire to increase production takes so long. Once they finally understand that they need to increase production rapidly… They think they understand. They think AI means we have to go from 60 to 100, in addition to the tools getting better and faster, the source getting higher power from 500 watts to 1,000, and all these other aspects of the supply chain advancing technically and increasing production. They think they’re actually increasing production a lot. But if you flow through the numbers… What does Elon want? He wants 100 gigawatts a year in space by 2028 or 2029. Sam Altman wants 52 gigawatts a year by the end of the decade. Anthropic probably needs the same, and Google needs that. You go across the supply chain, and it’s like, wait, no, the supply chain can’t possibly build enough capacity for everyone to get what they want on the side of compute. I feel like in the data center supply chain for the last few years, people have been making arguments like, “We are bottlenecked by this specific thing, therefore AI compute can’t scale more than X.” But as you’ve written about, if the grid is a bottleneck, then we just do behind the meter on the site, we do gas turbines, et cetera. If that doesn’t work, there are all these other alternatives that people fall back on. I want to ask whether we can imagine a similar thing happening in the semiconductor supply chain. If EUV becomes a bottleneck, what if we just went back to 7 nm and did what China is doing currently, producing 7 nm chips with multi-patterning with DUV machines? If you look at a 7 nm chip like the A100, there’s been a lot of progress obviously from the A100 to the B100 or B200. How much of that progress is just numerics? If you just hold FP16 constant from A100 to B100. The B100 is a little over one petaflop, and the A100 is like 300 teraflops. Yeah, 312.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.