Evidence receipt / prediction
Published · transcript-backedDylan Patel: prediction
2 Oct 2024 Dwarkesh Podcast @Asianometry & Dylan Patel — How the semiconductor industry actually works
“China has no issue with that at all because their supply chain adds as much power as like half of Europe every year.”
Source trail
Everything needed to verify it.
- Speaker
- Dylan Patel
- Attribution
- Verified speaker
- Claim type
- prediction
- Recorded
- 2 Oct 2024
- Publisher
- Dwarkesh Podcast
Transcript context
…Alright, we covered the chips themselves. How do they get the 10 gigawatts data center up? What else do they need? There is a true question of how decentralized do you go versus centralized. In the US, as far as labs and such, you have OpenAI, xAI, Anthropic. Microsoft has their own effort, Anthropic has their own efforts, despite having their partner. Then you have Meta. You also have all the interesting startups doing stuff. You go down the list and there's quite a decentralization of efforts. Today in China, it is still quite decentralized. It's not like, “Alibaba, Baidu, you are the champions.” You have DeepSeek doing amazing stuff and it’s like, “Who the hell are you? Does the government even support you?” If you are Xi Jinping and scale-pilled, you must now centralize the compute resources. Because you have sanctions on how many NVIDIA GPUs you can get in now. They're still north of a million a year, even post-October last year sanctions. We still have more than a million H20s, and other Hopper GPUs getting in through other means but legally the H20s. On top of that, you have your domestic chips, but that's less than a million chips. When you look at it, it's like, “Oh, well, we're still talking about a million chips.” The scale of data centers people are training on today slash over the next six months is 100,000 GPUs. OpenAI, xAI, these are quite well documented and others. But in China, they have no individual system of that scale yet. Then the question is, “How do we get there?” No company has had the centralization push to have a cluster that large and train on it yet, at least publicly and well-known. The best models seem to be from a company that has got like 10,000 GPUs or 16,000 GPUs. It's not quite as centralized as the US companies are and the US companies are quite decentralized. If you're Xi Jinping and you're scale-pilled, do you just say, “XYZ company is now in charge and every GPU goes to one place”? Then you don't have the same issues as in the US. In the US, we have a big problem with being able to build big enough data centers, being able to build substations and transformers and all this that are large enough in a dense area. China has no issue with that at all because their supply chain adds as much power as like half of Europe every year. It’s some absurd statistic. They're building transformer substations or building new power plants constantly. They have no problem with getting power density. You go look at Bitcoin mining. Around the Three Gorges Dam, at one point at least, there was like 10 gigawatts of Bitcoin mining estimated. We're talking about gigawatt data centers coming over in 2026 or 2027 in the US. This is an absurd scale relatively. We don't have gigawatt data centers ready. China could just build it in six months, I think, around the Three Gorges Dam or many other places. They have the ability to do the substations. They have the power generation capabilities. Everything can be done like a flip of a switch, but they haven't done it yet. Then they can centralize the chips like crazy. ability to do the substations. They have the power generation capabilities. Everything can be done like a flip of a switch, but they haven't done it yet. Then they can centralize the chips like crazy. Right now they can be like “Oh, a million chips that NVIDIA's shipping in Q3 and Q4, the H20, let's just put them all in this one data center.” They just haven't had that centralization effort yet.…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.