Nathan Lambert: belief

Transcript context

…I’m not claiming they’re all going to China, but a portion, many are going to Malaysia, including Microsoft and Oracle have big data centers in Malaysia. They’re going all over Southeast Asia probably, India as well. There’s stuff routing, but the diffusion rules are very de facto, like you can only buy this many GPUs from this country and you can only rent a cluster this large to companies that are Chinese. They’re very explicit on trying to stop smuggling. And a big chunk of it was, hey, random company buys 16 servers, ships them to China. There’s actually, I saw a photo from someone in the semiconductor industry who leads a team for networking chips that competes with NVIDIA, and he sent a photo of a guy checking into a first class United flight from San Francisco to Shanghai or Shenzhen with a super micro box that was this big, which can only contain GPUs, right? And he was booking first class because think about it, 3K to 5K for your first class ticket, server costs $240,000 in the US, $250,000, you sell it for $300,000 in China. Wait, you just got a free first class ticket and a lot more money. So it’s like… And that’s small scale smuggling. Most of the large scale smuggling is companies in Singapore and Malaysia routing them around or renting GPUs, completely legally- I want to jump in. How much does this scale? I think there’s been some people that are higher level economics understanding say that as you go from 1 billion of smuggling to 10 billion, it’s like you’re hiding certain levels of economic activity and that’s the most reasonable thing to me is that there’s going to be some level where it’s so obvious that it’s easier to find this economic activity. And- Yeah. So, my belief is that last year roughly, so NVIDIA made a million H20s, which are legally allowed to be shipped to China, which we talked about is better for reasoning, inference at least, not training, but reasoning inference and inference generally. Then they also had a couple hundred thousand, we think like 200,000 to 300,000 GPUs were routed to China from Singapore, Malaysia, US, wherever. Companies spawn up, buy 16 GPUs, 64 GPUs, whatever it is, route it, and Huawei is known for having spent up a massive network of companies to get the materials they need after they were banned in 2018. So, it’s not otherworldly, but I agree, right? Nathan’s point is like, hey, you can’t smuggle $10 billion of GPUs. And then the third source, which is just now banned, which wasn’t considered smuggling, but is China is renting, I believe from our research, Oracle’s biggest GPU customer is ByteDance. And for Google, I think it’s their second-biggest customer. And you go down the list of clouds and especially these smaller cloud companies that aren’t the “hyperscalers,” think beyond CoreWeave and Lambda even, there’s 60 different new cloud companies serving NVIDIA GPUs. I think ByteDance is renting a lot of these, all over it, right? And so these companies are renting GPUs to Chinese companies, and that was completely legal up until the diffusion rules, which happened just a few weeks ago. And even now, you can rent GPU clusters that are less than 2,000 GPUs, or you can buy GPUs and ship them wherever you want if there are less than 1,500 GPUs. There are still some ways to smuggle, but yeah, as the numbers grow a hundred something billion dollars of revenue for NVIDIA last year, 200 something billion this year, and if next year, it could nearly double again or more than double based on what we see with data center footprints being built out all across the US and the rest of the world, it’s going to be really hard for China to keep up with these rules. Yes, there will always be smuggling and DeepSeek level models, GPT-4 level models, o1 level models capable to train on what China can get, even the next tier above that. But if we speed run a couple more jumps to billion dollar models, $10 billion models, then it becomes, “Hey, there is a compute disadvantage for China for training models and serving them.” And the serving part is really critical, right? DeepSeek cannot serve their model today. It’s completely out of inventory. It’s already started falling in the app store actually, downloads, because you download it, you try and sign up, they say, “We’re not taking registrations,” because they have no capacity. You open it up, you get less than five tokens per second, if you even get your request approved, right? Because there’s just no capacity because they just don’t have enough GPUs to serve the model, even though it’s incredibly efficient.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.