Scott Alexander: belief

Transcript context

…Yeah, so I think you’re over-indexing or cherry-picking some of these fortuitous examples. But there’s also things on the other side. Think about the recent history of AGI where there is DeepMind, there’s various other AI companies, then there’s OpenAI and there’s Anthropic, and there’s just this repeated story of [a] big bloated company with tons of money, tons of smart researchers, et cetera, flailing around trying a ton of different things at different points. Smaller startup with a vision of “we’re going to build AGI” and overall working towards that vision more coherently with a few cracked engineers and researchers. And then they crush the giant company. Even though they have less compute, even though they have less researchers, they’re able to do fewer experiments. So yeah, I think that there are tons of examples throughout history, including recent relevant AGI history, of things in the other way. I agree that the random fortuitous stuff does happen sometimes and is important. But if it was mostly random fortuitous stuff, that would predict that the giant companies with zillions of people trying zillions of different experiments would be going proportionally faster than the tiny startups that have the vision and the best researchers. And that basically doesn’t happen. That’s rare. I would also point out that even when we make these random fortuitous discoveries, it is usually an extremely smart professor who’s been working on something vaguely related for years in a first world country. It’s not randomly distributed across everyone in the world. You get more lottery tickets for these discoveries when you are intelligent, when you have good technology, when you’re doing good work. And the best example I can think of is that Ozempic was discovered by looking at Gila monster venom. And maybe the AIs will decide using their superior research taste and good planning that the best thing to do is just catalog every single biomolecule in the world and look at it really hard. But that’s something you can do better if you have all of this compute, if you have all of this intelligence, rather than just kind of waiting to see what things the US government might fund normal fallible human researchers to do. One more thing I’ll interject. I think you make a great point that discoveries don’t always come from where we think, like Nvidia originally came from gaming. So you can’t necessarily aim at one part of the economy, expand it separately from everything else. We do kind of predict that the superintelligences will be somewhat distributed throughout the entire economy, trying to expand everything. Obviously more effort in things that they care about a lot, like robotics or things that are relevant to an arms race that might be happening. But we are predicting that whatever kind of broad based economic experimentation you need, we are going to have. We’re just thinking that it would take place faster than you might expect. You were saying something like 10 years and we’re saying something like one year. But we are imagining this broad diffusion through the economy, lots of different experiments happening.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.