Scott Alexander: belief

Transcript context

…Scott, you reviewed Joseph Henrik’s book Secrets of Our Success, and then I interviewed him recently, and there the perspective is very much AGI is not even a thing, almost. I know I’m being a little trollish here, but it’s just like: you get out there, you and your ancestors try for a thousand years to make sense of what’s happening in the environment. And some smart European coming around, you can literally be surrounded by plenty and you just will starve to death because your ability to make sense of the environment is just so little loaded on intelligence and so much more loaded on your ability to experiment and your ability to communicate with other people and pass down knowledge over time. I’m not sure. The Europeans failed at this task of, if you put a single European in Australia, do not starve. They succeeded at the task of creating an industrial civilization. And yes, part of that task of creating an industrial civilization was about collecting all of these cultural evolution pieces and building on them one after another. I think one thing that you didn’t mention in there was the data efficiency. Right now, AI is much less data efficient than humans. I think of superintelligence. There are different ways you could achieve it, but I would think of superintelligence as partly when they become so much more data efficient than humans that they are able to build on cultural evolution more quickly. And partly they do this just because they have higher serial speed. Partly they do it because they’re in this hive mind of hundreds of thousands of copies. But yeah, I think if you have this data efficiency such that you can learn things more quickly from fewer examples and this good research taste where you can decide what things to look at to get these examples, then you are still going to start off much worse than an Australian Aborigine who has the advantage of, let’s say 50,000 years of doing these experiments and collecting these examples. But you can catch up quickly. You can distribute the task of catching up over all of these different copies. You can learn quickly from each mistake and you can build on those mistakes as quickly as anything else. Part of me was, I was doing that interview, I’m like, “maybe ASI is fake”.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.