Scott Alexander: prediction

Transcript context

…Yeah, I don’t know. My guess is that the superintelligence would be better. I think partly it would be through figuring out that high level stuff from first principles rather than having to have experienced it. I do think that a superintelligence back in the Roman era could have guessed that eventually you could get autonomous machines that burn something to produce steam. They could have guessed that automobiles could be created at some point and that that would be a really big deal for the economy. And so a lot of these high level points that we’ve learned from history, they would just be able to figure out from first principles. And then secondly, they would just be better at learning by doing than us. And this is a really important thing. If you think you’re bottlenecked on learning by doing, well, then if you have a mind that needs less doing to achieve the same amount of learning, that’s a really big deal. And I do think that learning by doing is a skill, some people are better at it than others, and superintelligence would be better at it than the very best of us. This is also maybe getting too far into the godlike thing and too far away from the human concept handles. But number one, I think we rely a lot in our scenario on this idea of research taste. So you have a thousand different things that you could try when you’re trying to create the next steam engine or whatever. Partly you get this by bumbling about and having accidents and some of those accidents are productive. There are questions of, what kind of bumbling you’re doing, where you’re working, what kind of accidents you let yourself get into, and then what directed experiments do you do? And some humans are better than others at that. And then I also think at this point it is worth thinking about what simulations they’ll have available. If you have a physics simulation available, then all of these real world bottlenecks don’t matter as much. Obviously you can’t have a complete, perfect physics simulation available. But even right now we’re using simulations to design a lot of things. And once you’re super intelligent, you probably have access to much better simulations than we have right now. This is an interesting rabbit hole, so let’s stick with it before we get back to the intelligence explosion. I think we’re treating this really like all these technologies come out of this 1% of the economy that is research. And right now there’s like a million superstar researchers, and instead of that, we’ll have the superintelligences doing that. And my model is much more, “Newcomen and Watt were just like fucking around”. In human history there’s no clear examples of people being like, “here’s the roadmap”. And then we’re going to work backwards from that to design the steam engine because this unlocks the industrial revolution.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.