Daniel Kokotajlo: belief

Transcript context

…And we haven’t really got into this because you’re using the quote-unquote more conservative vision where it’s not like godlike intelligence, we’re still using the conceptual handles we would have for humans. But I think I would rather have humans go back with their big picture understanding of what has happened over the last 2000 years. Like me having seen everything, rather than a superintelligence who knows nothing. But it’s just in the Roman economy and they’re like 1000x this economy somehow. I think just knowing generally how things took off, knowing basically steam engine, dot dot dot, railroads, blah, blah, blah, is more valuable than a super intelligence. Yeah, I don’t know. My guess is that the superintelligence would be better. I think partly it would be through figuring out that high level stuff from first principles rather than having to have experienced it. I do think that a superintelligence back in the Roman era could have guessed that eventually you could get autonomous machines that burn something to produce steam. They could have guessed that automobiles could be created at some point and that that would be a really big deal for the economy. And so a lot of these high level points that we’ve learned from history, they would just be able to figure out from first principles. And then secondly, they would just be better at learning by doing than us. And this is a really important thing. If you think you’re bottlenecked on learning by doing, well, then if you have a mind that needs less doing to achieve the same amount of learning, that’s a really big deal. And I do think that learning by doing is a skill, some people are better at it than others, and superintelligence would be better at it than the very best of us. This is also maybe getting too far into the godlike thing and too far away from the human concept handles. But number one, I think we rely a lot in our scenario on this idea of research taste. So you have a thousand different things that you could try when you’re trying to create the next steam engine or whatever. Partly you get this by bumbling about and having accidents and some of those accidents are productive. There are questions of, what kind of bumbling you’re doing, where you’re working, what kind of accidents you let yourself get into, and then what directed experiments do you do? And some humans are better than others at that. And then I also think at this point it is worth thinking about what simulations they’ll have available. If you have a physics simulation available, then all of these real world bottlenecks don’t matter as much. Obviously you can’t have a complete, perfect physics simulation available. But even right now we’re using simulations to design a lot of things. And once you’re super intelligent, you probably have access to much better simulations than we have right now.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.