John Schulman: belief

Transcript context

…Obviously at some point they're going to be better than everyone at whatever they want to do. What would that process look like? Right now, they're clearly only helping you. At some point, they’ll be able to just do things for you and maybe run entire firms for you. Is it going to be a smooth process? At that point, is the hope that we have systems that are aligned with the user enough that they can count on the firm being run in the way they expect. We might not want to jump to having AIs run whole firms immediately. We might want to have people overseeing these important decisions and calling the shots, even if the models are good enough to actually run a successful business themselves. To some extent, there might be choices there. I think people will still have different interests and ideas for what kind of interesting pursuits they want to direct their AIs at. AI doesn't necessarily have any kind of intrinsic desire, unless we put it in the system. So even if AIs become extremely capable, I would hope that people are still the drivers of what the AIs end up doing. I wonder if the economic equilibrium is so far from that, where you have the equivalent of Amdahl's law in a firm. The slowest part of the process is the one that's going to bottleneck you. Even if AI makes all the non-human parts of the firm 10X more efficient, the firm is still bottlenecked by that step. If one company decides to proceed by keeping humans in the loop on all the things that you really want human oversight on, then they'll just be outcompeted by other companies. If one country decides to go this route, other countries will beat it. I wonder if this is a sustainable plan for keeping humans in the loop.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.