John Schulman: prediction

Transcript context

…Let’s talk about the websites that are designed for these AIs. Once they’re trained on more multimodal data, will they be in any way different from the ones we have for humans? What UIs will be needed? How will it compensate for their strengths and weaknesses? How would that look different from the current UIs we have for humans? That's an interesting question. I expect that models will be able to use websites that are designed for humans just by using vision, after the vision capabilities get a bit better. So there wouldn't be an immediate need to change them. On the other hand, there’ll be some websites that are going to benefit a lot from AIs being able to use them. We’ll probably want to design those to be better UXs for Ais. I'm not sure exactly what that would mean. Assuming that our models are still better at text mode than reading text out of images, you'd probably want to have a good text-based representation for the models. You’d also want a good indication of what all the things that can be interacted with are. But I wouldn't expect the web to get totally redesigned to have APIs everywhere. We can get models to use the same kind of UIs that humans use. I guess that's been the big lesson of language models, right? That they can act within the similar affordances that humans do. I want to go back to the point you made earlier about how this process could be more sample efficient because it could generalize from its pre-training experiences of how to get unstuck in different scenarios. What is the strongest evidence you’ve seen of this generalization and transfer? The big question for the future abilities models seems to be about how much generalization is happening. Is there something that feels really compelling to you? Have you seen a model learn something that you wouldn't expect it to learn from generalization?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.