Shawn Wang: belief

Transcript context

…Exactly. With synthetic data, we saw that if we just got you the right distribution of data that fit what you needed in the real world, then that was it. And you didn't have to train with as many samples as you needed otherwise. In a way, I see it like training. a, child in like Exeter, right? It doesn't matter how smart the child is because the information is being fed to it so well, in particular, like, you know, there are really incredible schools that fit the information to you really well and the right information. And by doing that as a human that works, I don't see why that doesn't work. It doesn't work with this kind of models and we saw it working in computer vision. It was just very small data set, just the right data, fit it well, and it will work. Um, yeah. And that was the experience. I think the problem here comes from like, I think we understand how to do this in a normal ML context, but when you're trying to build AGI, the real world is everything. There's nothing to optimize for because it's, it's everything. So how do you optimize for everything? I think the places where we're going to get AGI is where the AI can get complete feedback, but this is just my intuition behind it. So for example, in a coding environment that AI will have the ability to like rerun things and reevaluate if it's performing things well, and that will work, I still, I'm not sure how it would work with like something where you don't have. Feedback. So like in robotics, we first need to get like that really good, like grasping sensors or like really good vision sensors such that it can get some kind of feedback loop eventually started. But yeah, that goes more on like that reinforcement learning side where we've already seen superhuman performance, but it's still with LLMs. I think we're still approximating what we have available. It's a super interesting topic, but It really depends on like how you define it, and we will have to have a discussion on the definition and then how you measure it.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.