Ali Behrouz: belief

Transcript context

…Do you have a sense of what those are? I tend to just assume. It's honestly taking me pretty far just assuming everything's going to work because it does. My general sense of the field probably is like an unbelievable amount of things are working. And I thought it was really interesting. You mentioned earlier that the nested learning paper was like in, in development for over a year. That's so rare these days. So many people are going like on 6-8 week paper cycles and often those papers can be like really interesting and, and good too. So it's not a knock on them at all, but it is amazing how fast people are able to get results these days. What's your intuition for why it's too early for necessary learning to be brought to robotics? I think there are a lot of components in those those specific tasks that needs to be addressed. For example, these days a lot of like papers are coming about word models and why the current design is not great about like word on word models. And actually that's that's true. I think there are a lot of challenges and the current design might not be the best a way we can train the model or for example, we can design the architecture. You know, generally also there are some challenges in the infrastructure of of the model for more board modelling. And so all these things together, I think there are more important, you know, tasks for, for example, word modeling rather than start starting to work on these specific design shoes, but differently at some point when we could solve all those challenges, then then we can come back and use all these techniques for free, further improving. All those aspects. One thing I do worry about a little bit with continual learning. Let's say that you Google is able to retain you after your PhD and Zuckerberg, you, you were you, you get the good enough counter offer from Google to stay despite the whatever offer Zuckerberg going to throw at you. And you guys make it work, right? And it's like now we've got Gemini continual learning Edition and like Gemini CL, it's just learning from everything, right? So all these different ways it's deployed in the world, maybe you have some like enterprise deals where like you can't learn on their stuff or whatever. But like, you know, you've got hundreds of millions of users and it's just going, going out in the world. And increasingly, maybe it even isn't robots and it's, you know, so it's it's becoming it seems like it has the potential to create this sort of return to scale rich, get richer, you know, positive feedback loop dynamic where people have sort of sometimes painted a picture of like, well, what happens if one model like becomes the one model to rule them all? And right now people are like, yeah, we don't really see that. You know, it's a pretty competitive landscape. And, you know, they keep the different developers keep leapfrogging one another. But arguably this could be the thing that changes that if you could really fold all of the lessons learned back into the core thing, then you know, potentially you become the best and then because you're the best, you get all the business And you know that that pattern really could create sort of a winner take all dynamic. I wonder do you worry about that at all? And do we have any ways of Ilya's thing comes to mind? I don't know if you you watched the Ilya interview with Duarkesh. He didn't say too much about what they're doing over at safe super intelligence. But one thing that he did say that sort of, you know, it's it's certainly it's clearly related in some sense, but like how how similar the underlying ideas are, I have no idea. But he sort of described this idea of creating kind of what I would describe as like a proto intelligence or a precursor intelligence trying to create something that when deployed would like adapt into context, perhaps like crystallize in some way, become like an expert in its role. But it sounded the way I understood him to be speaking about it, like he was almost like a stem cell kind of concept. Like he's trying to create a stem cell. And then that stem cell as it does in our body, like it specializes into a particular kind of cell and it stays that kind of cell and then it does its job. It sounded to me like he was kind of trying to create something similar, something that could go out into any environment, figure out how to be great at it, but also in the process of becoming great at what it needed to do in that particular environment, also lose some of the generality that it originally started with. And in that process, be more safe.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.