Nathan Labenz: belief

Transcript context

…I think I think agent vanishingly is probably the best case scenario. Maybe they've been doing a bunch of theoretical work on a different architectural approach that is more like reliability bound and that has more like, it is much less of a weird black box of soup and that allows you to steer it and guide it better. Certainly seems like there's no reason it couldn't exist. That would be my top hope. Obviously, it could also be a fear if it's, like, even worse. But again, like, they've they don't talk, so we don't know. Yeah. I just I have no idea. I know that Ilya's statements in interviews on the problem space did not seem to be that well informed about, like, how the problems work that I am worried about. You know, I'm not necessarily sure he would be able to differentiate approaches that would be better at solving those problems than approaches that wouldn't be. But, you know, I'd be very curious. I can safely say that nobody's telling me anything. The invitation is open to the podcast, Ilya, if you're listening. For what it's worth, a vision that I have that I think could be pretty cool. Thinking Machines is doing kind of a version of this with their foundation model that's really designed to be fine tuned. But I was thinking a paradigm where, like, a stem cell where you create something that is a proto intelligence that can be adapted to a wide variety of circumstances and can get really good at doing its job in a particular niche. But in the process of getting good at that sort of trades away or prunes away the the super broad capabilities that the current models have so that you have something that's, like, small, fits its role well, does a good job, but can only do that thing in the way that once you go from a stem cell to a specialized cell, like, it has this particular role that it does it can fulfill effectively, but it's not gonna go off and do other things. Obviously, there's some problems in that analogy. But I think that could be a a I would love to see somebody pursue that kind of rather than scaling up always bigger, better, more. Can we create something that settles and fits into its place? This is very much like a Drexler reframing superintelligence vision from years ago as well. Two techniques that have recently come out that I wanna get your reaction to. One is GRAM, gradient routing. I forgot what the a m stands for, but the the promise of it is to localize certain kinds of knowledge to particular experts within an MOE architecture so that you can hopefully have your cake and eat it too in terms of distributing, maybe even open source or open weights at least, the model minus the experts of concern while potentially having also a structured access program for your trusted biologists or what have you. Right. I'm excited about it, but I wouldn't be doing my job if I didn't give you the chance to pour some cold water on it. I mean, I'm technically pretty skeptical that you can meaningfully train a general purpose AI and then just hold back areas of knowledge. And then people can't put them back pretty easily. But cool. You're welcome to try. I I have I have very technical skepticisms of the effectiveness slash the resistance defined to additional training or fine tuning or just give me the corpus of knowledge to work with. But you can try. I don't think it materially changes my view of open weights and until proven otherwise. I also just don't see any willingness from the people who are gonna produce the open weight models to intentionally control their models that way. Right? Like, the thing about, like, techniques like GRAM is even if they work, you need everybody who is releasing one of these models to use the technique properly, voluntarily at this point. Right? So, like, do you have any sense that DeepSeek has any interest in holding back some capabilities from its models? Because I don't.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.