Zvi Mowshowitz: belief

Transcript context

…The invitation is open to the podcast, Ilya, if you're listening. For what it's worth, a vision that I have that I think could be pretty cool. Thinking Machines is doing kind of a version of this with their foundation model that's really designed to be fine tuned. But I was thinking a paradigm where, like, a stem cell where you create something that is a proto intelligence that can be adapted to a wide variety of circumstances and can get really good at doing its job in a particular niche. But in the process of getting good at that sort of trades away or prunes away the the super broad capabilities that the current models have so that you have something that's, like, small, fits its role well, does a good job, but can only do that thing in the way that once you go from a stem cell to a specialized cell, like, it has this particular role that it does it can fulfill effectively, but it's not gonna go off and do other things. Obviously, there's some problems in that analogy. But I think that could be a a I would love to see somebody pursue that kind of rather than scaling up always bigger, better, more. Can we create something that settles and fits into its place? This is very much like a Drexler reframing superintelligence vision from years ago as well. Two techniques that have recently come out that I wanna get your reaction to. One is GRAM, gradient routing. I forgot what the a m stands for, but the the promise of it is to localize certain kinds of knowledge to particular experts within an MOE architecture so that you can hopefully have your cake and eat it too in terms of distributing, maybe even open source or open weights at least, the model minus the experts of concern while potentially having also a structured access program for your trusted biologists or what have you. Right. I'm excited about it, but I wouldn't be doing my job if I didn't give you the chance to pour some cold water on it. I mean, I'm technically pretty skeptical that you can meaningfully train a general purpose AI and then just hold back areas of knowledge. And then people can't put them back pretty easily. But cool. You're welcome to try. I I have I have very technical skepticisms of the effectiveness slash the resistance defined to additional training or fine tuning or just give me the corpus of knowledge to work with. But you can try. I don't think it materially changes my view of open weights and until proven otherwise. I also just don't see any willingness from the people who are gonna produce the open weight models to intentionally control their models that way. Right? Like, the thing about, like, techniques like GRAM is even if they work, you need everybody who is releasing one of these models to use the technique properly, voluntarily at this point. Right? So, like, do you have any sense that DeepSeek has any interest in holding back some capabilities from its models? Because I don't. I actually would so I just got back from China. Great opportunity for me to talk about my China trip. I think, actually, there's a lot more hope there than meets the eye, and I would put it in the Chinese government's department to say and I might I just did a two hour episode about this, so I won't even attempt to recap all the observations. But in short, what I think is happening right now in China is the government is very engaged. They do prerelease reviews of all the company's models before they come out. They're in very regular dialogue with the companies on what they're doing, and not every little point release has to go through the full treatment, but they're pretty on top of the situation. And I don't know that it matters actually what DeepSea thinks in the end because if the CCP says, you are not releasing a model that has certain bio capabilities, I think they'll have to abide by that. And and I can I it's not harder at all for me to imagine that the CCP would put certain constraints in place? And there is actually precedent too that, you know, going back to, like, 2023, obviously, stakes were lower. But a the account that I have is that the Chinese government said, hey. We're gonna slow you down here for a minute, and you're not gonna release your answer to ChatGPT quite so fast because we wanna get a handle on it first. And they actually did impose some real delays on companies until they could get comfortable with what was going on. And since then, they've been comfortable and releases have happened. But I think that it's not hard to imagine the technocratic government of China saying, this seems like a really bad idea, and we're just not gonna allow you to do this Yeah. Agree. Anymore.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.