Mike Israetel: belief

Transcript context

…But creativity requires deep understanding. So this is 3 levels down. Now the other thing is, right? You as an expert, you use these language models. And I don't think you appreciate the amount of supervision that you do. So every time the language imagine when you put a prompt into a language model, it's like doing a database query. And it's incredibly good. It's so good that most of the time, it will get it right the first time. But you probably find if you're doing some Gen AI coding or something like that, it'll make lots of mistakes. I say, give me 10 ideas for this software program and I've told it what kind of software program I wanna make. And let's say 2 of the 10 ideas are actually bad. So every time I'm implicitly steering it, no, that's bad, that's good, that's bad, that's good. And because we are understanding supervisors, can actually kind of weed out all of the glitches in the matrix, so to speak. And it works incredibly well. But the mistake are these people who say, oh yeah, we can make these things have agency. They don't have agency. We'll talk about that. But they role play agency. And the reason they role play agency is they are trained with behavior cloning. They're trained in the evolutionary way that we are. We actually make decisions. We say we could do this thing or we could do this thing and we understand the counterfactual of why we didn't do another thing. We weren't just behavior. I don't know. Most people don't understand shit about counterfactuals, me included. Just vibe your way to shit. The the the there's a very nice way that you're painting human cognition, which I think many cognitive scientists would say under the hood is a lot dirtier than you make it appear. I think it's a lot of vibes all the way down in many cases. And I think it's we have a frightening overlap with AI in it. So we think we're doing formal logical operation, but we're really doing is reasoning by analogy to 1 logical operator to the other. And we're we're committing 18 formal statistical fallacies every single render. We got a shitload wrong. And I think that now that reasoning models are a thing, they have decent context, decent memory, what they can do is render 1 pass, reexamine, re prompt themselves, render the pass, reexamine. And so I think that's what humans do except usually they don't put nearly as much thought into things as the machines do. So my view is that as the cogency of the the reasoning models increases and their time horizons increase, we're absolutely getting truly deep agency in the sense of like it's a thing in there. It's really thinking about stuff. It thinks about its own outputs and then it thinks about those as inputs to the next series. And if we can have a model that reasons for minutes and hours and then days, its output is going to be as not so good, then as good, and then substantially better than people. I don't think there's anything stopping us from that aside from the live learning problem. Live learning problems are gnarly problem for 2 reasons. 1 is algorithmic. How do you get a machine to do that without losing all of its data? But the other 1 is like, and you gotta you gotta trust the machine a lot for it to update its own weights, you know. Now you can have local cloning of weights to make sure it doesn't fuck the whole database up and all that stuff. There's good solutions to it. So it's computationally very expensive. Although compute cost has fallen about 300 x in the last year or 2 or something and that that's that's a bottomless pit of compute fall that's gonna keep going. So I think that we're giving humans needed credit for doing pretty impressive operations on long time horizons with both pinging a network for vibes, logically operating and kind of dissecting those vibes, re pinging the network with your best new understanding and going. I think machines are doing that now already with reasoning models. They started with o 1 doing it like total dog shit. Like the first time I fucked with o 1, fucked with it for about 5 minutes. I was like, that shit's not ready. Out. Went to 4.5, had lots of crying episodes that how brilliant it was. And then o 3, I thought was made plenty of mistakes, but was fucking real smart. Real goddamn smart and it understood things at an incredibly deep level would correct me and I was like, okay, this thing is the fucking real deal. GPT 5, the regular 1 is dope. as fucking real smart. Real goddamn smart and it understood things at an incredibly deep level would correct me and I was like, okay, this thing is the fucking real deal. GPT 5, the regular 1 is dope. GPT 5 Pro, I would trust its opinion about the real world more than I would trust most people's opinion often including my own. And I think we're about 1 or 2 big model releases away from machines that make unaided human thought the same thing as a vibing your way to a location in London without checking Google Maps. Something people used to do, but then they just got lost a lot. And it works and it's cool and it's embodied, but just go to Google Maps and find out how to really get there. I think that all these moats we set up for machine intelligence are moats that are pretty well understood by the modern AI frontier labs. And if we've come up with them as moats, these guys are I mean, they're smarter than me for sure by an incalculable amount. They probably already thought of them and they probably shut the fuck up about them in public and they've probably got models training right now that have cracked incrementally more of them. Here's to prove a point. This requires a bit of faith that they'll continue to do a good job, but I think that's probably where it's going. AI doesn't reason used to be an absolutely coherent and accurate critique of AI in 2023, 2022 for sure. Today, and I've seen this sometimes in comments where it could just was a sarcastic parrot like, nope. You're just out of date by 2 years on what AI can do. It really does reason. Why? Because it tells you how is you can literally look at what it's reasoned through and go holy fuck. And the way it reasons that level of precision and the level of documentation is way better than what I do reasoning wise. Most of time I just look at stuff and I'm like, okay. That's the right answer. There's some reasoning in there, but a lot more neural network pinging than I would like to admit in public. This isn't public. Right? It's not gonna go out on YouTube. Well. Oh, fuck. Yeah.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.