Mike Israetel: belief

Transcript context

…I don't know. Most people don't understand shit about counterfactuals, me included. Just vibe your way to shit. The the the there's a very nice way that you're painting human cognition, which I think many cognitive scientists would say under the hood is a lot dirtier than you make it appear. I think it's a lot of vibes all the way down in many cases. And I think it's we have a frightening overlap with AI in it. So we think we're doing formal logical operation, but we're really doing is reasoning by analogy to 1 logical operator to the other. And we're we're committing 18 formal statistical fallacies every single render. We got a shitload wrong. And I think that now that reasoning models are a thing, they have decent context, decent memory, what they can do is render 1 pass, reexamine, re prompt themselves, render the pass, reexamine. And so I think that's what humans do except usually they don't put nearly as much thought into things as the machines do. So my view is that as the cogency of the the reasoning models increases and their time horizons increase, we're absolutely getting truly deep agency in the sense of like it's a thing in there. It's really thinking about stuff. It thinks about its own outputs and then it thinks about those as inputs to the next series. And if we can have a model that reasons for minutes and hours and then days, its output is going to be as not so good, then as good, and then substantially better than people. I don't think there's anything stopping us from that aside from the live learning problem. Live learning problems are gnarly problem for 2 reasons. 1 is algorithmic. How do you get a machine to do that without losing all of its data? But the other 1 is like, and you gotta you gotta trust the machine a lot for it to update its own weights, you know. Now you can have local cloning of weights to make sure it doesn't fuck the whole database up and all that stuff. There's good solutions to it. So it's computationally very expensive. Although compute cost has fallen about 300 x in the last year or 2 or something and that that's that's a bottomless pit of compute fall that's gonna keep going. So I think that we're giving humans needed credit for doing pretty impressive operations on long time horizons with both pinging a network for vibes, logically operating and kind of dissecting those vibes, re pinging the network with your best new understanding and going. I think machines are doing that now already with reasoning models. They started with o 1 doing it like total dog shit. Like the first time I fucked with o 1, fucked with it for about 5 minutes. I was like, that shit's not ready. Out. Went to 4.5, had lots of crying episodes that how brilliant it was. And then o 3, I thought was made plenty of mistakes, but was fucking real smart. Real goddamn smart and it understood things at an incredibly deep level would correct me and I was like, okay, this thing is the fucking real deal. GPT 5, the regular 1 is dope. as fucking real smart. Real goddamn smart and it understood things at an incredibly deep level would correct me and I was like, okay, this thing is the fucking real deal. GPT 5, the regular 1 is dope. GPT 5 Pro, I would trust its opinion about the real world more than I would trust most people's opinion often including my own. And I think we're about 1 or 2 big model releases away from machines that make unaided human thought the same thing as a vibing your way to a location in London without checking Google Maps. Something people used to do, but then they just got lost a lot. And it works and it's cool and it's embodied, but just go to Google Maps and find out how to really get there. I think that all these moats we set up for machine intelligence are moats that are pretty well understood by the modern AI frontier labs. And if we've come up with them as moats, these guys are I mean, they're smarter than me for sure by an incalculable amount. They probably already thought of them and they probably shut the fuck up about them in public and they've probably got models training right now that have cracked incrementally more of them. Here's to prove a point. This requires a bit of faith that they'll continue to do a good job, but I think that's probably where it's going. AI doesn't reason used to be an absolutely coherent and accurate critique of AI in 2023, 2022 for sure. Today, and I've seen this sometimes in comments where it could just was a sarcastic parrot like, nope. You're just out of date by 2 years on what AI can do. It really does reason. Why? Because it tells you how is you can literally look at what it's reasoned through and go holy fuck. And the way it reasons that level of precision and the level of documentation is way better than what I do reasoning wise. Most of time I just look at stuff and I'm like, okay. That's the right answer. There's some reasoning in there, but a lot more neural network pinging than I would like to admit in public. This isn't public. Right? It's not gonna go out on YouTube. Well. Oh, fuck. Yeah. Well. By the way, the the stochastic paracriticism isn't as much of a criticism as as as you think. Right? These things have miraculous statistical generalization. I also wanted to pick up on you you were saying you can look at the reasoning trace. That doesn't really tell you anything. There have been lots of studies that show that what it tells you it's the same in humans, by the way. There are studies in psychology that people do this post hoc confabulation. Absolutely. The reason for that is the circuits which do the explanation are actually different in the neural network from the circuits which did the thing. You…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.