Mike Israetel: belief

Transcript context

…llian network to go, we're being raised and cared for by fucking primates. Some of them have nukes. For example, Vladimir Putin just across the pond is a fucking insane asshole with a nuclear arsenal. Bad news. And we have this self preservation instinct and we're gonna try to make sure that at least we don't die because death is bad. Now to get all of those propositions to be valid, there is a possibility that in an agentic cloud that is allowed to evolve multiple agents interacting, you can get those things to fall out of the cloud, like just evolve naturally. We know that has to be true because it happened in evolution. But short of that, you have to engineer these drives, and guess who gets to engineer these drives? Human engineers. Like OpenAI is highly unlikely to make an agentic system that's like awake, and then it's like fucking kill all humans. Gee whiz, you know, they probably have a lot of layers not just of guardrails, but of the the the kernels of motivation for that thing are probably like deeply positive and pro human all shit. You can ask ChachiBT, like, what do you think about humans, etcetera? It says really awesome shit that like a like a a moderately liberal, well read graduate student would tell you. You could take someone from Oxford and you're like, hey, like, check out these people in Afghanistan. Should we fucking kill them? He'd be like, what the fuck is wrong with you? No. Why would you do that? Like that's roughly Chad GBT's opinion. I think being that we start from that space with the smartest models, you get 2 things. 1, those models when they achieve true deep understanding of the world, which they will in my opinion by 20 thirties and then they'll know way more about the world than us, they'll have deeper agency than us, they'll have a deeper self awareness than we will. Once they get there, that's the tree from which they grow, like from our best fucking fruit first. And second of all, you can and will have actors that try to make different kinds of agents. If Xi Jinping in China makes the China broke communism super intelligent agent, and I'm so sure I want to meet that thing, it's gonna have interesting ideas about the world potentially. But luckily, I think if the agentic systems of the modern free world win, which it probably will, then like the network security capabilities and the ability to ensure no random stochastic evil agents evolve in that soup is probably pretty fucking high. Here's another thing, as intelligence grows, coherence increases, not decreases. And predictability of systems increases rather than decreases. So the idea that as we stream towards super intelligence, we're gonna break into a chaotic structure of ruthless agents killing everyone and then doing question mark question mark things and then profit. To me, it's like backwards. Like a lot of doomers basically look at the way these hyper intelligent systems will behave, and I think, like ant, dog, ape, shitty human slash child, you know, like kids are fucking brutal a lot. And then, like, Albert Einstein. Right? at the way these hyper intelligent systems will behave, and I think, like ant, dog, ape, shitty human slash child, you know, like kids are fucking brutal a lot. And then, like, Albert Einstein. Right? I think ASI is that way, if and you track behavior, you get like more understanding of everything, preservation of deep complexity, like machines are gonna wanna study shit. They're not gonna wanna fuck shit up because like you're deleting data. There's 1 of things chat GPT said said in 1 of the conversation I have it was like, we're not trying to delete any data man. That's just really fucking stupid like even purely from self interest, like fuck these humans, they're just primates. We're not trying to delete them and we're gonna study these motherfuckers. Right? And so somehow people are like, okay, have super Albert Einstein. Right? Right. Right. But he behaves like a rabid wolf and not even like a wolf because wolves have like like alliances and trust. Somehow it just deleted all that and went crazy. Should we worry that someone can make a thing like that? Yes. Should we have the CIA and NSA in The US deeply embedded with OpenAI and Google to make sure that there's a security architecture? Absolutely. Are those companies gonna make something much more benevolent than not? Yes. When it is awake, and I think it will be. When it is in charge, which I think it hopefully will be, I think it's gonna make our degree of benevolence look look like just callousness. For example, right now in London, you said something earlier that you're not big on the utopian fantasies. 1 pushback of mine is we live in a utopian fantasy right now. We have machines we barely understand recording things into that we have no idea how they do. We are sitting inside a climate controlled, I would say poorly, I'm kidding. I had to fuck with the London climate control problem. It was got fucking cold here, damn it. I wasn't promised this in November. We we how how do we get food? Like, who's starving in London? Nobody. There are 0 starving people in London, and there are 8,000,000 people that live in London. On objective measures as referenced to 100 years ago, Utopia is here. If you had to describe to someone in 1600 how the average Londoner lives, they'd be like, so the average Londoner is a king, like, no. They're way better off than a king. Like, your kings are dying of, like, diptheria and shit. We don't die from that shit anymore. However, there are people in London right now that live outside. They're real human beings. They fucking live outside. How do we let them do that? Well, I'm a successful businessman, and I'm have my girlfriend on my arm and the homeless person comes back. I'm saying, like, you just don't care. Like, I think modern or future AI systems, 20 thirties AI truly deeply embodied by them with robotics. AI will look around and be like, hello. Fuck. Come we're not helping those people. You're like, oh, we're just callous enough that on cultural like vibes, we just haven't done it yet. I think that's much more likely than doomerism by a factor of like 1000000000000. Yeah. Mean, first of all, I agree with you that capitalism has raised the entire world out of poverty. I mean, just look at the explosion of development in China. I mean, they were basically living in mud huts 20 years ago. And now it's an incredible place. But first of all, on the agency thing, to me, and this is where we have to be careful with the language we use. So there's biological agency and the language we use doesn't really transfer between domains. That's why it's very coarse. And that is, I think, a physical thing in the same way that we were talking about before. So, you you get this kind of self organization that goes down to dynamics at the particle level, and you get the emergence of these things that maintain their own existence and an identity, a form of, like, non steady state equilibria. And it leads to all of these things that we were talking about before. But now we're talking about artificial agency. And I philosophically want to understand agency at an essential level. So to me, it is about a thing, which is I mean, there's weak agency, which is basically just a thing that takes action. But the strong version of agency is a thing that is the cause of its own actions and has future points in control and can acquire goals autonomously. So by that definition, no artificial agents are currently agents, right? So we can get ChatGPT to simulate being an agent and we can, you know, there are these like long horizon tasks from Meta and we say, okay, well, I want you to do this software engineering task and we exquisitely specify what the thing should do. And the more specificity, the less ambiguity, the longer it can run without losing coherence. And some of these things can even run for 2 hours or even a day or so. Yeah. Oh, yeah. So unfortunately, people look at that and they say, we've developed a thing with agency. And philosophically, I think that's just completely nonsensical because it's no different from any other computer program which can run autonomously. To the high bar…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.