Lee Cronin: observation

Transcript context

…Just to push back on the argument where N of zero on the AGI … We could see on Earth that there’s increasing levels of intelligence of organisms. We can see what humans with extra intelligence were able to do to the other species. So, that is a lot of samples of data, what a delta in intelligence gives you. When you have an increase in intelligence, how you’re able to dominate a species on Earth. So, the idea there is that if you have a being that’s 10x smarter than humans, we’re not going to be able to predict what that being is going to be able to do, especially if it has the power to hurt humans. Which, you can imagine a lot of trajectories in which the more benefit AI systems give, the more control we give to those AI systems over our power grid, over our nuclear weapons, or weapons of any sort. And then it’s hard to know what an ultra-intelligence system would be able to do in that case. You don’t find that convincing? I think I would fail that argument 100%. Here’s a number of reasons to fail it on. First of all, we don’t know where the intention comes from. The problem is that people keep … I’ve been watching all the hucksters online with the prompt engineering and all this stuff. When I talk to a typical AI computer scientist, they keep talking about the AIs having some kind of decision-making ability. That is a category error. The decision-making ability comes from human beings. We have no understanding of how humans make decisions. We’ve just been discussing free will for the last half an hour, right? We don’t even know what that is. So, the intention, I totally agree with you, people who intend to do bad things can do bad things and we should not let that risk go. That’s totally here and now. I do not want that to happen, and I’m happy to be regulated to make sure that systems I generate, whether they’re computer systems, or … I’m working on a new project called “Chem Machina”. Nice. Well done.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.