Lex Fridman: belief

Transcript context

…I like AI’s. I, for one welcome our overlords. There’s a degree to which we… I mean it is very obvious as we already have, we’ve increasingly given our life over to software systems. And then it seems obvious given the capabilities of AI that are coming, that we’ll give our lives over increasingly to AI systems. Cars will drive themselves, refrigerator eventually will optimize what I get to eat. And, as more and more out of our lives are controlled or managed by AI assistants, it is very possible that there’s a drift. I mean, I personally am concerned about non-existential stuff, the more near term things. Because before we even get to existential, I feel like there could be just so many brave new world type of situations. You mentioned the term, “Behavioral drift.” It’s the slow boiling that I’m really concerned about as we give our lives over to automation, that our minds can become controlled by governments, by companies, or just in a distributed way. There’s a drift. Some aspect of our human nature gives ourselves over to the control of AI systems and they, in an unintended way just control how we think. Maybe there’ll be a herd-like mentality in how we think, which will kill all creativity and exploration of ideas, the diversity of ideas, or much worse. So it’s true, it’s true. But a lot of the conversation I’m having with you now is also kind of wondering almost at a technical level, how can AI escape control? What would that system look like? Because it, to me, is terrifying and fascinating. And also fascinating to me is maybe the optimistic notion it’s possible to engineer systems that defend against that. One of the things you write a lot about in your book is verifiers. So, not humans. Humans are also verifiers. But software systems that look at AI systems, and help you understand, “This thing is getting real weird.” Help you analyze those systems. So maybe this is a good time to talk about verification. What is this beautiful notion of verification? My claim is, again, that there are very strong limits in what we can and cannot verify. A lot of times when you post something on social media, people go, “Oh, I need citation to a peer reviewed article.” But what is a peer reviewed article? You found two people in a world of hundreds of thousands of scientists who said, “Ah, whatever, publish it. I don’t care.” That’s the verifier of that process. When people say, “Oh, it’s formally verified software or mathematical proof,” we accept something close to 100% chance of it being free of all problems. But you actually look at research, software is full of bugs, old mathematical theorems, which have been proven for hundreds of years have been discovered to contain bugs, on top of which we generate new proofs and now we have to redo all that. So, verifiers are not perfect. Usually, they are either a single human or communities of humans and it’s basically kind of like a democratic vote. Community of mathematicians agrees that this proof is correct, mostly correct. Even today, we’re starting to see some mathematical proofs as so complex, so large that mathematical community is unable to make a decision. It looks interesting, it looks promising, but they don’t know. They will need years for top scholars to study to figure it out. So of course, we can use AI to help us with this process, but AI is a piece of software which needs to be verified.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.