Lex Fridman: evaluation

Transcript context

…Well, I think a lot of people who historically worked on AI never considered what happens when they succeed. Stuart Russell speaks beautifully about that. Let’s look, okay, maybe superintelligence is too futuristic. We can develop practical tools for it. Let’s look at software today. What is the state of safety and security of our user software? Things we give to millions of people? There is no liability. You click, “I agree.” What are you agreeing to? Nobody knows. Nobody reads. But you’re basically saying it will spy on you, corrupt your data, kill your firstborn, and you agree and you’re not going to sue the company. That’s the best they can do for mundane software, word processor, tax software. No liability, no responsibility. Just as long as you agree not to sue us, you can use it. If this is a state of the art in systems which are narrow accountants, stable manipulators, why do we think we can do so much better with much more complex systems across multiple domains in the environment with malevolent actors? With again, self-improvement with capabilities exceeding those of humans thinking about it. I mean, the liability thing is more about lawyers than killing firstborns. But if Clippy actually killed the child, I think lawyers aside, it would end Clippy and the company that owns Clippy. So, it’s not so much about… There’s two points to be made. One is like, man, current software systems are full of bugs and they could do a lot of damage and we don’t know what, they’re unpredictable. There’s so much damage they could possibly do. And then we kind of live in this blissful illusion that everything is great and perfect and it works. Nevertheless, it still somehow works. In many domains, we see car manufacturing, drug development, the burden of proof is on a manufacturer of product or service to show their product or is safe. It is not up to the user to prove that there are problems. They have to do appropriate safety studies. We have to get government approval for selling the product and they’re still fully responsible for what happens. We don’t see any of that here. They can deploy whatever they want and I have to explain how that system is going to kill everyone. I don’t work for that company. You have to explain to me how it’s definitely cannot mess up.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.