Roman Yampolskiy: evaluation

Transcript context

…I don’t think it always works with the precedent. You’re not stuck doing it the way you always did. It sets a precedent of open research and open development such that we get to learn together and then the first time there’s a sign of danger, some dramatic thing happened, not a thing that destroys human civilization, but some dramatic demonstration of capability that can legitimately lead to a lot of damage, then everybody wakes up and says, “Okay, we need to regulate this. We need to come up with safety mechanism that stops this.” But at this time, maybe you can educate me, but I haven’t seen any illustration of significant damage done by intelligent AI systems. So I have a paper which collects accidents through history of AI and they always are proportionate to capabilities of that system. So if you have Tic-Tac-Toe playing AI, it will fail to properly play and loses the game, which it should draw trivial. Your spell checker will misspell word, so on. I stopped collecting those because there are just too many examples of AI’s failing at what they are capable of. We haven’t had terrible accidents in a sense of billion people got killed. Absolutely true. But in another paper I argue that those accidents do not actually prevent people from continuing with research and actually they kind of serve like vaccines. A vaccine makes your body a little bit sick so you can handle the big disease later, much better. It’s the same here. People will point out, “You know that AI accident we had where 12 people died,” everyone’s still here, 12 people is less than smoking kills. It’s not a big deal. So we continue. So in a way it will actually be confirming that it’s not that bad. It matters how the deaths happen, whether it’s literally murdered by the AI system, then one is a problem, but if it’s accidents because of increased reliance on automation for example, so when airplanes are flying in an automated way, maybe the number of plane crashes increased by 17% or something, and then you’re like, “Okay, do we really want to rely on automation?” I think in a case of automation airplanes, it decreased significantly. Okay, same thing with autonomous vehicles. Okay, what are the pros and cons? What are the trade-offs here? And you can have that discussion in an honest way, but I think the kind of things we’re talking about here is mass scale pain and suffering caused by AI systems, and I think we need to see illustrations of that in a very small scale to start to understand that this is really damaging. Versus Clippy. Versus a tool that’s really useful to a lot of people to do learning to do summarization of text, to do question-answer, all that kind of stuff to generate videos. A tool. Fundamentally a tool versus an agent that can do a huge amount of damage.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.