Roman Yampolskiy: belief

Transcript context

…Just to clarify, so verification is the process of something is correct, it is the formal, and mathematical proof, where’s a statement, and a series of logical statements that prove that statement to be correct, which is a theorem. And you’re saying it gets so complex that it’s possible for the human verifiers, the human beings that verify that the logical step, there’s no bugs in it becomes impossible. So, it’s nice to talk about verification in this most formal, most clear, most rigorous formulation of it, which is mathematical proofs. Right. And for AI we would like to have that level of confidence for very important mission-critical software controlling satellites, nuclear power plants. For small, deterministic programs We can do this, we can check that code verifies its mapping to the design. Whatever software engineers intended, was correctly implemented. But we don’t know how to do this for software which keeps learning, self-modifying, rewriting its own code. We don’t know how to prove things about the physical world, states of humans in the physical world. So there are papers coming out now and I have this beautiful one, “Towards Guaranteed Safe AI.” Very cool papers, some of the best [inaudible 01:07:54] I ever seen. I think there is multiple Turing Award winners that is quite… You can have this one and one just came out kind of similar, “Managing Extreme-“ … one just came out kind of similar, managing extremely high risks. So, all of them expect this level of proof, but I would say that we can get more confidence with more resources we put into it. But at the end of the day, we’re still as reliable as the verifiers. And you have this infinite regress of verifiers. The software used to verify a program is itself a piece of program. If aliens give us well-aligned super intelligence, we can use that to create our own safe AI. But it’s a catch-22. You need to have already proven to be safe system to verify this new system of equal or greater complexity. You just mentioned this paper, Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems. Like you mentioned, it’s like a who’s who. Josh Tenenbaum, Yoshua Bengio, Stuart Russell, Max Tegmark, and many other brilliant people. The page you have it open on, “There are many possible strategies for creating safety specifications. These strategies can roughly be placed on a spectrum, depending on how much safety it would grant if successfully implemented. One way to do this is as follows,” and there’s a set of levels. From Level 0, “No safety specification is used,” to Level 7, “The safety specification completely encodes all things that humans might want in all contexts.” Where does this paper fall short to you?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.