Roman Yampolskiy: uncertainty

Transcript context

…Can you give an example? GPT-4. I don’t know what else it’s capable of, but there are still things we haven’t discovered, can do. They may be trivial, proportionate with capability. I don’t know it writes Chinese poetry, hypothetical, I know it does, but we haven’t tested for all possible capabilities and we are not explicitly designing them. We can only rule out bugs we find. We cannot rule out bugs and capabilities because we haven’t found them. Is it possible for a system to have hidden capabilities that are orders of magnitude greater than its non- hidden capabilities? This is the thing I’m really struggling with. Where, on the surface, the thing we understand it can do doesn’t seem that harmful. So even if it has bugs, even if it has hidden capabilities like Chinese poetry or generating effective viruses, software viruses, the damage that can do seems like on the same order of magnitude as the capabilities that we know about. So this idea that the hidden capabilities will include being uncontrollable is something I’m struggling with because GPT-4 on the surface seems to be very controllable.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.