Daniel Kokotajlo: commitment

Transcript context

…Do you want to throw in any others now, or you can later in the conversation? I think that the type of thing that I'm probably going to end up advocating for is going to be more of a rather than, like, here's a line that we're all not going to cross, something more like, we are going to gradually develop AIs with these capabilities, but we're going to do it in a way that's, like, mutually transparent to each other and that proceeds sort of slowly and cautiously where we all debate whether it's safe to go to the next level. And then after we get there, we study it for a little bit and then debate whether it's safe to go to the next level and so forth. So the sort of thing that we're probably going to end up having query for is going to look something more like that rather than but yeah, in terms of the thing that you really need to stop from happening in the short term, that sort of recursive self improvement thing is, I would say, the number 1 thing. So this conversation is a little bit depressing in the sense that many of the things that we seem to be worried about actually seem fairly close. Maybe the person in the room who's most pessimistic if that's the word, let's not, most extended in time on this is me. But all of us I would say think that, well let me rephrase this question. It's a dark set of answers relative to the reality right now I think in the following sense. Even if you think it's going to take a while to get to AGI or ASI or something like that, and we'll talk about that in a little bit, the things that are red lines, and I like your red lines, people are already pushing against them. They may not be breaking through them, like depending on your definition of recursive self improvement, I might give you different estimates, mine might be a little longer than yours, but people are already trying to do that, right? And transparency is like 19 nineties talk. Like it's like it's in the rear, I'm exaggerating a little bit, but it's it's in the rear view mirror. I mean like OpenAI was open originally, it is not open anymore. So you know, there are elements still pushing for transparency, but there are certainly elements pushing against.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.