Alexander Meinke: preference

Transcript context

…And if AI ever needed slowing down, would we know when? The thing that makes this particular technology so difficult to reason about in terms of what would slow down and coordination for that even look like is that until things go really bad, they go really really great. Right? So if if we're right, then at some point, and we don't know when that would be, things get really dangerous and we should rather not continue developing. Until then, it is always true that just going 1 step further and 1 step further is actually really good in terms of progress, technology, it's yeah. Personally, I find it super exciting. That's exactly why we're trying to empirically study the emergence of the risks that we're worried about, because the theoretical arguments, they basically just tell you asymptotically, at some point, you should expect this to happen. But it is very unclear how exactly you get there, and there's 10 different mechanisms by which you might end up having a scheming AI, and we don't know if and when each of these mechanisms actually bites in practice. So this is why we're trying to develop this what we call science of scheming, where we empirically study given the the paradigm that we have now and forward chaining from here. If we keep scaling, what will happen next? And what will happen next? What are the things that we expect to get easier over time? What are the things that we expect to get harder over time? And having such an understanding is upstream of all the other good things that you want, including coordination. So right now, because nobody knows if and when, and nobody agrees if and when that danger zone actually arrives, We're just bottlenecked by scientific evidence, so this is what we're trying to gather. Guys, thank you so much for coming on today. It's been great. Thanks. What…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.