Tim Scarfe: belief

Transcript context

…of agency that you ascribe. Exactly. But then you're pointing to this other thing, right? So I think that right now, we have built systems that could potentially be intelligent and could potentially understand. I'm bullish. I think that even though there's this grounding problem, there are many things in the world that can be abstracted and essentialized and we could have this constructive form of understanding, you know, that goes several levels deep. And we can build agents that can do certain things for some reasonable amount of time. I'm bullish about that. But the thing is though, this agency is different. The doomers are saying that these things will have intrinsic goals. And it's called instrumental convergence. I'm sure you know about this. But basically, more kind of power and agency you have, you will get these convergent instrumental goals, and they will just want to kill all of us. They'll be power seeking, and they'll think of us as minnows. You know, there's that paper clip maximizer. Yeah. Because we're all trying to kill fucking minnows. Also, the paper clip maximizer is dumb as fucking rocks. What's the long term goal to to make paper clips? Know, like, isn't that fucking stupid? They're like, yeah. But we it wouldn't realize that. What? It can't abstract 1 level above its own goal?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.