High Signal Podcasts Evidence ledger
Method
Browse
← All source episodes

Dwarkesh Podcast / episode intelligence

Joe Carlsmith — Preventing an AI takeover

22 Aug 2024 39 published claims 2 attributable people

Speakers in the public record

Claim mix

belief 18evaluation 9uncertainty 6prediction 3commitment 1recommendation 1preference 1

Evidence policy

Every row below preserves an exact excerpt. Identified speakers are linked; unresolved voices are labeled and excluded from people counts.

Claim ledger

The useful parts, with receipts.

39 published records

01 / evaluation

Then when the children show some obviously fake opportunity to defect to the allies, it's not necessarily going to be a good test of what it will do in the real circumstance because it's able to tell the difference.

“Then when the children show some obviously fake opportunity to defect to the allies, it's not necessarily going to be a good test of what it will do in the real circumstance because it's able to tell the difference.”
Speaker
Joe Carlsmith
Publisher
Dwarkesh Podcast

11 / belief

Again, we then have to ask how janky is pleasure, how specific and contingent is the thing we care about in pleasure versus how robust is this as a functional role in minds of all kinds.

“Again, we then have to ask how janky is pleasure, how specific and contingent is the thing we care about in pleasure versus how robust is this as a functional role in minds of all kinds.”
Speaker
Joe Carlsmith
Publisher
Dwarkesh Podcast

14 / belief

"I'm inclined to think that utopia, however weird, would also be in a certain sense recognizable; that if we really understood and experienced it, we would see in it the same thing that made us sit bolt upright long ago when we first touched love, joy, beauty; that we would feel in front of the bonfire the heat of the ember from which it was lit. There would be, I think, a kind of remembering.

“"I'm inclined to think that utopia, however weird, would also be in a certain sense recognizable; that if we really understood and experienced it, we would see in it the same thing that made us sit bolt upright long ago when we first touched love, joy, beauty; that we would feel in front of the bonfire the heat of the ember from which it was lit. There would be, I think, a kind of remembering.”
Speaker
Joe Carlsmith
Publisher
Dwarkesh Podcast

15 / evaluation

With these models, at least so far, it doesn't seem to matter. They just get, "Hey, don't help people make bombs" or whatever, even if you ask in a different way how to make a bomb.

“With these models, at least so far, it doesn't seem to matter. They just get, "Hey, don't help people make bombs" or whatever, even if you ask in a different way how to make a bomb.”
Speaker
Dwarkesh Patel
Publisher
Dwarkesh Podcast

21 / uncertainty

We're starting off with something that has this intricate representation of human values and it doesn't seem that hard to sort of lock it into a persona that we are comfortable with. I don't know what changes.

“We're starting off with something that has this intricate representation of human values and it doesn't seem that hard to sort of lock it into a persona that we are comfortable with. I don't know what changes.”
Speaker
Dwarkesh Patel
Publisher
Dwarkesh Podcast

25 / belief

Obviously in some sense, when we talk about a good future, we need to be thinking, “What are all the stakeholders here and how does it all fit together?” When I think about it, the thing that matters about the lineage is this.

“Obviously in some sense, when we talk about a good future, we need to be thinking, “What are all the stakeholders here and how does it all fit together?” When I think about it, the thing that matters about the lineage is this.”
Speaker
Joe Carlsmith
Publisher
Dwarkesh Podcast

26 / uncertainty

I don't know if moral realism is the right word, but you mentioned the thing. There's something that makes hearts converge to the thing we are or the thing we would be upon reflection.

“I don't know if moral realism is the right word, but you mentioned the thing. There's something that makes hearts converge to the thing we are or the thing we would be upon reflection.”
Speaker
Dwarkesh Patel
Publisher
Dwarkesh Podcast

28 / prediction

I expect the US government to protect me, not because of its “motives,” but just because of the system of incentives and institutions and norms that has been set up.

“I expect the US government to protect me, not because of its “motives,” but just because of the system of incentives and institutions and norms that has been set up.”
Speaker
Dwarkesh Patel
Publisher
Dwarkesh Podcast

31 / evaluation

The most straightforward thing to focus on—I don't blame people for just talking about this one—is just the first one. It's quite robust according to our own ethics, when we think about in which context is it appropriate to try to exert various types of control, or to have more of what I call in the series "yang," which is this active controlling force, as opposed to "yin," which is this more receptive and open, letting go.

“The most straightforward thing to focus on—I don't blame people for just talking about this one—is just the first one. It's quite robust according to our own ethics, when we think about in which context is it appropriate to try to exert various types of control, or to have more of what I call in the series "yang," which is this active controlling force, as opposed to "yin," which is this more receptive and open, letting go.”
Speaker
Joe Carlsmith
Publisher
Dwarkesh Podcast

32 / evaluation

I agree with the sentiment of obviously approaching this situation with caution, but I do want to point out the ways in which the analyses we've been using have been maximally adversarial.

“I agree with the sentiment of obviously approaching this situation with caution, but I do want to point out the ways in which the analyses we've been using have been maximally adversarial.”
Speaker
Dwarkesh Patel
Publisher
Dwarkesh Podcast

33 / evaluation

They're almost like random number generators. They're not especially calibrated, but once in a while they'll be like, “Oh, this one weird philosopher I care about, or this one historical event I'm obsessed with has an interesting perspective on this.

“They're almost like random number generators. They're not especially calibrated, but once in a while they'll be like, “Oh, this one weird philosopher I care about, or this one historical event I'm obsessed with has an interesting perspective on this.”
Speaker
Dwarkesh Patel
Publisher
Dwarkesh Podcast

34 / recommendation

I don’t know practically if it works out that well. But that experience made me think that I should try to expand my horizons in an undirected way because there’s lots of different things you have to understand about the world to understand any one thing.

“I don’t know practically if it works out that well. But that experience made me think that I should try to expand my horizons in an undirected way because there’s lots of different things you have to understand about the world to understand any one thing.”
Speaker
Dwarkesh Patel
Publisher
Dwarkesh Podcast

35 / preference

There’s some way in which I'm trying to embody attitudes towards them that I hope they would embody towards me. It's unclear exactly what the ground of that is, but I really like the golden rule and think a lot about it as a basis for treatment of other beings.

“There’s some way in which I'm trying to embody attitudes towards them that I hope they would embody towards me. It's unclear exactly what the ground of that is, but I really like the golden rule and think a lot about it as a basis for treatment of other beings.”
Speaker
Joe Carlsmith
Publisher
Dwarkesh Podcast

36 / evaluation

To the extent that the reason we're worried about motivations in the first place, it’s because we think a balance of power which includes at least one thing with human-descended motivations is difficult.

“To the extent that the reason we're worried about motivations in the first place, it’s because we think a balance of power which includes at least one thing with human-descended motivations is difficult.”
Speaker
Dwarkesh Patel
Publisher
Dwarkesh Podcast

37 / evaluation

I'm curious where that's coming from because conventionally I think the thing matters because it's conscious and its conscious experience as a result of that pursuit matters.

“I'm curious where that's coming from because conventionally I think the thing matters because it's conscious and its conscious experience as a result of that pursuit matters.”
Speaker
Dwarkesh Patel
Publisher
Dwarkesh Podcast

38 / evaluation

I think if something is not conscious—and there's like a deep difference between conscious and unconscious—then I definitely have the intuition that there's something that matters especially a lot about consciousness.

“I think if something is not conscious—and there's like a deep difference between conscious and unconscious—then I definitely have the intuition that there's something that matters especially a lot about consciousness.”
Speaker
Joe Carlsmith
Publisher
Dwarkesh Podcast
Search evidence