Joe Carlsmith: evaluation

Transcript context

…Well, this actually seems related to your post on sincerity. Maybe I'm getting the vibe of the piece right. Certain intellectuals have a vibe of shooting the shit. They're just trying out different ideas. How do these analogies fit together? Those seem closer to looking at the particulars and like, “Oh, this is just like that one time in the 15th century where they overthrew this king…” Whereas this guy who was like, “Oh, if you look at the growth models from a million years ago to now, here's what's happening.” That one has a more sincere flavor. Some people, especially when it comes to AI discourse, have a very sincere mode of operating. “I've thought through my bio anchors and I disagree with this premise. My effective compute estimate is different in this way. Here's how I analyze the scaling laws.” If I could only have one person to help me guide my decisions on AI, I might choose that person. But if I had ten different advisors at the same time, I might prefer the shooting-the-shit type characters who have these weird esoteric intellectual influences. They're almost like random number generators. They're not especially calibrated, but once in a while they'll be like, “Oh, this one weird philosopher I care about, or this one historical event I'm obsessed with has an interesting perspective on this.” They tend to be more intellectually generative as well. I think one big part of it is that if you are so sincere, you're like, “Oh, I’ve thought through this. Obviously, ASI is the biggest thing that's happening right now. It doesn't really make sense to spend a bunch of your time thinking about how the Comanches lived? What is the history of oil? How did Girard think about conflict? What are you talking about? Come on, ASI is happening in a few years.” But therefore, the people who go on these rabbit holes because they're just trying to shoot the shit, I feel are more generative. It might be worth distinguishing between intellectual seriousness and the diversity and idiosyncrasies of one's interests. There might be some correlation. Maybe intellectual seriousness is also distinct from "shooting the shit." There’s a bunch of different ways to do this. Having exposure to various data sources and perspectives is valuable. It's possible to curate your intellectual influences too rigidly in virtue of some story about what matters. It's good to give yourself space to explore topics that aren't necessarily "the most important thing." Different parts of yourself aren't isolated. They feed into each other. It’s a better way to be a richer and fuller human being in a bunch of ways. Also, these sorts of data can be really directly relevant. Some intellectually sincere individuals I know who focus on the big picture also possess an impressive command of a wide range of empirical data. They're really interested in empirical trends, not just abstract philosophies. It’s not just history and the march of reason. They’re really in the weeds. There’s an “in the weeds” virtue that I think is closely related to seriousness and sincerity. There's a different dimension of trying to get it right versus throwing ideas out there. Some people ask, "What if it's like this?" or "I have a hammer, what if I hit everything with it?" There's room for both approaches, but I think just getting it right is undervalued. It depends on the context. Certain intellectual cultures incentivize saying something new, original, flashy, or provocative. There’s various cultural and social dynamics. People are being performative and doing status-related things. There’s a bunch of stuff that goes on when people do thinking. But if something's really important, just get it right. Sometimes it's boring, but that doesn't matter. Things are also less interesting if they're false. Sometimes there's a useful process where someone says something provocative, and you have to think through why you believe it's false. It’s an epistemic project. For example, if someone says, "Medical care doesn't work," you have to consider how you know it does work. There's room for that. But ultimately, real profundity is true. Things become less interesting if they're not true. It's possible to lose touch with that in pursuit of being flashy. After interviewing Leopold, I realized I hadn't thought about the geopolitical angle of AI. The national security implications are a big deal. Now I wonder how many other crucial aspects we might be missing. Even if you're focused on AI's importance, being curious about various topics, like what's happening in Beijing, might help you spot important connections later. There might not be an exact trade-off, but maybe there's an optimal explore-exploit balance where you're constantly searching things out. I don’t know practically if it works out that well. But that experience made me think that I should try to expand my horizons in an undirected way because there’s lots of different things you have to understand about the world to understand any one thing.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.