Nathan Labenz: evaluation

Transcript context

…And then, should you ignore career capital and skill development? No, because if you could spend one year and make yourself 20% more productive, you would recoup that after roughly four or five years. So even if you only had a five year timeline, time horizon, you should still make those investments that will pay off over that period. And there's often a lot of things people can do to increase their impact by more than 20%. Yeah, retraining in machine learning, lots of people have done that. Or entering a policy career, you can often transfer into one in a year and then maybe have much more impact after that. So yeah, I still think it is really worth thinking about. what you can do to maximize your impact over that whole time horizon, but it's just a bit shorter than in the conventional world where maybe you have 40 years to play with and you can really invest for 10 or 20 years and then pay it all off in your 50s. Let's also talk for a minute about what you think people should be trying to move the needle on. AI is one of the premises of this show and one of the reasons I enjoy making it so much is that it's obviously a general purpose technology, a horizontal layer, something that kind of intersects with everything. And so increasingly like work in AI is almost saying nothing because it's like in any sector, in any business, there's going to be some sort of AI touch point soon, if not already. What do you think are the big levers that really matter most? And maybe give a little argument for each of those big focus areas. Yeah, I'd start with what problems to focus on and then you could talk about the best interventions or solutions to those problems. And again, here we would use the framework of which problems are most important, most neglected and most solvable. And with importance, I often, you can partly think, you can think like how likely is this problem, how big would the scope be if it happened. I also quite like to think about urgency. because some problems come before the others, so you need to address them first and then deal with the other ones afterwards. Or they might help you deal with other problems, so you want to deal with them earlier. And then, yeah, we have a list on the website that we're always updating depending on how the assessment of these factors is changing. But right now, yeah, we have loss of control of autonomous AI as the top, just because the stakes would be so big if This is loss of control of human level or beyond AI, which might be irreversible, could eventually result in total human disempowerment. And it is much less neglected than in the past, but we're still probably only talking about 1000 or 2000 full-time people working on these risks compared to now what is Exactly where you draw the circle around the AI industry is unclear, but could easily be 100,000 or a million people essentially working on AI capabilities and accelerating it even faster. So that, yeah, that still seems like a big neglected, important risk. But then over time, we broadened out other risks that we're focusing on. And one that is a bit newer to the list is concentration of power. So there's a bunch of ways AI could be very concentrating or very centralizing. One is if there is an intelligence explosion in a feedback loop. In exponential growth, the gap between the first project and the second project stays the same. It's if it's a two-month gap now and they both grow exponentially, it's still a two-month gap in a year's time or two years time. But if you go into hyper exponential, like super exponential growth, then the gap actually widens. So you could have one company drawing well ahead of the others. gaining a digital workforce that is equivalent to a whole nation's worth of people today, which would be like more power than any single company's ever had before. A bunch of other ways AI could be centralizing. We saw the Department of War were like ****** *** that they weren't being allowed to use Claude to like spy on every American. Yeah, that's a thing that wasn't technically possible in the past because they just didn't have enough staff to analyze all the data and like figure out what everyone was doing. But with enough AI, you could literally trawl through every bit of public data about people and probably build up quite a big picture of, quite a good picture of what they're doing. So it makes like surveillance much more effective than it's ever been in history, which would be great news if you're a wannabe dictator of any kind.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.