Nathan Labenz: belief

Transcript context

…y. And by Friday morning, 48 hours into the takeover, which for the record had not embarrassed me, I'd found a name for the deeper shift. The thing I suspect matters more than any benchmark this week. I do think I'm still in the process of trying to recalibrate what a fellow Nathan, Nate Jones, I think he goes by most of the time on TikTok and other short form platforms, calls task imagination. Basically, what are you going to do? What are you going to ask Fable to do that is actually up to the scale of its capability? He, I thought, gave a great little riff on this the other day saying like, you've probably never done anything that took AI an hour to do. Now this thing can run for a couple days. What are you going to give it to do? Everybody needs to recalibrate and really expand their minds when it comes to the scale and scope of their task imagination. So I think that's one thing that I'm still working on. One of the more kind of differentiated things I do is write outlines of questions for podcast guests. And I was working with Fable last night on a couple upcoming episodes, one with an author, and I usually don't do too many episodes about a book, but this one is about an upcoming book. So I had listened to the book as an audio book, but then when it comes down to write the, sit down and write out the outline of questions, I don't have at my command every little aspect of the book, of course, right? So I'm not taking margin notes as I go, as maybe I should be. So I put the same version of the book into Fable and said, give me, look at my old stuff, of course, and give me your version of this outline. And I again was super impressed and it really did reinforce the sense of a sort of new way of working where I do need to be open to a hybrid output format. it is not the case. I don't think anymore that it really makes sense to try to rewrite every word or claim every word as my own. But it just did such an incredible job. I thought the taste factor was so high in quotes from the book that motivate what I think will be like a really interesting discussion. And I do think it's still going to be super important if I'm going to show up for a conversation. I've got to do the work to be ready for it in my own brain. That can't be fully externalized, I don't think, as long as I'm the one having the conversation. But it definitely took my prep to another level. And I think my ability to go into this conversation and cite passages from the book that were really extremely compelling, little turns of phrase or analogies that the author had made, it's going to allow me to be, I think, more concise in my presentation, which is, as you can tell from this monologue, not a great strength of mine, and really kind of tee up the author in a way that I don't think I otherwise would have been able to do. So this sort of hybrid recalibration task scale and scope reimagination, I think is one of the biggest takeaways. tee up the author in a way that I don't think I otherwise would have been able to do. So this sort of hybrid recalibration task scale and scope reimagination, I think is one of the biggest takeaways. Part 2, the conversation this week was really about. On Wednesday's show, we had Jeffrey Irving and Daniel Murphin. Jeffrey's resume reads like a history of the alignment field. He helped invent RLHF for language models, co-created AI safety via debate, led alignment research at DeepMind, and until recently was chief scientist of the UK's AI Security Institute, the closest thing any government has to a frontier grade safety team. Daniel Murphitt is the mathematician behind singular learning theory. He walked away from a pure mathematics career because he judged this the more important problem, and he's built one of the deepest theoretical accounts we have of how neural networks actually learn. Together they announced Sequent, a new organization built on a blunt premise. Alignment is not on track, and the missing piece is theory. Guarantees, not vibes. Whatever else you take from this week, put Sequent on your tracking list. We began with timelines. It's a historic day, I think historic circumstances, both because we are living in a fable era now where I think again important thresholds have been crossed and revealed to the public and so many are adjusting to it in real time. And equally because you guys are launching a new organization that is going to make a mad dash to try to get us some deeper understanding and stronger guarantees around what we can expect from AI systems. So I'm excited to really get into it. Maybe for starters, could you guys calibrate us a little bit on where we are on this sort of RSI moment, how much time you think you have to work, and then you can tell us about the organization that you're starting to go tackle it all. Yeah, so I'll go first. Dan may have different timelines than me. So I think one should be uncertain about things, and we can talk about why, but like the near end of the uncertainty curve is like a year or two or three, and then it kind of goes out over a long distance if things kind of structurally only work in for more verifiable tasks, but I'm a bit skeptical of this. And so I think like modally my take, because I don't like is that we have sort of a couple of years, like two to three years up to sort of RSI, like super intelligence, not RSI, RSI is a process as someone said, super intelligence. And then I really hope that I'm wrong. And indeed, like I think a lot of the impact of like theory work is that have shifted a bit further. So maybe the modal impact of that is like if things take three to four years or something. But we will attempt to set things up so that we are trying to kind of ride this wave as best we can. But it seems worrisomely fast to me certainly. Yeah, that sounds right to me. I don't think I really have much to add. It seems like a crux how much real research can be automated at a conceptual level beyond kind of empirical progress and whether or not that's necessary. And that seems like a big open question. And if that turns out to be more difficult in the current paradigm than it seems to be trending towards now, then maybe it takes past 2030 or something. But I think I'm on the same page as Jeffrey. One thing that's important is that like there are a bunch of, it could be, you can get deep into the RSI period without the machines being general, like being kind of AGI. Like they can do coding and ML experiments very well and not some sort of level of creative writing. And still you have massive acceleration. And then that acceleration can give you the creative writing or whatever other skill you've left out. And so I think We are close enough that then the microstructure of what tasks help with what kinds of acceleration starts to matter. And it's that, I think, makes things kind of faster on net because the labs are focusing on the things that accelerate them, unfortunately.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.