Leopold Aschenbrenner: uncertainty

Transcript context

…Why can't they figure that out, but not how to get the RL thing working? I don't know. Germany during World War II went down the wrong path with heavy water. There's an amazing anecdote in The Making of the Atomic Bomb about this. Secrecy was one of the most contentious issues early on. Leo Szilard really thought a nuclear chain reaction and an atomic bomb were possible. He went around saying, "this is going to be of enormous strategic and military importance." A lot of people didn't believe it or thought, "maybe this is possible, but I'm going to act as though it's not, and science should be open." In the early days, there had been some incorrect measurements made on graphite as a moderator. Germany thought graphite wasn't going to work, so they had to do heavy water. But then Enrico Fermi made new measurements indicating that graphite would work. This was really important. Szilard assaulted Fermi with another secrecy appeal and Fermi was pissed off, throwing a temper tantrum. He thought it was absurd, saying, "come on, this is crazy." But Szilard persisted, and they roped in another guy, George Pegram. In the end, Fermi didn't publish it. That was just in time. Fermi not publishing meant that the Nazis didn't figure out graphite would work. They went down the path of heavy water, which was the wrong path. This is a key reason why the German project didn't work out. They were way behind. We face a similar situation now. Are we just going to instantly leak how to get past the data wall and what the next paradigm is? Or are we not? The reason this would matter is if being one year ahead would be a huge advantage. In the world where you deploy AI over time they're just going to catch up anyway. I interviewed Richard Rhodes, the guy who wrote The Making of the Atomic Bomb. One of the anecdotes he had was when the Soviets realized America had the bomb. Obviously, we dropped it in Japan. Lavrentiy Beria — the guy who ran the NKVD, a famously ruthless and evil guy — goes to the Soviet scientist who was running their version of the Manhattan Project. He says, "comrade, you will get us the American bomb." The guy says, "well, listen, their implosion device actually is not optimal. We should make it a different way." Beria says, "no, you will get us the American bomb, or your family will be camp dust." The thing that's relevant about that anecdote is that the Soviets would have had a better bomb if they hadn't copied the American design, at least initially. That suggests something about history, not just for the Manhattan Project. There's often this pattern of parallel invention because the tech tree implies that a certain thing is next — in this case, a self-play RL — and people work on that and are going to figure it out around the same time. There's not going to be that much gap in who gets it first. Famously, a bunch of people invented the light bulb around the same time. Is it the case that it might be true but the one year or six months makes the difference?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.