Daniel Kokotajlo: belief

Transcript context

…Great question. So I don’t know, let me try to reason aloud here. So my wife and I talked about it for a while and we also talked with some friends and got some legal advice. One of the filters that we had to pass through was even noticing this stuff in the first place. I know for a fact a bunch of friends I have who also left the company just signed the paperwork on their last day without actually reading all of it. So I think some people just didn’t even know that. It said something at the top about “if you don’t sign this, you lose your equity”. But then on a couple pages later it was like, “and you have to agree not to criticize the company”. So I think some people just signed it and moved on. And then of the people who knew about it, well, I can’t speak for anyone else but A. I don’t know the law. Is this actually not standard practice? Maybe it is standard practice. Right? From what I’ve heard now there are non-disparagement agreements in various tech industry companies and stuff. It’s not crazy to have a non-disparagement agreement upon leaving, it’s more normal to tie that agreement to some sort of positive compensation where you get some bonus if you agree. But whereas what OpenAI did was unusual because it was like your equity if you don’t. But non disparate disagreements are actually somewhat common. So basically in my position of ignorance, I wasn’t confident that- I didn’t actually expect that all the journalists would take my side and I think what I expected was that there’d be a little news story at some point, and a bunch of AI safety people would be like, “grr, OpenAI is evil, and good for you, Daniel, for standing up to them”. But I didn’t expect there to be this huge uproar, and I didn’t expect the employees of the company to really come out and support and make them change their policies. That was really cool to see. It was kind of like a spiritual experience for me. I sort of took this leap, and then it ended up working out better than I expected. I think another factor that was going on is that it wasn’t a foregone conclusion that my wife and I would make this decision. It was kind of crazy because one of the very powerful arguments was, “come on, if you want to criticize them in the future, you can still do that. They’re not going to actually sue you”. So there’s a very strong argument to be like, “just sign it anyway and then you can still write your blog post criticizing them in the future”. And it’s no big deal. They wouldn’t dare actually anchor equity. Right? And I imagine that a lot of people basically went for that argument instead. And then, of course, there’s the actual money. And I think that one of the factors there was my AI timelines and stuff. If I do think that probably by the end of this decade, there’s going to be some sort of crazy superintelligent transformation, what would I rather have after it’s all over? The extra money or… Yeah. So I think that was part of it. It’s not like we’re poor. here’s going to be some sort of crazy superintelligent transformation, what would I rather have after it’s all over? The extra money or… Yeah. So I think that was part of it. It’s not like we’re poor. I worked at OpenAI for two years. I have plenty of money now. So in terms of our actual family’s level of well being, it basically didn’t make a difference, you know? Yeah. I will note that I know at least one other person who made that same choice.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.