Ilya Sutskever: belief

Transcript context

…And if you at SSI have 50 different ideas, how will you know which one is the next transformer and which one is brittle, without having the kinds of compute that other frontier labs have? I can comment on that. The short comment is that you mentioned SSI. Specifically for us, the amount of compute that SSI has for research is really not that small. I want to explain why. Simple math can explain why the amount of compute that we have is comparable for research than one might think. I’ll explain. SSI has raised $3 billion, which is a lot by any absolute sense. But you could say, “Look at the other companies raising much more.” But a lot of their compute goes for inference. These big numbers, these big loans, it’s earmarked for inference. That’s number one. Number two, if you want to have a product on which you do inference, you need to have a big staff of engineers, salespeople. A lot of the research needs to be dedicated to producing all kinds of product-related features. So then when you look at what’s actually left for research, the difference becomes a lot smaller. The other thing is, if you are doing something different, do you really need the absolute maximal scale to prove it? I don’t think that’s true at all. I think that in our case, we have sufficient compute to prove, to convince ourselves and anyone else, that what we are doing is correct. There have been public estimates that companies like OpenAI spend on the order of $5-6 billion a year just so far, on experiments. This is separate from the amount of money they’re spending on inference and so forth. So it seems like they’re spending more a year running research experiments than you guys have in total funding.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.