Andrej Karpathy: belief

Transcript context

…Quintin Pope had this interesting blog post where he’s saying the reason he doesn’t expect a sharp takeoff is that humans had the sharp takeoff where 60,000 years ago we seem to have had the cognitive architectures that we have today. 10,000 years ago, agricultural revolution, modernity. What was happening in that 50,000 years? You had to build this cultural scaffold where you can accumulate knowledge over generations. This is an ability that exists for free in the way we do AI training. In many cases they are literally distilled. If you retrain a model, they can be trained on each other, they can be trained on the same pre-training corpus, they don’t literally have to start from scratch. There’s a sense in which it took humans a long time to get this cultural loop going, but it just comes for free with the way we do LLM training. Yes and no. Because LLMs don’t really have the equivalent of culture. Maybe we’re giving them way too much and incentivizing not to create it or something like that. But the invention of culture and of written record and of passing down notes between each other, I don’t think there’s an equivalent of that with LLMs right now. LLMs don’t really have culture right now and it’s one of the impediments I would say. Can you give me some sense of what LLM culture might look like?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.