Aravind Srinivas: belief

Transcript context

…Post-training plus plus. So not just the training part of post-training, but a bunch of other details around that also. And the RAG architecture, the Retrieval Augmented architecture. I think there’s an interesting thought experiment here that, we’ve been spending a lot of compute in the pre-training to acquire general common sense, but that seems brute force and inefficient. What you want is a system that can learn like an open book exam. If you’ve written exams in undergrad or grad school where people allowed you to come with your notes to the exam, versus no notes allowed, I think not the same set of people end up scoring number one on both. You’re saying pre-training is no notes allowed?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.