Sholto Douglas: observation

Transcript context

…Interesting. On the computer, your stuff, will it be end-to-end? Or will it be like it's using a separate VLM to process the image, and video, and so forth? I'm a bit of an end-to-end maxi. I think, in general, when people are talking about the separate model… For example, most of the robotics companies are doing this bi-level thing, where they have a motor policy that's running at 60 hertz or whatever, and some higher-level visual language model. I'm pretty sure almost all of the big robot companies are doing this. They're doing this for a number of reasons. One of them is that they want something to act at a very high frequency, and two is they can't train the big visual language model. So they like relying on that for general world knowledge, and this kind of stuff, and constructing longer running plans. But then they're like, you offload to the motor policy. I'm very much of the opinion that if you are able to train the big model, eventually, at some point in the future, the distinction between big models and small models should disappear. Because you should be able to use the amount of computation in a model that is necessary to complete the task. Ultimately, there's some amount of task complexity. You don't have to use 100% of your brain all the time. Welcome to my world.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.