John Collison: evaluation

Transcript context

…I think we definitely could be. Again, back to the question of asking, "What problem does this solve?" The human is pretty good with their eyes at some of this stuff. The bar is pretty high in order to do some of the things you're talking about. As I think through AI applications for DoorDash, one thing I'm struck by is that the LLMs are pretty good recommenders, just by tossing things into context. You don't need to train a custom model. I don't know if you've ever tried for book recommendations or TV recommendations. You just tell it a bunch of the stuff you like already. Restaurant recommendations, it's like, "Here are 15 restaurants we like to go to. Give us other recommendations," and it'll do a really nice job. Yet within products, if I open DoorDash, it's the same categories, and it's less personalized to me than if I took my DoorDash history and put it into an LLM. Shouldn't we be somehow using the fact that LLMs are pretty good recommenders within products? This is not just DoorDash, it's every product I use, it feels like. Those recommender capabilities are underutilized. No, I think you're definitely right that there's an opportunity here where… There's almost like the traditional school of thought, which is to use the information that you have, and you build the best personalized models that you can. One of the things that LLMs obviously does is they throw efficiency out of the wall, and they throw as much compute towards it. Interestingly enough, one of the things that spits out when you put in enough tokens and have big enough context windows is you're right. It has actually better models because it's just using much larger data sets.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.