Public evidence record
Published podcast speaker
Books, apps, and tools
paper / likes
“I really like Rich Sutton's paper that he wrote about the Bitter Lesson and the Bitter Lesson effectively is this nice one-page paper but the essence of it is you can try lots of approaches, but the two techniques that are incredibly effective are learning and search.”
service / likes
“One of the things I like about Google is our ambition has always been sort of something that would require pretty advanced AI. Because I think organizing the world's information and making it universally accessible and useful, actually there is a really broad mandate in there.”
Claim ledger
7 transcript-backed records
01 / evaluation
“I, I think once it hits kind of 95% or something, you get very diminishing returns from really focusing on that benchmark, cuz it’s sort of, it’s either the case that you’ve now achieved that capability, or there’s also the issue of leakage in public data or very related kind of data being, being in your training data.”
02 / evaluation
“Uh, because I think everyone sort of sees that the models, you know, are great at some things and they fall down around the edges of those things and, and are not as capable as we’d like in those areas.”
03 / evaluation
“Um, what happens if traffic were to double or triple, you know, will that system work well? And I think a good design principle is you’re going to want to design a system so that the most important characteristics could scale by like factors of five or 10, but probably not beyond that because often what happens is if you design a system for X.”
04 / evaluation
“I mean, I think one of the things that is quite nice about the Flash model is not only is it more affordable, it’s also a lower latency. And I think latency is actually a pretty important characteristic for these models because we’re going to want models to do much more complicated things that are going to involve, you know, generating many more tokens from when you ask the model to do so.”
05 / evaluation
“The architectural improvements in multi-core processors and so on are not giving you the same boost that we were getting 20 to 10 years ago. But I think at the same time, we're seeing much more specialized computational devices, like machine learning accelerators, TPUs, and very ML-focused GPUs, more recently, are making it so that we can actually get really high performance and good efficiency out of the more modern kinds of computations we want to run that are different than a twisty pile of C++ code trying to run Microsoft Office or something.”
06 / evaluation
“Well, I would say that the pivot to hardware oriented around that was an important transition, because before that, we had CPUs and GPUs that were not especially well-suited for deep learning.”
07 / evaluation
“I think one of the things we were a little, our view of things from a search perspective was these models hallucinate a lot, they don't get things right a lot of the time- or some of the time- and that means that they aren't as useful as they could be and so we’d like to make that better.”