David Rosenthal: belief

Transcript context

…They were doing things like BERT and MoM, this other model. They could work it into search results quality. I think that did meaningfully move the needle, even though Google wasn’t bragging about it and talking about it. They got better at query comprehension. They were working it into the core business, just like every other time Google Brain came up with something great. Perhaps one of the greatest decisions ever for value to humanity and maybe one of the worst corporate decisions ever for Google, Google allows this group of eight researchers to publish the paper under the title, Attention Is All You Need, obviously a nod to the classic Beatles song about love. As of today in 2025, this paper has been cited over 173,000 times in other academic papers, making it currently the 7th most cited paper of the 21st century. I think all of the other papers above it on the list have been out much longer. Also, of course, within a couple of years, all eight authors of the Transformer paper had left Google to either start or join AI startups, including OpenAI. Brutal. Of course, Noam starting Character.AI, which, what do we call it? A hackquisition? He would end up back at Google via some strange licensing, IP, and hiring agreement on the few billion dollars order.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.