Ben Gilbert: belief

Transcript context

…They did a lot with Transformer-based large language models. After the paper came out, what they didn’t do was treat it as a wholesale technology platform change. They were doing things like BERT and MoM, this other model. They could work it into search results quality. I think that did meaningfully move the needle, even though Google wasn’t bragging about it and talking about it. They got better at query comprehension. They were working it into the core business, just like every other time Google Brain came up with something great. Perhaps one of the greatest decisions ever for value to humanity and maybe one of the worst corporate decisions ever for Google, Google allows this group of eight researchers to publish the paper under the title, Attention Is All You Need, obviously a nod to the classic Beatles song about love. As of today in 2025, this paper has been cited over 173,000 times in other academic papers, making it currently the 7th most cited paper of the 21st century. I think all of the other papers above it on the list have been out much longer. Also, of course, within a couple of years, all eight authors of the Transformer paper had left Google to either start or join AI startups, including OpenAI.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.