Evidence receipt / evaluation
Published · transcript-backedYi Tay: evaluation
5 Jul 2024 Latent Space The 10,000x Yolo Researcher Metagame — with Yi Tay of Reka
“all the changes, all the changes that, like, Swiglu was this, like, okay, Swiglu is probably one of my favorite papers of all time, just because of the divine benevolence, like, the Noam (Shazeer) actually wrote, like we owe this success to divine benevolence, like, that was, like, it's always a meme thing, right?”
— Yi Tay
Source trail
Everything needed to verify it.
- Speaker
- Yi Tay
- Attribution
- Verified speaker
- Claim type
- evaluation
- Recorded
- 5 Jul 2024
- Publisher
- Latent Space
Transcript context
…like, we have to have learned something in the last, like, No, all the changes, all the changes that, like, Swiglu was this, like, okay, Swiglu is probably one of my favorite papers of all time, just because of the divine benevolence, like, the Noam (Shazeer) actually wrote, like we owe this success to divine benevolence, like, that was, like, it's always a meme thing, right? Okay, so, like, GQA, MQA was always, like, the multi career type, was always, like A big controversial thing because MQA usually you get a hit because it's MQA and everything so people kind of know that like it was a very hit or miss like it was like it could you could get a hit in a performance from MQA like MQA alone MQA was always like You know, the choice, right? It's always like, okay, should we use MQA, should we not use MQA, right? When GQ came in, right, it became like a no brainer to use GQA because you don't get the hit anymore, and then you just get the fast, like, inference benefits of GQA, right? So I think GQA I mean, 2 now. Yeah,…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.