Yi Tay: recommendation

5 Jul 2024 Latent Space The 10,000x Yolo Researcher Metagame — with Yi Tay of Reka

“I think inefficiency misnomer was like, we found that a lot of people, like, they use params, like, especially, like, like, to the kind of, like, right, and then MOEs was not very hot, like, in the community at that time, right, but MOEs were, like, a thing long ago. So I think using active params, I'm comfortable with using active params to kind of approximate like cost of the model, but like in the efficiency misnomer paper, we actually made it quite clear that you should always look holistically about like, because you have serving, like additional serving costs, like fitting in the GPUs, like fitting on single node, and something like that.”

— Yi Tay

Source trail

Everything needed to verify it.

Speaker: Yi Tay
Attribution: Verified speaker
Claim type: recommendation
Recorded: 5 Jul 2024
Publisher: Latent Space
Episode: The 10,000x Yolo Researcher Metagame — with Yi Tay of Reka

Transcript context

…Yeah, thanks, thanks. And also, I mean, I'm interested in efficiency that works. It's just very hard to find efficiency that works. And so, like, anything that helps me have high signal on efficiency is helpful. So I think for the inefficiency misnomer, by the way, I love the paper, by the way, it's quite a fun time working on it. I think inefficiency misnomer was like, we found that a lot of people, like, they use params, like, especially, like, like, to the kind of, like, right, and then MOEs was not very hot, like, in the community at that time, right, but MOEs were, like, a thing long ago. So I think using active params, I'm comfortable with using active params to kind of approximate like cost of the model, but like in the efficiency misnomer paper, we actually made it quite clear that you should always look holistically about like, because you have serving, like additional serving costs, like fitting in the GPUs, like fitting on single node, and something like that. The interesting one was speed. Nobody really talks about speed, but your paper actually talks about speed.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.

Open original source ↗ See episode intelligence