Evidence receipt / belief
Published · transcript-backedYi Tay: belief
5 Jul 2024 Latent Space The 10,000x Yolo Researcher Metagame — with Yi Tay of Reka
“Oh, you probably get more gains from, from more, more than eight, I think. But like, I think in general it's like.”
— Yi Tay
Source trail
Everything needed to verify it.
- Speaker
- Yi Tay
- Attribution
- Verified speaker
- Claim type
- belief
- Recorded
- 5 Jul 2024
- Publisher
- Latent Space
Transcript context
…I think always the way to go. Is it like a hundred experts, a thousand experts , for some reason the, the community settled on eight. Oh, you probably get more gains from, from more, more than eight, I think. But like, I think in general it's like. MOE's are just a trade off with like, prime and flop, right? And then you're able to like, kind of, make, like, you kind of make that. That, that in like that, that scaling log increase from, from that additional. So you, you can keep a low flop but kind of have more parameters. It's just changing the flop parameter ratio. Mm-Hmm. Keeping in mind there's a lot of inefficiency between the experts.…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.