Evidence receipt / belief
Published · transcript-backedShawn Wang: belief
5 Jul 2024 Latent Space The 10,000x Yolo Researcher Metagame — with Yi Tay of Reka
“I think always the way to go. Is it like a hundred experts, a thousand experts , for some reason the, the community settled on eight.”
Source trail
Everything needed to verify it.
- Speaker
- Shawn Wang
- Attribution
- Verified speaker
- Claim type
- belief
- Recorded
- 5 Jul 2024
- Publisher
- Latent Space
Transcript context
…Yeah. I think always the way to go. Is it like a hundred experts, a thousand experts , for some reason the, the community settled on eight. Oh, you probably get more gains from, from more, more than eight, I think. But like, I think in general it's like. MOE's are just a trade off with like, prime and flop, right? And then you're able to like, kind of, make, like, you kind of make that. That, that in like that, that scaling log increase from, from that additional. So you, you can keep a low flop but kind of have more parameters. It's just changing the flop parameter ratio. Mm-Hmm. Keeping in mind there's a lot of inefficiency…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.