Evidence receipt / uncertainty
Published · transcript-backedJonathan Frankle: uncertainty
20 May 2023 Latent Space MPT-7B and The Beginning of Context=Infinity — with Jonathan Frankle and Abhinav Venigalla of MosaicML
“Um, so Sasha better be very generous to me cause it's possible that could change, but at the end of the day, we're still doing Transformers the way, you know, Vaswani had all intended back six years ago now, so, I don't know, things.”
Source trail
Everything needed to verify it.
- Speaker
- Jonathan Frankle
- Attribution
- Verified speaker
- Claim type
- uncertainty
- Recorded
- 20 May 2023
- Publisher
- Latent Space
Transcript context
…Well, I suppose if, if we ever replace like the Qk multiplication, something that looks sort of like it, I, I wonder who, who, who comes out on top here. Yeah. I mean at the end of the day is a feed forward network, you know, that's fully connected, just a transformer with very simple attention. Mm-hmm. Um, so Sasha better be very generous to me cause it's possible that could change, but at the end of the day, we're still doing Transformers the way, you know, Vaswani had all intended back six years ago now, so, I don't know, things. Six years is a pretty long time. What's another four years at this point? Yeah. What do you think will replace it if you lose Ben? What do you think? You would've lost it time?…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.