Latent Space / episode intelligence
Speakers in the public record
Claim mix
Evidence policy
Every row below preserves an exact excerpt. Identified speakers are linked; unresolved voices are labeled and excluded from people counts.
Claim ledger
28 published records
01 / belief
“There's a beautiful picture which I think Naveen tweeted out, which, um, shows a long boy on a whiteboard.”
02 / belief
“You know, we mentioned some of the podcasts you had done, Jonathan, I think in one of them you mentioned Mosaic was not planning on building a model and releasing and obviously you eventually did.”
03 / belief
“I think there's like a joke going on online that they're like long boy and that that came up internally because we were training them for really, really long durations.”
04 / belief
“Um, but you know, he had to let that one out publicly. And then I believe there was a YouTube video where someone happened to see it mentioned before the model came out and called it the Long G boy or something like that.”
05 / belief
“I think fast training in inference was like one of the goals, right? So there's always the trade off between doing the hardest thing and like.”
06 / belief
“You know, so stay balanced and follow the science. I think that's really, that's what we try to do around Mosaic.”
07 / uncertainty
“Um, so Sasha better be very generous to me cause it's possible that could change, but at the end of the day, we're still doing Transformers the way, you know, Vaswani had all intended back six years ago now, so, I don't know, things.”
08 / belief
“Um, I think, uh, basically it's sort of, we can help you figure out what model I should train to scale up so that when I go for my big run company, your here run, it's, uh, it's predictable.”
09 / belief
“I, you know, I even, even though I think there are papers that, you know, did not hold up under scrutiny, I don't think any of this was out of malice.”
10 / uncertainty
“I haven't actually tried any of these models, but I, I don't know. It seems like a very educational thing.”
11 / belief
“We have to know that we can do something great if customers are gonna do something great. And one too many people may have challenged me on Twitter about the fact that, you know, mosaic claims all these amazing numbers, but, you know, I believe not to, you know, call out Ross Whiteman here, but, you know, I believe he said at some point, you know, show us the pudding.”
12 / uncertainty
“There's been a debate recently about the emergence phenomenon, whether or not it's a mirage, right? I don't know if you guys have opinions about that process.”
13 / belief
“Um, that's really, if I learned one thing teaching at a law school, it was that I'm not a lawyer and all this stuff is a little complicated, especially open source licenses were not designed for this kind of world.”
14 / belief
“I think I'm, I'm still in my heart, the second year grad student who was jealous of all his friends who had GPUs and he didn't, and I couldn't train any models except in my laptop.”
15 / belief
“Cuz I, I think that, um, that is something that as a research community, as people interested in progress, we need to see these things instead of just, uh, seeing marketing papers from the advertising GPT 4.”
16 / belief
“I think, um, you can actually run just normal tensor flow and pie torch on there.”
17 / belief
“Um, and it's, I think, one of the commenters, you know, I really appreciated this said, you know, well, they're trying to do the right thing, but nobody knows what the right thing is to even do, you know, the, I guess the, the most right thing would've been to literally not release a model at all.”
18 / belief
“I mean, I think like, you know, loneliness mental health is a really a big problem everywhere. And so the most interesting I think I've found is that if you go to the subreddits, you know, for those communities and you see like how they talk about and think about their like AI friends and like these characters, it's, it's, it's like out of a science fiction book, like I would never expect this to be like reality.”
19 / prediction
“Um, I think the biggest thing about CEREUS is that they're really building, you know, kind of the NextGen computing platform beyond, like GPUs.”
20 / evaluation
“I think you've done a few podcasts about it over the years, which has been highly influential, and we'll talk about sparse models at Mosaic.”
21 / evaluation
“And I, I do think the vibe based eval cannot be, you know, underrated really at the, I mean, at the end of the day we, we did stop our models and do vibe checks and we did, as we monitor our models, one of our evals was we just had a bunch of prompts and we would watch the answers as the model trained and see if they changed cuz honestly, You know, I don't really believe in any of these eval metrics to capture what we care about.”
22 / preference
“I don't like to follow the hype. I like to see what sticks around, what people actually manage to build off of.”
23 / evaluation
“Is training on words of fair use because you don't have a monopoly on words, but some certain arrangements of words you do.”
24 / commitment
“We're not gonna train on it. But, we don't like to do that currently because, um, it makes us give up determinism, stuff like that.”
25 / prediction
“Yeah. So I think, um, this is probably one of the most exciting years, I think for training efficiency, just generally speaking, because we have the, the combination of a couple things, like one is like this next generation of hardware, like the H 100 s coming out from Nvidia, which on their own should be like, at least like a two x improvement or they 100 s on top of that, there's also a new floating point format f P eight, um, which could also deliver that alone.”
26 / commitment
“Our line two defense was that we used determinism and basically really smart resumption strategies so that if something catastrophic happened, we can resume very quickly, like a few batches before.”
27 / evaluation
“Um, boy inference, that's a whole second thing, and a whole different animal from training. And at the end of the day, when we founded the company, our belief was that inference was relatively well served at that time.”
28 / commitment
“We're very small and I, I can't promise we're gonna do what those labs used to do in terms of scale or quantity of research, but we will share what we learn and we will try to create resources for the community.”