Speakers in the public record
Claim mix
evaluation 8belief 4recommendation 2preference 2commitment 1uncertainty 1prediction 1
Evidence policy
Every row below preserves an exact excerpt. Identified speakers are linked; unresolved voices are labeled and excluded from people counts.
Claim ledger
The useful parts, with receipts.
19 published records
“So right now we're instrumenting with our own product. So we're instrumenting groups of inferences and individual inferences, which means we can then create charts that show how often they fail, why they fail, how often we need to retry to get good answers.”
- Publisher
- Latent Space
“It's like, all right, compute all those embeddings. That's more work upfront because you have to go through your database of all of these things and you got to commit like that engineering work, but it's like you validate with the general model because it's just easy.”
- Publisher
- Latent Space
“It can be really scary. And that's another sort of mental model of framing we've been thinking around is we find LLMs right now are most useful either when you come from, either when you have a narrow input space and a broad output space, because you can be very, you know exactly what format of data, what kind of data is gonna be passed in.”
- Publisher
- Latent Space
“I always try to raise the awareness of this, that Copilot I think did a really interesting metric or North Star, which was how much code is kept or retained by the user.”
- Publisher
- Latent Space
“I think the most interesting thing about hackathons is you come with an idea and then you kind of hit a wall trying to build it.”
- Publisher
- Latent Space
“I don't know how to measure that. Except by thumbs up, thumbs down, which is a difficult measure in the first place.”
- Publisher
- Latent Space
“Now, when we talk about large behavioral models or like actually embedding events, there are many, many trillions of those. And yeah, Postgres probably doesn't work there.”
- Publisher
- Latent Space
“All right. With that, I think we're all gonna open up to questions. We're gonna toss you the mic.”
- Publisher
- Latent Space
“For me, one of the most challenging questions is something I think you guys have maybe thought about a lot which is chat.”
- Publisher
- Latent Space
“I think I've talked about this on the podcast, but this idea of like just-in-time UIs, you know, like each type of user wants to interact in a different way.”
- Publisher
- Latent Space
“And we hit a bunch of problems there with LLMs hallucinating, losing context, even within the context windows, not really sort of recalling everything within the context window. So we sort of did a bunch of experimentation and realized if we split this down to seven different questions, so instead of saying, generate me a chart and a query for this one question, let's split that into lots of sub queries, like what kinds of events should I use?”
- Publisher
- Latent Space
“Yeah, of course. It's funny that we talk about this because the start of Amplitude is actually almost more AI than the current state.”
- Publisher
- Latent Space
“Because the LLMs can do a lot more of the reasoning, but we can make sure that the context we're providing is much more specific and generally much better by using the traditional ML models.”
- Publisher
- Latent Space
“At the same time, I think the future of AI applications and doing really cool things with it will be in the data, partially because like, you know, ChatGPT has done such a huge advance, right?”
- Publisher
- Latent Space
“I don't know if I can share the actual customer, but actually from before the LLM days, where they literally generated the versions of the copy themselves, and they made their product basically adapt, you know, multi-arm bandit style of like, hey, here's all these different variations, like just go figure out the best one.”
- Publisher
- Latent Space
“Sure, yeah, happy to speak to that. So, you know, if we think about the fundamental thing that our customers of Amplitude try to do, it's they want to look at their product data and they want to figure out how do I make my product better?”
- Publisher
- Latent Space
“I think that would make everyone's lives a lot easier if you could somehow ask behavioral questions of data without having to instrument.”
- Publisher
- Latent Space
“I don't know what the constraints on our lightning round, but I think maybe creativity is the best word where it's, you know, with the image generation stuff, text generation, you know, one thing that still blows my mind, I used to be a competitive like math guy and like there's this international math Olympiad problem in one of the papers and it solves it.”
- Publisher
- Latent Space
“We do have some smaller, not even fine-tuned, some smaller models that we've sort of borrowed from Hugging Face that we run internally for more specific tasks. And that's often sort of selecting specific values before we pass it to a general model right now, just because the general models are much easier to communicate with and they understand most of the words we use.”
- Publisher
- Latent Space