Nathan Labenz: belief

Transcript context

…So this is not my area of expertise, so I don't have a lot of depth, but purely from talking to both the cloud providers and the model providers, when you start looking at what are all of the attack vectors and what is possible, it doesn't seem like this is a human factor is an issue. The way the systems are set up is inherently very secure. That said, in security, of course, you can never say, Hey, this is completely airtight and it can never be penetrated. Yeah, as I said, security is taken very seriously, and I don't think it's a human factor necessarily. The way the systems are set up is such that the execution environment doesn't have access to, you cannot do a lot with it other than just run inference through it. So by design, access to the weights are limited. Yeah. Okay, cool. Thank you. I'm always trying to get a little bit better read on that particular corner of the world, and it's not one that is as freely and openly as some of us curious minds might like. Going back to the models, though, themselves, what's your read right now on this is another thing where I think people have very different intuitions. Are the models going to be commoditized or are they going to be sufficiently differentiated as to maintain pricing power as we continue to go into the future? I guess there's that. There's like, how do you help customers decide which model to use for a given case? Do you have an evals platform built in or do you help them do evals? How do you help them think about being like keeping agile so they can switch. Obviously, new models are coming out all the time. So there could be something better or faster or cheaper that you could upgrade to, but you have to know with some confidence that you're going to be upgrading with good reason. There's a whole ball of wax there. Take your time in melting it, but... Yeah, super interesting. I mean, as you mentioned, the differences between these models is not large. Each model keeps getting better, and then there is great healthy competition out there between the model providers, which again benefits companies like ours, our customers, and so forth. For us, because we're providing the choice to customers, the second part of the question is also really important, which is how do we help our customers choose which model is the best fit for their needs? There's a couple of considerations. One is, for many customers, they do not want to leave, again, because of data residency requirements. If they are, for instance, an Amazon shop, and today OpenAI is available through Azure, or through directly OpenAI, that becomes a consideration. So some customers are okay with their data leaving that Amazon cloud boundary, others aren't. So that's one decision point. The second one is, of course, from a quality perspective. Many customers will go run evals side by side to decide which model is best suited for their needs. What's interesting is some of these models are cheaper, faster, but when you add reasoning on top of it, the equation changes, right? So, you know, certain models are very good at certain things. Again, just to call out, you know, Claude is incredibly good at coding and continues to be a great model for that. So we help our customers in assessing which models to use for their needs.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.