Ankur Goyal: evaluation

Transcript context

…I don't know. But anyway, he gave me this really good advice, which was, as a startup, you only get to make a few technology bets and you should be really careful about those bets. Actually, at the time, I was asking him for advice about how to make arbitrary code execution work, because obviously they've solved and in JavaScript, arbitrary code execution is itself such a dynamic thing. There's so many different ways of, there's workers and Deno and Node and Firecracker, there's all this stuff. Ultimately, we built it in a way that just supports Node, which I think Vercel has sort of embraced as well. But where I'm kind of trying to go with this is, in AI, there are many things that are changing, and there are many things that you've got to predict whether or not they're going to be durable. If something's durable, then you can build depth around it. But if you make the wrong predictions about durability and you build depth, then you're very, very vulnerable. Because a customer's priorities might change tomorrow, and you've built depth around something that is no longer relevant. And I think what's happening with frameworks right now is a really, really good example of that playing out. We are not in the app framework universe, so we have the luxury of sort of observing it, as intended, from the side. You are a little bit... I captured when you said if you structure your code with the same function extraction, triple equals to run evals. Sure, yeah.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.