Eric Glyman: evaluation

Transcript context

…It's really interesting. By the way, one of the other interesting sub-conversations I'm hearing a lot too is when you think about where tech debt comes from, there's a set of conditions and trade-offs you make. You write things in a discrete and deterministic way. Under these circumstances, follow this code path, under this circumstance, follow this other one. In a world where there's LLMs and kind of the models themselves are improving, I think it's entirely possible that the way that code is written is you say, “Here's what I'm solving for. Under these kinds of conditions, here's what I want to occur. Go write the code that drives this outcome.” And maybe with the models such as they are today can get it done in sort of a spaghetti code written fashion. But it kind of works even though it's some kind of Rube Goldberg machine underneath the hood. But as these models get much smarter. You might write your code base in such a way where you say, “Every year, rewrite the underlying code, but here's the outcome I can drive.” And today you accomplish my outcome 90% of the time, 95, 98, 99, 100. And do you just have kind of self-healing and writing code all the way through? And this notion of underlying code does go away because we're writing things in this different manner. Obviously there's real conditions in which that won't occur. If you're writing code that needs to have four nines of accuracy and uptime, that's probably not the methods you're using. But if you're a growth engineering team, that's probably what you're doing today if you're kind of on the leading edge of this stuff. And so, I find this stuff very fascinating. When I think about the deeper implications, I think that if you are completing a small amount of cognitive work… Let's say you just are an expense app. The spend has occurred, it needs to go get the… Write it down somewhere and get someone's approval. And that's all the knowledge work that you're doing. That's very few tokens in order to accomplish that, both to create the infrastructure in order to facilitate it, that probably evaporates. I think anyone can probably custom write that kind of app. Whereas if you're doing much deeper kind of work, such to underwrite a company, provide financing, automate areas of accounting, I think the fitness function for companies becomes can you actually do things in such a way where even if you could spend tokens on it, it would take more tokens to create the thing or do that work than the system maybe that you've built to drive that outcome. I just think the rules of what it means to be a software company are changing. I think network effects and what you're building is more important than ever. If you think about the classic set of moats that are there, I think that this question of where are their moats in a world where… I like Dario’s way of putting this, if you have a country of geniuses living somewhere and they're writing code and they're incentivized to compete with you, too. ere are their moats in a world where… I like Dario’s way of putting this, if you have a country of geniuses living somewhere and they're writing code and they're incentivized to compete with you, too. If you're not really following what are the classic four moats and building towards that I think life gets a lot harder. Yes. Do you guys have a house view on the new competitive equilibrium with a lot of AI engineering really working?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.