Alex Komoroske: belief

Transcript context

…What I want to do instead is pick on some of the Bits and Bobs that you've been focusing on noodling on that I think are going to be really helpful to listeners in how they think about product and the future of AI and all these things that I think are emerging. And the first one is actually that I want to get your thoughts on how AI and LLMs are likely to impact product development. A lot of the listeners of this podcast are PMs, engineers, founders, people building software, and you're spending a lot of time thinking about AI and product development. A lot of your Bits and Bobs have been just like, "Here's what's happening. Holy shit. This is how things are going to be." So let me just ask you this how do you anticipate LLMs and gen AI are going to change how products are built in the next three to five years? I think they change a lot. I think LLMs are truly a disruptive technology. In fact, I would argue that what we're seeing in the industry is us trying to use mature playbooks from the end stage of the last tech era in one that doesn't really fit yet. To me, LLMs are magical duct tape. They're formed principally by the distilled intuition of all of society into a thing that operates between, a cost structure between human and plain old computing. So much of how the industry is built presupposes the idea that software is expensive to write and cheap to run, and LLMs undermine both of these. So it makes it LLMs allow writing shitty software to be significantly cheaper, not necessarily good software, but good enough in certain contexts. And also it means that there's certain software now that isn't plain old computing that can be run cheaply. It's relatively expensive marginal cost. And so if you're going to do a consumer startup, it can't be based on advertising. It's just too expensive. Advertising cannot clear the inference costs even with inference costs declining. So I think the way to me, a disruptive technology changes tons of stuff, all these assumptions you didn't even realize you were making because you didn't realize it could be any different. Imagine if you were locked in a room for your entire career, no windows, and you have all these experiences. You're going all this know how. You're getting this sense of how things work, what will happen. And then imagine that room tilts on its axis by five degrees. Everything looks roughly the same, and yet now the dynamics of the force of gravity is pulling in a different direction than it was before. You didn't even think about the force of gravity before because it was so omnipresent. It never changed that it's just a blank in your head. And now gravity has changed effectively for your perspective and all kinds of intuition is now wrong. I put this thing on the table, it's going to stay there, and then it slides off and falls into the wall. All kinds of weird stuff will happen. And I think LLMs to me feel like that. I can't tell you the number of people who are... At some point at a year or two, someone came to me, they're like, "I just built a prototype, product that would've taken me three months. And I can believe it's going to start up." I was like, "How differentiated do you think that is?" Everybody can do that now. It changes the basis of competition. I think today I see a lot of folks using LLMs, and LLMs are like a squishy computer. We're used to computers doing exactly what we told them to do, which is not necessarily what we meant. And only some people have learned the skill of programming, the arcane magical incantations to make computers do exactly what you meant. Now, LLMs can do all kinds of stuff. And they don't do exactly what you told them, but they do typically do roughly what you meant. ne magical incantations to make computers do exactly what you meant. Now, LLMs can do all kinds of stuff. And they don't do exactly what you told them, but they do typically do roughly what you meant. I see all these places where people will build products and they'll say 80% of the time, 90% percent of the time, it's great. 5% of the time it punches the user in the face and they're like, "Oh, we're going to reduce the number of times it punches us in the face." It's like even if you get it down to 99% of the time, it's fine. If it punches in the face, that's not a viable product. And so how do you design your products assuming that this thing will be squishy and not fully accurate and fully work? People use these things a lot as oracles. "I'm going to have it. I'm going to formulate the answer and it's going to be a fully fledged answer." And of course, strawberry has been really stabbed at. Gotten a chance to play with it. They are getting better at some of these kinds of behaviors at great expense. But in a lot of cases, I instead would rather say, "How can you take LLMs for granted? How can you assume that you now have this magical duct tape? Don't assume it's going to solve all your problems. Don't assume it's going to do autonomously be able to give high quality results of every case. But what can you now build now that you have magical duct tape?"…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.