Lazar Jovanovic: evaluation

Transcript context

…Two of the, I think, concerns maybe traps people that don't have a technical background fall into in theory, one is if you get blocked, it's not obvious how to solve a problem. And two is just, are you building this teetering slop that will collapse someday because you don't know system architecture, you don't know if this is going to scale, those sorts of things? So coming back to what you've learned about how to be successful and build successful products, talk us through just things you've done and things you've learned for how to avoid those sort of things and what you do when you get stuck as one example. I'm happy that you mentioned those limitations. I have some other ones that I want to bring in, but let's address this one first, which is the most important one. And that is you have to be self-aware. I didn't come into this... Yes, I am delusional, as I mentioned, in the sense that I just don't want to accept something's not possible, but I'm also well aware that I need to be better in order for it to become a reality from my own point of view and my own sake. So I understood very early that coding is not the problem that we're solving for here, that the problem we're solving for is clarity. The output that AI can do is much faster than human output anyways. So very early on, I started leveraging chat mode. And to this day, I can say I spent 80% of my time in planning and chatting and only 20% in executing the plan actually. I'm optimizing for the right kind of speed. Most people optimize for the wrong one. That's the first lesson that I learned literally on day two, because I came into Lovable, that was my first exposure to this. I've tested and played around with all the tools, obviously, but whether somebody's doing it in Cursor or Claude Code, doesn't matter where you are, the problem remains the same. You need to be clear on what you want to do and you need to know what you're doing because these are still just tools. Yes, AGI is coming, but it's not there yet. So until it's here, you're still steering the ship. In order for you to steer the ship, you have to know the instructions, right? And the best way to learn is by building, but treating these tools almost as technical co-founders and educators, and learning while doing, and religiously reading the agent output. Not the code output. I don't care about the code. The syntax is none of my interest. It's what the agent tells me that matters to me. I put a lot of trust in LLMs and AI these days, and I understand that there may be some people that are not as confident as I am. I just feel that the models today are good enough for me to trust in their syntax output. However, I'm concerned about the agent output because of the two limitations that I want to tackle on next. The first one being that there's a limitation when you work with LLMs. So there's a machine level limitation and there's a human level limitation. The first one is there's something that is known as the context memory window. And for non-technical people, I like to use the Aladdin and the Genie analogy when I explain. It's very simple. Everybody knows the storyline. You rub the lamp, a genie comes out and tells you, "Okay, I'll grant you three wishes. Not 3,000 wishes, not three million, just three at a time." ain. It's very simple. Everybody knows the storyline. You rub the lamp, a genie comes out and tells you, "Okay, I'll grant you three wishes. Not 3,000 wishes, not three million, just three at a time." To me, when I translate it into working with AI, that simply means, "Hey, I can only make so many requests within a request at a time for AI to be able to listen, understand what it needs to do, scope it, do the research, read, take all the actions, all the inputs and ingredients that it needs to produce a high quality output." So that's the first part, understanding that there's a limit and it's denominated in tokens. Maybe that's going to be different a year from now, but today there's a token limitation. I'll take an arbitrary number of 100,000 tokens, for example. So when you make a request, a part of those tokens AI spends to read stuff, another to browse the web, another to think, and then another to execute the code. Then there comes the second limitation, which is you, me and you, humans, which is, let's go back to the analogy of the Genie and the Aladdin. I asked the Genie for the first wish, and the first wish is I want to be taller. And guess what happens? Genie makes me 13 feet tall. All of a sudden, I can't sit in the car, I can't get into my house. I'm a dysfunctional human being because I was not specific. So the part that we need to optimize for today, it's going to get better, but today it's still not there yet, is that AI just don't understand what do you mean when you say, "You know what I mean?" You do when I tell you that. We as humans, I'm 36, so I have 36 years of experience of living as a human to know what you mean, but AI doesn't have that. So you need to be specific, you need to provide references, you need to provide the right context. So what I've learned is how to combat that part. And I think because I can't control the first part, which is the token memory window, the quality of the LLM models, you are 100% control of the latter. And that's what I want to dive into today as well and just try to teach people, "Okay, if I'm the malleable part, how do I fix that part?" I think that's the key lesson here.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.