Lazar Jovanovic: evaluation

Transcript context

…Oh, man. Essentially, these Venn diagrams of engineer designer, PM, they used to be very separate. Now, they're converging and people with a specific, with deeper PM, engineering design background, they can all do the same thing essentially. All the roles are converging. What a time to be alive. And it's so hard to predict exactly how this all goes, but it's fun to pontificate. I want to get back to when you get blocked. Speaking of elite engineers, in reality, you are still writing code using these tools. Sometimes code goes, things go wrong. Bugs are introduced. There's a weird database thing. There's some network issue. What do you do when you get stuck? Do you have a workflow you go through of unblocking yourself? Yes. Great question. And absolutely true. No matter how good of a plan you have in place, you're going to run into problems eventually. And I have a small little framework that I call, four by four, just again, analogies. Right? Four by four, if you have it on your car, you're going to get yourself out of the mud much easier than the other way around. So in that sense, four different ways to debug. Attempt one of each only once, and I'll explain why in the end. First one is, again, every tool is different. I'll reference Lovable's workflow, which is when something breaks, Lovable's agent is smart enough to say, "Hey, I made a mistake." It will label that message in orange and have this little button usually, which is called, try to fix. So your agent basically admits it made a mistake. You click on a button and most times, when it's a smaller issue, it corrects the course, fixes it. No problem. Right? Now, there are situations obviously, when the problem is a little bit deeper than that. Right? You click to try to fix, but the problem persists. And sometimes even the problem persists, but Lovable's agent is unaware that it persisted. So there's no more try to fix button. Lovable thinks everything's working, but in reality, it isn't. And the culprit there is usually, you're using a third party integration. You did not give enough context to Lovable what to observe and what to see. So it can't see that the problem exists because Lovable, Cursor, Claude Code, you name it, all of these tools are good enough today to fix any problem they're aware of. Again, awareness is the key here. Right? So when they're unaware of it, there comes the second part, which is, "Okay, I need to bring the awareness layer." And what I do there is I go and very simply open the preview sandbox dev environment of my app, whatever, try to run the function that's broken, right click, read the console log. Right? Every browser allows you to just go and read the console log. And a lot of times, it will record stuff. If it doesn't, you can prompt any tool and say, "Hey, I don't think you're seeing the problem. So instead of me yelling at you, let's find it together." Right? I think it's a problem with X, Y, Z. I want you to write console logs in relevant files so that we can monitor every step along the way. Let's just bring awareness layer into the equation. It writes the console logs, you rerun it. Guess what? Now you have a full history of everything that was happening. You copy that, you paste it inside your chat. 99% of the time, that's enough, that's already enough. AI is like, "Okay, got it, found it, fixed it." But then, there's situations when even that's not sufficient. So you're like, "Okay, I need to go even deeper." And that's where code reviews and evaluations come into play. My go-to tool today for that is Codex, OpenAI. What I do is any build that I do, I will export it to GitHub. Lovable allows you to own your code cursor as well. reviews and evaluations come into play. My go-to tool today for that is Codex, OpenAI. What I do is any build that I do, I will export it to GitHub. Lovable allows you to own your code cursor as well. All of these tools allow you to have a copy of the code that you can export to GitHub, and then import it into wherever you want to. So I use Codex since beta, like import it in there, and then I'm using an external tool. So I'm like, in the first try, if you remember, I used the tool and I was like, total vibes, I'm relying on the tool. Right? In the second try, I use myself as the awareness facilitator. In the third one, I'm using an external tool as a facilitator, which is like, I'll either connect to Codex and chat with Codex to then fix the problem in Lovable. Right? I don't allow Codex to make code changes for me. A lot of people will say, "Why don't you like, it's a good model?" I just don't know its agent well enough. I don't want to go and use a tool that I don't know how to steer. So I use it only for diagnostic purposes and I'll also do it manually. It's an old workflow that I had before Codex and before Claude Code, which is there's a tool called Repomix, which allows you to compress your entire code base into a single file. You download it, and then I upload it to Claude, just regular Claude or ChatGPT. And I'm like, "This is what I'm building. Read it and this is the problem that I have. These are the console logs." Again, it's almost like having an external consultant at that point. You're hiring help elsewhere because your team just can't handle it. Right? And then the fourth one is usually the best one, because another time when there are problems, it's my fault. Like, no matter how your ego is big, guys, that you're watching this, it's your fault. Trust me. You had a bad prompt. You premised your request in the wrong way. You just don't want to admit it or you can't remember that you did, but it's your fault. So again, in Lovable and in all these other tools, you can revert back. There's version control built into Lovable, Cursor, Claude Code. You go and say, "Okay, I tried these three things. I'm just going to take three steps back and I'm going to think about my prompt a little bit more." Take a couple breaths, go for a walk, have some coffee, come back with a clear mind and try again. Because guess what? AI is just writing code very fast and sometimes it stumbles on a very small rock and it only happens then and never again. So you just got to make the same request again. And usually, that just fixes the problem. It's just a snag. It's a syntax error. It's something minute. Right? And then I do the final thing, which is this. And this is the key one actually. When the problem gets fixed, I go into the chat mode and I ask Lovable, I say, "Okay, I needed to do four different things to fix this. How can you help me learn how to prompt you better so that next time I have a problem, we do it in one go?" 99% of the time, I get such a great answer that I don't have the problem of not knowing what to do next time. Right? Again, we all need to be aware and realistic.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.