Jungwon Byun: belief

Transcript context

…Cool, interesting. Everybody's doing their own experiments in recursive self-improvement these days. That's what I'm noticing across the board is there's always this, not always, but it's been striking that in the last, I don't know, 6 to 8 weeks, it seems like everybody's tipping into this moment of, maybe we too can be an experiment in recursive self-improvement. So you're now. elicit as a test case for can we get elicit to build effective world models and then be able to from its own, use these detailed representations that it itself has constructed to inform its own analysis of what it itself should become in the future. The hall of mirrors there is deep and fascinating. This obviously relates to a blog post that you put out not too long ago, which is called Planning is Unsolved. And I think, obviously that's totally true. Or we could just ask the AIs to handle all this and retire to the beach as I think it was an Anthropic person once famously put it. I do wonder a little bit, and this is probably a cultural question as much as it is a technology question, but when I think about like my own company that I'm now just the AI advisor to and not running, And I think about our planning process that we developed before AI. Sometimes I'm like, maybe we should just scrap the whole thing. Maybe all these times that we like come together and sit around the table and talk about this or that and try to convince each other that if we do this, it'll be more successful than if we do that. I'm often like, man, what we should do is build it all, launch all these things and see what happens. We can like coding has gotten cheap. So maybe the future of planning is less about guessing and more about like fast iteration and actually like making more contact with reality. Obviously, again, there's not like a true strict binary there. But how are you guys thinking about that question. And how are your clients at, especially like pharma companies thinking about that question? There's also this, I think it's more of an aspiration than a trend at this point, but there's the notion of clinical trial abundance, which sounds great. I don't think we're, again, I don't think we're there, but you can imagine a very different vision for a pharmaceutical company where one is like, we're gonna use these world models, we're gonna make much better decisions, and we're gonna we're going to deploy these like scarce resources and these like precious few at bats we have at clinical trials in the best way possible. And then there's this other vision that's we'll do 10 times as many clinical trials and that'll be, or 100 times, who knows, and that'll be bigger on lock because we'll actually get the real answers in far more cases. Where do you want to be on that spectrum? Where do you think pharmaceutical companies should be on that spectrum? Maybe it's different. Maybe Waymark should be one place and pharma should be somewhere else. Yeah, I think it's great that the cost of software engineering has come down, but it feels like that was just one of the bottlenecks and the others haven't moved. I don't think user attention or feedback is infinite. So I still feel cautious about just throwing things out there and giving people bad experiences or leaving people with bad impressions. And I also think that the cost of It's definitely possible to get stuck on a local max. I would worry about that a bit. Let's say you just ship a feature, you're like, oh, cool, this works. Let's just keep going. And you might be able to keep going for some time, but then still end up maintaining, improving, and investing in something that wasn't the best possible thing you could have done. So I think there's still a lot of room for judgment and purpose. And there are places where I'm not sure the shape of the problem has changed that much. I feel like there's always an explore-exploit trade-off, and you want to navigate those two thoughtfully, and sometimes you want to do one, sometimes you want to do the other. And maybe the cost of exploring has gone down a little bit, but I think there are still some things where you know you have to get it right the first time, or it's still, it's not all software engineering has literally gone to zero. There are still large software engineering projects. So I'm not sure it's changed. that it's only, I feel like for now, it's mostly changed things at the margins, like fixing bugs, small kind of admin features, things that we don't see as our core capability that we want to fully automate and are happy to take liberal experiments with. And then things that we see as being like core to our mission and our purpose and differentiator as a product still involve a lot of careful thinking and intent. And then with our customers, I find that many of them are just like, yes, there's a lot of excitement to build. I guess maybe like unsurprised, my unsurprising take is that, okay, my take on the build versus buy problem is you should build internally the things that are your core comparative advantage and basically nothing else. And if you're building something, for example, there are certain workflows that are just regulated across the industry. Every company has to do them pretty much the exact same way. And they are very particular. And as a result, they're very interface heavy. And I just don't think it's the core competency of pharma companies to design nice software. And that's not your comparative advantage as a company is not going to come from solving this regulatory compliance problem. So I don't think, for example, systematic review, which solves is one of those where I just don't think it makes sense for pharma companies to try and build this thing that every company has to do the same way and is actually very involved. c review, which solves is one of those where I just don't think it makes sense for pharma companies to try and build this thing that every company has to do the same way and is actually very involved. Other types of certain, especially in the early stage research or even in development, certain kind of predictive models make sense that a pharma company would want to build in-house. And then on the clinical trial point, I think why not have both? And I think we'll try, there's a lot of interest in kind of digital twins and simulating trial effects digitally as much as possible to draw, design the trial well. And I'm sure there are certain trials where with the right regulatory framework and kind of operational improvements, we might be able to take a lot more bets. I think especially like in rare diseases where you have a kind of, where a trial is actually almost like a treatment option, being much more flexible there. And then I think in other domains, depending on the type of drug and what we already know about, it's toxicity profile would probably want to hold a higher bar. So again, my hope is that it's great to have multiple tools and options and choices. And so maybe we can just build more tools, make the tools better, and then build a good framework for like when you reach for what tool?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.