Andrew Lee: belief

Transcript context

…Yeah, okay, that's interesting. So it's basically constant maintenance of the higher level summaries that will be fed into the LLM and then pretty short kind of single burst style caching to actually reduce the cost of incremental calls within like one agent run. And it sounds like at least for Anthropic, that kind of is typically limited to like the cache. is hit for one run, but not hit across runs for the most part. Yep, that's the current approach. One thing I wanted to note is the way our system is built today, we basically get no cache benefit across users. So it's like caching for, actually even per agent, like it's basically caching per agent. There are some changes that we can make to do a lot of cash optimization across agents and potentially even across organizations and users. And I don't want to get into the specifics of that because that's still upcoming, but I think there's a lot of potential actually to save money across agents as well. Yeah. Okay. Well, that'll be important. I do want to circle back to the OpenAI question because you guys have been clawed maximalists and you Your other mantra that rings in my head a lot is always bet on the models. I'd say it's safe to say that that bet has gone well over the last six months. Obviously we've seen some of the most notable model releases in the sense that the community has sort of flagged like four, five and four, six as kind of qualitative shifts where like things went from not working to working and people are like, oh, I can really get pretty general purpose knowledge work out of these things on a pretty consistent basis now. I would love to hear how you would characterize the advances that we've seen. Maybe you could do that in terms of like what new use cases have opened up, maybe, you know, things that have surprised you, possibly also like things that are still not working that you, you know, that might be surprising given all the things that do work. And then the Then we can get to the latest models. I want to, so kind of give me the like four, five, four, six history, and then we can go to four, seven and five, five present.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.