Andrew Lee: belief

Transcript context

…It strikes me that like system of record and just kind of like really reliable storage are not the same thing, but like really reliable storage is like a key part of what drives system of record value. Like I have had instances in my personal cloud code local AI productivity stack development process where it has, in fact, dropped a bunch of data. I'm trying to export stuff out of Slack, for example, and it realizes like, oh, we didn't quite export it right the first time. I'll just delete everything and go try it again, not realizing that it was so rate limited that that actually took like four days to export what I previously exported. And so I certainly value the fact that Slack is not about to delete all my stuff by accident. But that also sort of suggests that there's maybe an opportunity for the horizontal platforms to, and I know you're a database guy historically, right? So is there an opportunity or a paradigm shift where the horizontal platforms say, here's why you can trust us with your data, even if the agents make mistakes or even if there's sort of a, this or that kind of goes bad, we're going to have some sort of snapshotting rollback durability guarantees where mistakes can't lead to data loss. It seems like if you could make that guarantee for people, they could like get much more comfortable with the idea that they don't necessarily need Salesforce anymore. Totally. And I think this is a huge place where harnesses matter, where, you know, is the harness going to make the LLM smarter, we can discuss whether that is true or whether it matters. But can the harness do this sort of thing? I think totally. So let me give you a few examples of how I think we can help. So one, as you mentioned, versioning. There's a whole bunch of startups working on file systems for agents right now, and some of those folks are working on versioning. The basic idea is if your agent goes rogue, you just want to roll back to some previous state. In a simple chatbot, you can just throw away the messages at the end. But in something that's touching the world, you've got to be able to roll back the world. For a file system, you can just change the file system, but if it's touched APIs and stuff, you might need to keep logs of things. I think is pretty key. So I think there's a lot you can do there. I think another area is having like oversight and like logging and stuff. So you actually have the ability to have a human in the loop in places where it matters and do that in smart ways. With our product today, you have to activate tools. One of the things we're going to add soon is the ability for you to have some tools that you approve every run. e-mail is an example of this, where people are pretty confident to say, Hey, you could read my e-mail as much as you want. You could make as many drafts as you want, but you can't send anything unless I say yes. And we want to get to the point where that is really ergonomic. So for example, it can send you a push notification that's ready to send an e-mail, where it's like, it'll go crazy reading and searching and making drafts. And then when it's ready to send, you get a push notification that's like, hey, do you want to review this before it goes? And then you can say yes. And that's all like pushed to you. So I think permissioning can be another big area. I think another big area is using code better and in a more like way. So, you know, let's take data migration from like one system to another. The naive way to do this is to like load that data through an API, feed it into the LLM, have the LLM then like call some tools to put it somewhere else. And basically, when you do that every time, you're putting it through language model context and trusting it to not hallucinate and reproduce that data, which I think the models get better at over time, but it's very hard to have a lot of confidence there. The better way to do this is have the model just generate a migration script. and then run the migration script. And that gives you an artifact in the middle that you can test and you can have human approval for. So they think, yeah, if you're moving data from one to the other, you still want to have an agent that's thinking through how to solve the problem. at you can test and you can have human approval for. So they think, yeah, if you're moving data from one to the other, you still want to have an agent that's thinking through how to solve the problem. But what it should probably do is generate a migration script, generate some tests, run the tests, and then send the thing to the human being like, Here we have the migration plan and the code and the test, and this is why we think it's going to work. Are you okay with this? And then you say, Yes, and then we run it. You could even have test environments. I think the ability to have tools within the agent that allow it to do really high liability stuff and to have approval, there's a lot of opportunity there.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.