Evidence receipt / evaluation
Published · transcript-backedSimon Willison: evaluation
14 Jun 2023 Latent Space Emergency Pod: OpenAI's new Functions API, 75% Price Drop, 4x Context Length (w/ Alex Volkov, Simon Willison, Riley Goodside, Joshua Lochner, Stefania Druga, Eric Elliott, Mayo Oshin et al)
“Like cuz the, the problem if, if the function is delete my email, it doesn't matter if you get the schema right or not, it's still, it can cause a harmful action.”
Source trail
Everything needed to verify it.
- Speaker
- Simon Willison
- Attribution
- Verified speaker
- Claim type
- evaluation
- Recorded
- 14 Jun 2023
- Publisher
- Latent Space
Transcript context
…or no? I don't think so. I feel like that's kind of irrelevant. Like cuz the, the problem if, if the function is delete my email, it doesn't matter if you get the schema right or not, it's still, it can cause a harmful action. What you have to do instead is when you're designing the system, you have to say things like, okay, make every action reversible. So at least if the LLM goes rogue and deletes all of my emails, I can under delete my emails again, that kind of thing. Or if it's an action that cannot be reverted, like sending an email to your boss, that's the point where you have to have human approval designed in. So when you're designing these, you have to assume that a prompt injection attack could succeed and make sure that the damage caused by that is either reversible damage or at least has some way of the human, of, of, of a human catching what's going on, stopping it. I really love this. I'm gonna try to use that as a template for small developer. This, yeah, there's different modes. You, you market, you market function as reversible or requires human input and you build up a library of them.…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.