Public evidence record
Published podcast speaker
Books, apps, and tools
book / likes
“My favorite book is The River of Doubt, in which Theodore Roosevelt, after losing, I believe, the 1912 campaign, goes to Southern America and traverses a never before traversed river, and along the way gets all of these horrible infections, almost dies.”
Claim ledger
12 transcript-backed records
01 / evaluation
“You'd never actually say this in practice because it's very difficult to estimate adversarial robustness because the search space here is massive, which we'll talk about soon.”
02 / evaluation
“CAMEL can't really help because it's like, "Okay, I'm going to give you read email permissions and also send email permissions," and now this is enough for an attack to occur.”
03 / evaluation
“I think another thing worth pointing out is looking at anthropic constitutional classifiers and other models, it does seem to be more difficult to elicit CBRN and other really harmful outputs from chatbots, but solving indirect prompt injection, which is basically prompt injection against agents done by external people on the internet is still very, very, very unsolved, and it's much more difficult to solve this problem than it is to stop CBRN elicitation, because with that kind of information, as one of my advisors just noted, it's easier to tell the model, "Never do this," than with emails and stuff, "Sometimes do this.”
04 / evaluation
“I found some major problems with the AI security industry. AI guardrails do not work.”
05 / evaluation
“Very interesting industry. And I'll quickly differentiate and separate out the Frontier Labs from the AI security industry because there's the Frontier Labs and some Frontier adjacent companies that are largely focused on research like pretty hardcore AI research.”
06 / evaluation
“Humans are adaptive attackers because they test stuff out and they see what works and they're like, "Okay, this prompt doesn't work, but this prompt does.”
07 / evaluation
“I think you gave the perfect example with ServiceNow, and that's the reason that this stuff is so important to talk about right now because with chatbots, as you said, very limited damage outcomes that could occur, assuming they don't invent a new bioweapon or something like that.”
08 / evaluation
“If all you're doing is deploying chatbots that answer FAQs, help users to find stuff in your website, answer their questions with respect to some documents. It's not really an issue because your only concern there is a malicious user comes and, I don't know, maybe uses your chatbot to output hate speech or C-burn or say something bad, but they could go to ChatGPT or Claude or Gemini and do the exact same thing.”
09 / evaluation
“There's no effect, no predictable effect of these roles." And so my thinking on this is that at some point with the GPT-3, early ChatGPT models, it might've been true that giving these roles provides a performance boost on accuracy-based tasks, but right now, it doesn't help at all.”
10 / evaluation
“I do remember at some point we put out a tweet and it was just, "Role prompting does not work.”
11 / evaluation
“Yeah, I will say I think that the stop AI folks are entirely different from the regulate AI folks.”
12 / evaluation
“I'll come back and again and again to things like math questions because a lot of these techniques are judged based off of data sets of math or reasoning questions simply because you're going to evaluate the accuracy programmatically as opposed to something like generating interview questions, which is no less valuable, but just very difficult to evaluate success for in an automated way.”