Public evidence record
Published podcast speaker
Books, apps, and tools
book / likes
“My favorite book is The River of Doubt, in which Theodore Roosevelt, after losing, I believe, the 1912 campaign, goes to Southern America and traverses a never before traversed river, and along the way gets all of these horrible infections, almost dies.”
Claim ledger
28 transcript-backed records
01 / belief
“I think I have slides on this from probably two years ago and it's straightforward enough.”
02 / uncertainty
“I don't know if that's what killed them, but they don't seem to be in business anymore.”
03 / belief
“How these get found, how do they get implemented at companies. And I think the easiest way of thinking about it is like, I'm a CSO at some company we are a large enterprise.”
04 / belief
“There are theoretically ways to train the eyes to be smarter, to be more adversarially robust, and we haven't really seen this yet, but there's this idea that if you start doing adversarial training in pre-training earlier in the training stack, so when the AI is a very, very small baby, you're being adversarial towards it and training it then, then it's more robust, but I think we haven't seen the resources really deployed to do that.”
05 / evaluation
“You'd never actually say this in practice because it's very difficult to estimate adversarial robustness because the search space here is massive, which we'll talk about soon.”
06 / evaluation
“CAMEL can't really help because it's like, "Okay, I'm going to give you read email permissions and also send email permissions," and now this is enough for an attack to occur.”
07 / prediction
“It's kind of funny, because AI researchers are the only people who can solve this stuff long-term, but cybersecurity professionals are, they're the only ones who can kind of solve it short term, largely in making sure we deploy properly permission systems and nothing that could possibly do something very, very bad.”
08 / evaluation
“I think another thing worth pointing out is looking at anthropic constitutional classifiers and other models, it does seem to be more difficult to elicit CBRN and other really harmful outputs from chatbots, but solving indirect prompt injection, which is basically prompt injection against agents done by external people on the internet is still very, very, very unsolved, and it's much more difficult to solve this problem than it is to stop CBRN elicitation, because with that kind of information, as one of my advisors just noted, it's easier to tell the model, "Never do this," than with emails and stuff, "Sometimes do this.”
09 / evaluation
“I found some major problems with the AI security industry. AI guardrails do not work.”
10 / evaluation
“Very interesting industry. And I'll quickly differentiate and separate out the Frontier Labs from the AI security industry because there's the Frontier Labs and some Frontier adjacent companies that are largely focused on research like pretty hardcore AI research.”
11 / evaluation
“Humans are adaptive attackers because they test stuff out and they see what works and they're like, "Okay, this prompt doesn't work, but this prompt does.”
12 / evaluation
“I think you gave the perfect example with ServiceNow, and that's the reason that this stuff is so important to talk about right now because with chatbots, as you said, very limited damage outcomes that could occur, assuming they don't invent a new bioweapon or something like that.”
13 / prediction
“When it comes to AI security, the AI security industry in particular, I think we're going to see a market correction in the next year, maybe in the next six months, where companies realize that these guardrails don't work.”
14 / evaluation
“If all you're doing is deploying chatbots that answer FAQs, help users to find stuff in your website, answer their questions with respect to some documents. It's not really an issue because your only concern there is a malicious user comes and, I don't know, maybe uses your chatbot to output hate speech or C-burn or say something bad, but they could go to ChatGPT or Claude or Gemini and do the exact same thing.”
15 / recommendation
“My favorite book is The River of Doubt, in which Theodore Roosevelt, after losing, I believe, the 1912 campaign, goes to Southern America and traverses a never before traversed river, and along the way gets all of these horrible infections, almost dies. They run out of food. They have to kill their cattle. I think half or more than half of their party died along the way. And it ended up just being this insane journey that really spoke to his mental fortitude. And one of my favorite anecdotes in that book was that he would do these point-to-point walks with people, where he'd look at a map and just kind of put two dots on the map and be like, "Okay, we're here. We're going to walk in a straight line to this other place." And straight line really meant straight line. I'm talking like climbing trees, bouldering, wading through rivers, apparently naked with foreign ambassadors. I feel like politics would be a lot better if our president would do that. It's only stories like those that are just core America to me. And I am actually entirely into bushwhacking and foraging. And if you had a plants podcast, that would be an episode. But I love that story. I love that book. It was entirely fascinating to me.”
16 / recommendation
“My favorite book is The River of Doubt, in which Theodore Roosevelt, after losing, I believe, the 1912 campaign, goes to Southern America and traverses a never before traversed river, and along the way gets all of these horrible infections, almost dies.”
17 / evaluation
“There's no effect, no predictable effect of these roles." And so my thinking on this is that at some point with the GPT-3, early ChatGPT models, it might've been true that giving these roles provides a performance boost on accuracy-based tasks, but right now, it doesn't help at all.”
18 / evaluation
“I do remember at some point we put out a tweet and it was just, "Role prompting does not work.”
19 / commitment
“There are cases that I will bring in those other techniques, but the most important places to use those techniques is the product-focused prompt engineering.”
20 / belief
“I guess you could do it infinitely. I think the model would go crazy at some point.”
21 / belief
“I actually was at the AI Engineer World's Fair yesterday, and there was somebody, I think before me, giving a talk that prompt engineering is dead.”
22 / belief
“If you want to compete in the HackAPrompt competition, I think we have like a $100,000 up in prizes.”
23 / prediction
“I also found this a little bit difficult to believe because that's not really how AI is supposed to work, but I don't know, we see all sorts of weird things from it.”
24 / evaluation
“Yeah, I will say I think that the stop AI folks are entirely different from the regulate AI folks.”
25 / commitment
“Fernandez Building, I think he actually invented and has the patent on E Ink technology.”
26 / evaluation
“I'll come back and again and again to things like math questions because a lot of these techniques are judged based off of data sets of math or reasoning questions simply because you're going to evaluate the accuracy programmatically as opposed to something like generating interview questions, which is no less valuable, but just very difficult to evaluate success for in an automated way.”
27 / recommendation
“One, it can get cached. So subsequent calls to the LM with that same context at the top of the prompt are cheaper because the model provider stores that initial context for you as well as the embeddings for it.”
28 / recommendation
“It's a more research-y approach. But just take some common format out there that the LLM is comfortable with, and I say that with air quotes because it's a bit of a strange thing to say the LLM is comfortable with something, but it actually comes empirically from studies that have shown that formats of questions that show up most commonly in the training data are the best formats of questions to actually use when you're prompting it.”