Public evidence record
Published podcast speaker
Claim ledger
14 transcript-backed records
01 / preference
“One of the things I wrote that was like, a lot of the ways that I use these models are for one-off things that I just need to happen that I'm going to throw away in five minutes.”
02 / evaluation
“Like I feel like this is like the significant way in which I use machine learning models is probably very different than most people is that I'm a researcher and I get to pick what tools that I use and most of the things that I work on are fairly small projects.”
03 / uncertainty
“I probably spent, I don't know, definitely less than 10 hours putting this together.”
04 / belief
“Probably not. I feel like, you know, I think people should work on more fun things.”
05 / belief
“I think probably this is a thing that if you really cared about this task being solved well, you would train a model for.”
06 / uncertainty
“Maybe you don't have to do it. I don't know. This is for the people whose job is to make these things better.”
07 / prediction
“And the types of security questions I would want to ask would be very different. And so I think, you know, in some very large part, understanding what the future will look like a couple of years ahead of time is helpful for figuring out which problems, as a security person, I want to solve now.”
08 / preference
“I like Turing completeness. The definition of Turing completeness is a computer that can run anything, essentially.”
09 / commitment
“I will take whatever tools are available in front of me and try and see if I can use them in meaningful ways.”
10 / evaluation
“I feel like lots of people, there are far too many X plus AI, where X is like arbitrary thing in the world that has nothing to do with it and could not be benefited at all. And they're just doing it because they want to use the word.”
11 / preference
“Currently though, make a model into an agent by just copying and pasting error messages for the most part. That's what I do, is you run it and it gives you some code that doesn't work, and either I'll fix the code, or it will give me buggy code and I won't know how to fix it, and I'll just copy and paste the error message and say, it tells me this.”
12 / evaluation
“We didn't have attacks that were useful enough that you could show it to someone who actually designed a real system and they'd be willing to say, I am going to make my system less useful because the attack that you've presented to me is so compelling that I will break the functionality of my system.”
13 / preference
“I feel like I still don't like writing. And so because of this, the way I tend to treat this is like, I will put it together into the best format that I can at a time, and then put it on the internet, and then never change it.”
14 / evaluation
“You want to construct a task that is hard, and we will measure progress on this benchmark, not because we care about the problem per se, but because we know that progress on this is in some way correlated with making better models.”