person / likes
Melanie Mitchell
“There there has been a bit of an obsession, I think, with, headline accuracy when we do evaluations so that I'm a huge fan of Melanie Mitchell, for example, when she speaks about construct validity.”
Public evidence record
Beth Barnes
Published podcast speaker
- Claims
- 13
- Episodes
- 1
- Shows
- 1
- Named items
- 1
Books, apps, and tools
The evidenced stack.
Claim ledger
What Beth said.
5 transcript-backed records
01 / belief
Think he's he's probably, like, you know, more more confident on some things where I'm I'm more uncertain, and I think some of the, like, AI futures project models are, like, more sensitive to the meteor time horizon metrics than they should be.
“Think he's he's probably, like, you know, more more confident on some things where I'm I'm more uncertain, and I think some of the, like, AI futures project models are, like, more sensitive to the meteor time horizon metrics than they should be.”
- Speaker
- Beth Barnes
- Publisher
- Machine Learning Street Talk
02 / belief
I think may maybe there's some difference in, like, how much you've you think that this has improved between, like, g b d 2 and where we are now, or I would say sort of in, you know, the amount of adapting to new things that are happening that models could do now does seem like it's much higher and, you know, they're much better at, like, editing their own scaffolding or or, you know, sort of reasoning about their, like, you know, their their sort of embodiment, like, this thing of, like, knowing not to kill your own process or no like, you know, stuff stuff like that where it's like there is like, yes, they are limited, but there's also, some trend of improvement.
“I think may maybe there's some difference in, like, how much you've you think that this has improved between, like, g b d 2 and where we are now, or I would say sort of in, you know, the amount of adapting to new things that are happening that models could do now does seem like it's much higher and, you know, they're much better at, like, editing their own scaffolding or or, you know, sort of reasoning about their, like, you know, their their sort of embodiment, like, this thing of, like, knowing not to kill your own process or no like, you know, stuff stuff like that where it's like there is like, yes, they are limited, but there's also, some trend of improvement.”
- Speaker
- Beth Barnes
- Publisher
- Machine Learning Street Talk
03 / belief
Like, these 2 things can coexist, and I think people often sort of you know, positions are surprisingly correlated on some axis of, like, how, you know, how soon you think AIs or how good you think AIs or something.
“Like, these 2 things can coexist, and I think people often sort of you know, positions are surprisingly correlated on some axis of, like, how, you know, how soon you think AIs or how good you think AIs or something.”
- Speaker
- Beth Barnes
- Publisher
- Machine Learning Street Talk
04 / belief
I think people u usually use scheming to refer to the model is doing what it's currently doing in in service of some long term goal and is deliberately doing things like appearing aligned or or getting a high score, like, in service of, you know, eventually accomplishing that goal versus you can be reward hacking both you know, you could be reward hacking in some extremely dumb way, like the boat example where it's just like, this is what RL kind of found or, like, this is what, you know, like, a star search found.
“I think people u usually use scheming to refer to the model is doing what it's currently doing in in service of some long term goal and is deliberately doing things like appearing aligned or or getting a high score, like, in service of, you know, eventually accomplishing that goal versus you can be reward hacking both you know, you could be reward hacking in some extremely dumb way, like the boat example where it's just like, this is what RL kind of found or, like, this is what, you know, like, a star search found.”
- Speaker
- Beth Barnes
- Publisher
- Machine Learning Street Talk
05 / belief
I think, you know, there's a good chance that this, you know, makes our lives a lot better or a lot worse, and people disagree, you know, about even what what current models can do, let alone where we're heading.
“I think, you know, there's a good chance that this, you know, makes our lives a lot better or a lot worse, and people disagree, you know, about even what what current models can do, let alone where we're heading.”
- Speaker
- Beth Barnes
- Publisher
- Machine Learning Street Talk