High Signal Podcasts Evidence ledger
Method
Browse
How the evidence works

Public evidence record

Andreas Stuhlmüller

Published podcast speaker

Claims
28
Episodes
2
Shows
2
Named items
0

Claim ledger

What Andreas said.

7 transcript-backed records

01 / evaluation

I think with software engineering, if it's the case that 80% of the time when the model says this is like an automatically reviewable feature than it is actually is, then that's not good enough because we don't want to break production 20% of the time.

“I think with software engineering, if it's the case that 80% of the time when the model says this is like an automatically reviewable feature than it is actually is, then that's not good enough because we don't want to break production 20% of the time.”
Publisher
The Cognitive Revolution

02 / evaluation

I think if you're trying to create a reward signal, that's pretty rough because The models are going to optimize pretty hard against your signal.

“I think if you're trying to create a reward signal, that's pretty rough because The models are going to optimize pretty hard against your signal.”
Publisher
The Cognitive Revolution

03 / evaluation

These papers are small and so on, but the same thing applies at much larger scale where like you can actually, the process still remains an important waste checkable because we do see the tool calls and the tool calls are an important input into the model's reasoning.

“These papers are small and so on, but the same thing applies at much larger scale where like you can actually, the process still remains an important waste checkable because we do see the tool calls and the tool calls are an important input into the model's reasoning.”
Publisher
The Cognitive Revolution

04 / evaluation

Because you're like, I told you what to do, you didn't do it. And so I think the reason for that is because the models are not trained on process.

“Because you're like, I told you what to do, you didn't do it. And so I think the reason for that is because the models are not trained on process.”
Publisher
The Cognitive Revolution

05 / evaluation

In one sense, I think you're right that throw everything into the context window thing is easier to maintain because you just can swap out a model.

“In one sense, I think you're right that throw everything into the context window thing is easier to maintain because you just can swap out a model.”
Publisher
Latent Space
Search evidence