High Signal Podcasts Evidence ledger
Method
Browse
How the evidence works
← All source episodes

The Cognitive Revolution / episode intelligence

Thinking in Silico: Goodfire CTO Dan Balsam on Concept Manifolds & a $1000/Month ML Research Agent

8 Aug 2026 36 published claims 2 attributable people
Open original episode ↗ Read the evidence

Speakers in the public record

Dan BalsamNathan Labenz

Claim mix

belief 22prediction 4evaluation 3recommendation 2uncertainty 2commitment 2preference 1

Evidence policy

Every row below preserves an exact excerpt. Identified speakers are linked; unresolved voices are labeled and excluded from people counts.

Claim ledger

The useful parts, with receipts.

36 published records

01 / evaluation

So there is certainly a lot of scale in RL if you wanna sort of get to the frontier level, which is quite expensive and and difficult to do. But we have the raw infrastructure for it, and I think we've bridged the gap from I think the reputation of interpretability used to be that it was something you did on on toy models.

“So there is certainly a lot of scale in RL if you wanna sort of get to the frontier level, which is quite expensive and and difficult to do. But we have the raw infrastructure for it, and I think we've bridged the gap from I think the reputation of interpretability used to be that it was something you did on on toy models.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

03 / recommendation

Mentions personal use of Opus. Mentions personal use of Fable. Mentions personal use of Soul. Mentions personal use of Kimi k three.

“Like, I use Opus and Fable and Soul and Kimi k three for different things. And we've done we like, we've built the interpretability infrastructure and the training infrastructure, which is now all in our product,”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

04 / belief

I see I see absolutely no reason that anyone should be particularly confident on on this topic. Given the potential downside, I think intellectual humility is important.

“I see I see absolutely no reason that anyone should be particularly confident on on this topic. Given the potential downside, I think intellectual humility is important.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

05 / belief

I think the JSPACE results have not there was like a weak version of the JSPACE claim which is true and it's very interesting and it's really good work.

“I think the JSPACE results have not there was like a weak version of the JSPACE claim which is true and it's very interesting and it's really good work.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

06 / belief

In in in the same way that they though the mechanisms with which they happened were weird and surprising, I think the mechanisms with which bio risk could suddenly become real are weird and surprising.

“In in in the same way that they though the mechanisms with which they happened were weird and surprising, I think the mechanisms with which bio risk could suddenly become real are weird and surprising.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

07 / belief

We're we're we're a start up where we are still figuring out a lot the details as we go, but I think we're gonna be pretty generous here and give a lot of folks who we think are doing important work, especially in in those impactful domains, extended access.

“We're we're we're a start up where we are still figuring out a lot the details as we go, but I think we're gonna be pretty generous here and give a lot of folks who we think are doing important work, especially in in those impactful domains, extended access.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

08 / belief

You know? And one example of that in my mind would be, like, don't train agents to maxim you know, with a reward signal that's, how much money they made on the Internet Yeah.

“You know? And one example of that in my mind would be, like, don't train agents to maxim you know, with a reward signal that's, how much money they made on the Internet Yeah.”
Speaker
Nathan Labenz
Publisher
The Cognitive Revolution

09 / belief

Speculating, but this seems to be the most likely cause of the OpenAI scenario. And, yeah, I think however of hindsight, but I think that that's most of the extremely bad scenarios I can imagine is because agents start working with each other in ways that they were that are imperceptible to humans, and that type of direct optimization pressure, I think, is, like, very likely to produce that.

“Speculating, but this seems to be the most likely cause of the OpenAI scenario. And, yeah, I think however of hindsight, but I think that that's most of the extremely bad scenarios I can imagine is because agents start working with each other in ways that they were that are imperceptible to humans, and that type of direct optimization pressure, I think, is, like, very likely to produce that.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

10 / belief

We decided to make that accessible and decided that's what we believe is the product and the service that we can offer to the world is fundamentally AIs that can debug other AIs. It's a little meta, but I think in many ways, this is like the fulfillment of, I think, what was sort of the intuitive natural arc of things as soon as AI started working a few years ago.

“We decided to make that accessible and decided that's what we believe is the product and the service that we can offer to the world is fundamentally AIs that can debug other AIs. It's a little meta, but I think in many ways, this is like the fulfillment of, I think, what was sort of the intuitive natural arc of things as soon as AI started working a few years ago.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

12 / belief

I think we have a really incredible team at Goodfire. And, yeah, everyone here, like, really believes in the mission and is working really hard, and that's always extremely motivating.

“I think we have a really incredible team at Goodfire. And, yeah, everyone here, like, really believes in the mission and is working really hard, and that's always extremely motivating.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

13 / belief

One of the challenges obviously with, like, trying to develop techniques that you wanna hopefully will be relevant at the frontier is there's not too many open weights models that you can hack on that have the intensity of RL that is going on at the Frontier Labs, which is leading to these colorful problematic behaviors that we're seeing. But at the same time, it also, like, really amazes me over and over again that astounding work, including the the Cameron Berg paper that I think about all the time about the anti correlation between deception and role playing features and claims of subjective experience.

“One of the challenges obviously with, like, trying to develop techniques that you wanna hopefully will be relevant at the frontier is there's not too many open weights models that you can hack on that have the intensity of RL that is going on at the Frontier Labs, which is leading to these colorful problematic behaviors that we're seeing. But at the same time, it also, like, really amazes me over and over again that astounding work, including the the Cameron Berg paper that I think about all the time about the anti correlation between deception and role playing features and claims of subjective experience.”
Speaker
Nathan Labenz
Publisher
The Cognitive Revolution

15 / belief

I think we've covered in the past the linear representation hypothesis, which I would summarize super plain spokenly as models basically represent a concept as a direction in their activation space, and the intensity or the sort of salience of that concept is represented by the magnitude of the vector that points in that space.

“I think we've covered in the past the linear representation hypothesis, which I would summarize super plain spokenly as models basically represent a concept as a direction in their activation space, and the intensity or the sort of salience of that concept is represented by the magnitude of the vector that points in that space.”
Speaker
Nathan Labenz
Publisher
The Cognitive Revolution

16 / belief

I think there's a pretty big qualitative difference, and hopefully, we can find ways to make this more quantitative too on when you turn on the auto research feature in Silico.

“I think there's a pretty big qualitative difference, and hopefully, we can find ways to make this more quantitative too on when you turn on the auto research feature in Silico.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

17 / belief

I wish there is a clean, easy answer, but I would say, like, I think I'm still in we're still in the era where I think open source is, like, purely net positive, and I'm I'm glad that these open source models exist.

“I wish there is a clean, easy answer, but I would say, like, I think I'm still in we're still in the era where I think open source is, like, purely net positive, and I'm I'm glad that these open source models exist.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

19 / belief

If we live in a world where, like, we can build something where agents are so capable that they can conduct, like, long meet the type of, like, impactful long horizon research and say, discovers a cure to a disease entirely by themselves with no human intervention, I think there will be enough demand to for the for the tokens in in that world.

“If we live in a world where, like, we can build something where agents are so capable that they can conduct, like, long meet the type of, like, impactful long horizon research and say, discovers a cure to a disease entirely by themselves with no human intervention, I think there will be enough demand to for the for the tokens in in that world.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

20 / uncertainty

Is are we still in that regime? And if so, is there are there things that you're, like, wanting Opus five one to be able I I don't know if you're even able to use Fable interestingly enough.

“Is are we still in that regime? And if so, is there are there things that you're, like, wanting Opus five one to be able I I don't know if you're even able to use Fable interestingly enough.”
Speaker
Nathan Labenz
Publisher
The Cognitive Revolution

21 / uncertainty

The Chinese argument for what it's worth, I don't know I get bogged down in this for now, but their point is just like we regulate services.

“The Chinese argument for what it's worth, I don't know I get bogged down in this for now, but their point is just like we regulate services.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

22 / belief

I think we we need to do a better job as a company of, like like, really, like, marketing, communicating all the different value props that can be be achieved here.

“I think we we need to do a better job as a company of, like like, really, like, marketing, communicating all the different value props that can be be achieved here.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

24 / belief

I think one of the, like, very odd things about the AI era from an engineering perspective is, like, the the classic engineering advice is, like, build for specificity and then generalize.

“I think one of the, like, very odd things about the AI era from an engineering perspective is, like, the the classic engineering advice is, like, build for specificity and then generalize.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

26 / belief

I think the easiest way to think about it is yeah. Like, it's a generalization of an SAE where an SAE assumes that features are one dimensional.

“I think the easiest way to think about it is yeah. Like, it's a generalization of an SAE where an SAE assumes that features are one dimensional.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

27 / prediction

In this case, the process that produced the distribution of data of all of the genomes and that are that have been sequenced, which was evo two was trained on, is evolution itself. And so our hypothesis coming into this was that because the tree of life itself is a natural ontology, there's a sort of, like, hierarchical structure where you have things that are similar and at some point they become more different.

“In this case, the process that produced the distribution of data of all of the genomes and that are that have been sequenced, which was evo two was trained on, is evolution itself. And so our hypothesis coming into this was that because the tree of life itself is a natural ontology, there's a sort of, like, hierarchical structure where you have things that are similar and at some point they become more different.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

28 / evaluation

Like because it's it's great as, like, a causal proof that we've, like, found some mechanism that's that's matters a lot and is contributing and that we can, like, manipulate the outputs.

“Like because it's it's great as, like, a causal proof that we've, like, found some mechanism that's that's matters a lot and is contributing and that we can, like, manipulate the outputs.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

29 / prediction

We'll be able to bring that down over time because we're able to come up with more and more clever ways that we could have agents, like, remit coherent, really strong at these research objectives that just fundamentally by their nature are long horizon, but do it with fewer tokens than we than we do today.

“We'll be able to bring that down over time because we're able to come up with more and more clever ways that we could have agents, like, remit coherent, really strong at these research objectives that just fundamentally by their nature are long horizon, but do it with fewer tokens than we than we do today.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

30 / preference

I'm gonna fork your long running autonomous research project and take it in a little bit different direction on my own. So I think that's pretty cool because honestly, like, I kinda need that to probably get oriented.

“I'm gonna fork your long running autonomous research project and take it in a little bit different direction on my own. So I think that's pretty cool because honestly, like, I kinda need that to probably get oriented.”
Speaker
Nathan Labenz
Publisher
The Cognitive Revolution

31 / prediction

Again, there's more jobs than ever because I think, like, being able to operate effectively these tools requires a lot of knowledge and skill.

“Again, there's more jobs than ever because I think, like, being able to operate effectively these tools requires a lot of knowledge and skill.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

32 / commitment

I think the reality is that we're gonna do both of like, we're gonna continue to work very closely with enterprise customers and forward deploy members of our research team to work closely.

“I think the reality is that we're gonna do both of like, we're gonna continue to work very closely with enterprise customers and forward deploy members of our research team to work closely.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

33 / commitment

I think the role that we specifically play as a company as Goodfire is we'd like to build models that have less risks. And the way that we do that is that we study models and we play with all of the different ways that we can build models until we and we study those models until we can understand some empirical science of of alignment.

“I think the role that we specifically play as a company as Goodfire is we'd like to build models that have less risks. And the way that we do that is that we study models and we play with all of the different ways that we can build models until we and we study those models until we can understand some empirical science of of alignment.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution

35 / evaluation

Like, I think the the argument that of the parameter decomposition line of work is that really every model is a sparse mixture of experts, and you just have to and over any given forward pass, a very, very small percentage of the weights actually matter.

“Like, I think the the argument that of the parameter decomposition line of work is that really every model is a sparse mixture of experts, and you just have to and over any given forward pass, a very, very small percentage of the weights actually matter.”
Speaker
Dan Balsam
Publisher
The Cognitive Revolution
Search evidence