High Signal Podcasts Evidence ledger
Method
Browse
How the evidence works

Public evidence record

Erik Schluntz

Published podcast speaker

Claims
28
Episodes
1
Shows
1
Named items
0

Claim ledger

What Erik said.

19 transcript-backed records

01 / belief

I think some way, like, right now, it just takes, I think we sort of force people to do the best practices of writing out sort of these full JSON schemas, but it would be really nice if you could just pass in a Python function as a tool.

“I think some way, like, right now, it just takes, I think we sort of force people to do the best practices of writing out sort of these full JSON schemas, but it would be really nice if you could just pass in a Python function as a tool.”
Speaker
Erik Schluntz
Publisher
Latent Space

03 / belief

I would say on the positives, like, there is really sort of incredible progress that's happened in the last five years that I think will be a big unlock for robotics.

“I would say on the positives, like, there is really sort of incredible progress that's happened in the last five years that I think will be a big unlock for robotics.”
Speaker
Erik Schluntz
Publisher
Latent Space

04 / belief

I think the computer use demo that we released is an extension of that. It has the same bash and edit tools, but it also has the computer tool that lets it get screenshots and move the mouse and keyboard.

“I think the computer use demo that we released is an extension of that. It has the same bash and edit tools, but it also has the computer tool that lets it get screenshots and move the mouse and keyboard.”
Speaker
Erik Schluntz
Publisher
Latent Space

05 / belief

I think that for robotics, the limiting factor is going to be reliability, that these models are really good at doing these demos of doing laundry or doing dishes.

“I think that for robotics, the limiting factor is going to be reliability, that these models are really good at doing these demos of doing laundry or doing dishes.”
Speaker
Erik Schluntz
Publisher
Latent Space

06 / belief

I think there's still need for sort of many different varied evals. Like sometimes you do really care about just sort of greenfield code generation.

“I think there's still need for sort of many different varied evals. Like sometimes you do really care about just sort of greenfield code generation.”
Speaker
Erik Schluntz
Publisher
Latent Space

07 / belief

Now, I would say that, you know, this is definitely an area of future research, especially if we talk about these problems that are going to take a human more than four hours.

“Now, I would say that, you know, this is definitely an area of future research, especially if we talk about these problems that are going to take a human more than four hours.”
Speaker
Erik Schluntz
Publisher
Latent Space

08 / belief

I think there's been a lot more progress in the software in the last few years. And I think a lot of the humanoid robot companies now are really trying to build amazing hardware.

“I think there's been a lot more progress in the software in the last few years. And I think a lot of the humanoid robot companies now are really trying to build amazing hardware.”
Speaker
Erik Schluntz
Publisher
Latent Space

10 / belief

I think examples are really good in things like descriptions, like so many, like just using the Linux command line, like how many times I do like dash dash help or look at the man page or something.

“I think examples are really good in things like descriptions, like so many, like just using the Linux command line, like how many times I do like dash dash help or look at the man page or something.”
Speaker
Erik Schluntz
Publisher
Latent Space

11 / belief

There's certain things that it's not very good at yet. But I'm really excited, I think, most broadly, not just for new things that weren't possible before, but as a much lower friction way to implement tool use.

“There's certain things that it's not very good at yet. But I'm really excited, I think, most broadly, not just for new things that weren't possible before, but as a much lower friction way to implement tool use.”
Speaker
Erik Schluntz
Publisher
Latent Space

13 / belief

I mean, this, so I'd say that a lot of previous agent work built sort of these very hard coded and rigid workflows where the model is sort of pushed through certain flows of steps. And I think to some extent, you know, that's needed with smaller models and models that are less smart.

“I mean, this, so I'd say that a lot of previous agent work built sort of these very hard coded and rigid workflows where the model is sort of pushed through certain flows of steps. And I think to some extent, you know, that's needed with smaller models and models that are less smart.”
Speaker
Erik Schluntz
Publisher
Latent Space

14 / belief

One is like the, I'd say maybe like doing the easier solution rather than the hard solution. And I'd say the second one, I think what you're talking about is like the lazy model is like when the model says like dot, dot, dot, code remains the same.

“One is like the, I'd say maybe like doing the easier solution rather than the hard solution. And I'd say the second one, I think what you're talking about is like the lazy model is like when the model says like dot, dot, dot, code remains the same.”
Speaker
Erik Schluntz
Publisher
Latent Space

15 / belief

From the day-to-day job. But I think one of the most interesting things about SWE-Bench is that all these other benchmarks are usually just isolated puzzles, and you're starting from scratch.

“From the day-to-day job. But I think one of the most interesting things about SWE-Bench is that all these other benchmarks are usually just isolated puzzles, and you're starting from scratch.”
Speaker
Erik Schluntz
Publisher
Latent Space

17 / belief

I think Eider, they have a really good blog where they explore some of these different methods for editing files, and they post results about them, which I think is interesting.

“I think Eider, they have a really good blog where they explore some of these different methods for editing files, and they post results about them, which I think is interesting.”
Speaker
Erik Schluntz
Publisher
Latent Space

19 / belief

I think the really interesting question for me, for all the startups out there, is this kind of divergence between the benchmarks and what real customers will want.

“I think the really interesting question for me, for all the startups out there, is this kind of divergence between the benchmarks and what real customers will want.”
Speaker
Erik Schluntz
Publisher
Latent Space
Search evidence