Shawn Wang: uncertainty

Transcript context

…because they maybe use other open source code and so we can't redistribute it necessarily. Other ones, because, you know, that's, I think evaluations and data are like a core, important part of, you know, the business. And I think we take evaluations very seriously and are spending a lot of effort in terms of like, what exactly do we make as part of the evaluation set? How do you evaluate these things? We've done a lot of other stuff, you know, since these evaluations. But I think a lot around like code understanding for us, since that's our main focus. And it's a nice place to explore reasoning as well. It sounds like you talk a little bit about like code understanding as like sort of variable level, like sort of very micro context. Is there a sense of like larger code context as well? I don't know what I mean by that, by the way. It's mostly just like if I told the senior engineer to go look at a code base, they would understand at a broad level, the architecture, but also the design decisions and be able to tell me that. I don't know if that's useful or not, but I mean, that's useful to me as a, as someone who might be working with them. Yeah. This particular dataset is like the more low level code understanding,…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.