Speaker unverified: belief

Transcript context

…ike you know, what what what is kind of the kind of ultimate interface that we want to interact with the model with. So, so, the model gives some the models give some UI harness, right? They can build their own UIs, they can kind of do things that uh you know, people would find very time-consuming. Um but I yeah, I I definitely think there is also just like lot of space to kind of enable the models to access like the current interfaces that we use for for people, right? So, I think like we want to have um um you know, AI's on Slack, for example, or that that are kind of plugged into our our contacts and uh and yeah, and are able to to learn from it and I will I'm able to kind of yeah, to to realize these existing things, right? So, definitely like there is some meet in the middle here, but definitely I believe like long term uh you know, like by default the AI should just kind of meet you where where you are uh and if not, that will be because it kind of it has new abilities, not because it has limitations. Yeah, that's an interesting point that basically today it feels like these harnesses are so bespoke to certain environments, but like over time as you add more and more skills and tools and models can navigate uh across those effectively, it's like there'll just be a general like, you know, the way humans have uh that that makes a a tremendous amount of sense. I guess I'm curious like, you know, you uh obviously I must I'm sure like every day you see kind of crazy stuff on the research side. At this point like, what are the milestones that are like still meaningful to you as you think about like, "It would be pretty crazy if I, you know, uh did it one day and saw like X or Y?" Like, what are the things you're paying most attention to? Yeah. Um I mean, at this point it really is about um research, right? Like, isn't about it it is about kind of model discovering new things, kind of executing on like a longer horizon um research problem. It's almost like looking for some sort of insight that you're like, "Oh, if someone on my team had come up with that, that would be I I'd been pretty intrigued by it." >> Yeah, we we've actually had like some minor uh uh but I I think I think quite impactful ideas uh come from uh even like GPT-5.2 Pro uh that that's really entirely, but you know, I I think it's still very very small compared to what I expect it to be. Yeah. I mean, it seems like almost inevitably like these models are going to get better. They will be used in research. They'll be used in science more generally. You're like one of the first people interacting directly with these models as like research partners almost at this stage. Anything like you've learned around the right way to do that or as you think about like what a research organization, you know, as these models continue to get better might look like? Yeah. Would I think we're definitely kind of at a transition point where kind of the short-term immediate quality of the model uh is about to be a quite determining factor for the pace of our research progress because the models are going to drive a lot of that. And so that definitely requires um you know, uality of the model uh is about to be a quite determining factor for the pace of our research progress because the models are going to drive a lot of that. And so that definitely requires um you know, rewiring some intuitions about how to um run a research organization. Uh you know, normally you kind of try to not to focus on like immediate quality. You try to be much more focused on like the longer term. I think we have like a lot of very exciting uh stuff good up that we are kind of working towards, but I feel a lot of urgency to kind Yeah. Yeah, so I told actually um execute on it and to actually use this advances in model intelligence to um accelerate research on the AI and especially AI alignment. Yeah, it's such a fascinating point cuz I've heard you talk before about running a research organization. I feel like in the past it was like giving people the space to you know, pursue a lot of things that weren't like directly, you know, hey, this is for a month or two months of progress, but it's like what are the ideas that are really going to drive things forward. But it makes total sense that we're in a time now where uh you're like, look, everything we do will be so much better if we just focus on this in the in the short term and make it better. It must be like fascinating to navigate uh that and like these maybe further off research ideas at the same time and like running an organization. >> Yeah. Yeah, it's definitely yeah, it's definitely something we we spend a lot of time on uh with Mark nowadays, yeah. Right now you have um you know, a a ton of computers company, but you obviously you have great scaling laws on the pre-training side. You have great scaling on the RL side. You have probably lots of experiments going on that have nothing to do with either of those vectors, but are like interesting new ways. How do you even think about like allocating compute across all of this stuff? Yeah, it's going to be very complicated, right? Because there's so many things that we need to do. One thing we've been What one kind of discipline we've started keeping is we we try to make sure we just like explicitly budget like a large chunk of our compute to the most scalable methods of the things that we believe are the most responsible for driving general model intelligence. And you know, even if it's not the most efficient allocation of computer at all times, because, you know, if you're allocating so much computer to like one experiment or like one set of experiments, you know, there's so many things you can accelerate with a little bit of that computer elsewhere. Uh but, you know, but I think it's easy to kind of like with all the all the all the interesting and important things that we're doing, I think it'll be very easy to kind of partition all of it and like not not not really end up doing the things that we believe are most important. You definitely want to like understand the kind of empirical evidence. You definitely want to make sure your evaluations are in order and the kind of experimental rigor is there. And then you also want to apply some regularization based on like, "Okay, do we understand this method? Do we nt to make sure your evaluations are in order and the kind of experimental rigor is there. And then you also want to apply some regularization based on like, "Okay, do we understand this method? Do we actually expect it to will scale? Do we expect this is something you can actually build on in the future? Is this kind of a one-off, right?" And I think and based on that, uh determine the price. >> Yeah. It's so interesting. You probably find all the yeah ways that you like know you could improve things, but they feel maybe like uh off off a little bit to the side of where you think the overall arc of progress is. And so, you end up leaving some of these like low-hanging fruits to some extent, because really the most important thing is finding the future direction and then the scaling within that and uh devoting computer with that. Obviously, the the place where we talked about Codex a lot and and the success of coding. And it feels like, you know, last year was like the year of just incredible hill climbing on on coding. I I'm curious, you know, obviously, you know, Codex has been a super successful product. In many ways, like Anthropic was kind of first to this market, you know, Claude code, you know, it was it was a dominant product there. What do you kind of like, you know, reflecting on that, I guess, like what do you make of it the success Anthropic's had in this space? Yeah, I think I think it's a matter of, you know, really focusing your product direction or on where where you believe the kind of that the next application of the technology is, right? And um you know, if you look at the kind of prioritization we've had on the on our product at OpenAI, like I mean, we have been, right? Like working on on coding projects, but they have kind of been like a secondary thing, right? Compared to like our main priorities. And the interesting thing is that is not very reflective of like the priorities of the research organization within OpenAI. Uh I think you know, given that like we've kind of had this you know, explosive success of ChatGPT, you know, ChatGPT as it was, you know, I think I think ChatGPT is evolving quite a bit and it's going to evolve quite a bit, but as it was in 2023, right? This particular, you know, product that's maybe not, you know, I think it's definitely a quite aligned with our vision of like where AI is going, but it but like it's not really like the the like representative of like everything that that that that it enables. And so the majority of like our work in research has been focused on like that that future thing, and I think increasingly it has decoupled from our our our kind of like short-term product strategies, right? Yeah, I'm very kind of um confident about um the things we've been building and the things we we we are building on on on the research on the model intelligence side. You know, a lot of our our rep- representation and increased focus on the on the product side is about actually kind of getting to deploy them. And the belief that actually they are uh the thing that really matters now. Yeah, and now it feels like, you know, the uh clearly the whole company priority, you know, is still locked in…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.