Nathan Labenz: prediction

Transcript context

…Those decisions are coming. Maybe some of them have already been made. Those decisions as of right now, I don't think are very well informed by like very rigorously epistemic accurate forecasting AIS. But if you just give it another couple of years, we might be in the world where everybody has the same grounding, like as smart as Kissinger, but actually like trying to help and like trying to give better outcomes that we can all have and that could usher us through this crazy phase before the crazy paper clip type of stuff starts to happen. So I feel like I'm racing, you know, from AI forecasting to make it useful and make it help. It's kind of a broader epistemics and safety process because otherwise it's just going to get away from all of us and then a lot of the work we're doing just doesn't matter. Wednesday's guest Zeve Farman, Co founder and CEO of Litrix, the company behind Facetune and now one of the only frontier scale open weights efforts in video and world models under the LTX brand. I've stayed up late making music videos with their audio condition model. In his honour. We started with what a world model even is. OK, wow, that's a big question. Because we released the LTX 2.3 like roughly 1/4 ago. And in the high years it feels like, I don't know, like a decade. OK, so a bunch of things. I think there's like a growing realisation that what started as video models is becoming a backbone of what we call now like world models. And I think like the best way to explain why this is so powerful is to use the analogy to LLMS, right? Like in the end of the day, at their core, LLMS are still predicting the next talk in the next word. And when we do the pre training at the scale of the Internet, it allows us to create models that do textual reasoning incredibly well. And the emerging world models, they're kind of doing the same, right? Like giving some kind of boundary condition, some kind of history, some kind of constraints. They predict the next moment, OK. And the moment includes how the world appears, how it like sounds, and what kind of action we can do. I think the action part is the most maybe surprising one. And like roughly, I would say like 1/4 ago, maybe a bit more in video showed in their Dream 0 paper that it's fairly easy to add to video tokens some kind of encoding of the the joints of the robot and then basically completely ditch the VLA paradigm that was the reigning supreme before it. So I think that's like one of the big surprises. And for us, realizing that was like this big moment that validated something that we always strive for is to create an extremely efficient models. it. So I think that's like one of the big surprises. And for us, realizing that was like this big moment that validated something that we always strive for is to create an extremely efficient models. Because I think once you start to realize that the robot will need to create like the simulation 30 times a second, you just like realize the amount of tokens that is going to be burned for the simulations. So I think that was like one of the maybe exciting validations of the overall thesis in terms of architectural like a bunch of things that we can, I don't know, discuss in depth. We're planning to release on our mixture of expert architecture. Besides the dense models that we're already releasing, I think we finally were able to crack variable tokens architecture. It's also exciting and kind of teaches the model to invest more tokens where let's say the physics is challenging or like something necessitate to create more tokens. So anyhow, a ton of things are going on. We're gearing towards the release of our next module really soon. So yeah, busy times. Prakash asked where the real bottleneck is compute data or model design. Well, it's like our constraint. It obviously compute. We're like a company that funded the development of the model using profits from mobile content creation apps. So like we definitely compute constraints and like the big guys. And as to efficient inference, like recently it it really depends on the use cases, right? Like, so let's think about about a bunch of them. If you are, let's say you want to create like a real time avatars or like virtual environments, then OK, you can take like a huge model. You can do like a weight distillation to weigh kind of smaller architecture in terms of a parameter count, right? You can then they're like distillate, I don't know, two to four steps. And we're already at the point where for a lot of these use cases, we're like at the latency like way below a second, right? So I think we are hitting a point where these things are becoming production ready for some use cases. But for real time use cases, I think like avatars are extremely easy. We're going to see like a ton of avatars soon that they're going to be like, I don't know, like virtual teachers and visual customer support professionals etcetera to create an actual gaming environment environments. We still have a problem of having enough tokens for world consistency, right? So think about Genie free and the similar models, right? You typically create some kind of autoregressive model that has a lot of tokens that you already generated as your in in your kind of context window. And that blows up pretty quickly, right. So we were having some models that have like, I don't know, 30 seconds, like 60 seconds, It's still not enough to have an actual game. And if you think about that, the sort of brute force compression methods that we're using so far where for example, you just like sub sample tokens, they're like not really robust, right? Like just like imagine this scenario, what you, when you start to generate some kind of environment like my room, for example, and then I open the drawer and there's like a small coin there, right? t? Like just like imagine this scenario, what you, when you start to generate some kind of environment like my room, for example, and then I open the drawer and there's like a small coin there, right? You kind of expect that. Now when you get out of the room and you come back and you open the same drawer, you're still going to see the same coin right at the same place. But like this coin it just like this like tiny token that was generated and to creating a system that knows how to compress like the whole context in a way that's still going to preserve like these critical details. Well, we don't have it yet, right. So although like we do have like real time models that can do the things that the context is still missing there. And I don't think we're going to have like, you know, games that are running on the system like an actual games in the next quarter or two. And in terms of robotics, a lot of the use cases around robotics actually do not require like a ton of context window, right? Like think about like robotic arms and dexterity use cases. Then like the whole context is in front of you, right. You, let's say you want to, I don't know, figure out how the robot can create a sandwich. Well, everything is kind of the front of you. And then latency and auto regressive models. Yeah, like this part we already have. So you're going to start seeing them of robotic arms doing things like fairly quickly in the next quarter or two. Like so far if you're looking, a lot of these videos actually kind of speed them up, right. So it looks like the robot is something cool with its arms, but it's like, OK, extend the speed. I think that that's like mostly solved then the business question, why give a frontier model away? Yeah, so great question. So there's like a really lot of unpack there here. So just like a little bit of the background, the reason that we started to create our own foundation on models like this realisation that what closed model providers are offering does not make sense for us economically. OK, we were A at logics, we're a mobile creativity company.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.