Petar Velichkovich: recommendation

Transcript context

…And so, like, really teasing apart on a very basic level, like Newton's 3 laws of motion, has it encapsulated it, Whether that's Vio or Genie, have these models encapsulated the physics of that a 100% accurately? And right now, they're not. They're kind of approximations, and they look realistic when you just casually look at them. They're not accurate enough yet to rely on, for say, robotics. Just because we can achieve some level of moving the needle by hooking up a really potent tool to a language model doesn't mean that we shouldn't think about what would the next generation of these models look like and how can we make them intrinsically better. Because even if you have the best tool in the world, that is not going to save you if you cannot predict the right inputs for that tool. Even some of the current frontier models, they will, as you probably know, perform hundreds of billions of multiplications just to produce a single token of output, yet they cannot reliably multiply even relatively small numbers together without failing. Right? So this is something that, to me, hints at a great misalignment between what we are trading these systems to do and how we're building them and what we might want to use them for downstream, especially if we're doing reasoning or if we're doing science. But it seems like, let's say, LLM, if you teach it properly, you can just teach it to do, let's say, addition up to some failure of its memory. Just like with humans, we might forget a digit or we forget to carry over or so, you know, we do the algorithm wrong sometimes with some probability. Up to that, it can learn this this procedure of adding long digit numbers. You know, it's is it neural? Is it symbolic? It's doing something some algorithm, something symbolic perhaps, but it's doing that with its neural machinery and the neural machinery also allows it to, well, first of all, absorb tons and tons of world knowledge and deal with the vagueness of concepts. Right? The fact that that things don't exactly want to fit your good old fashioned AI symbolic theory most of the time.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.