Lex Fridman: evaluation

Transcript context

…They come back and just blow your mind. If we can achieve that, that amount of inference compute, where it leads to a dramatically better answer as you apply more inference compute, I think that will be the beginning of real reasoning breakthroughs. The following is a conversation with Aravind Srinivas, CEO of Perplexity, a company that aims to revolutionize how we humans get answers to questions on the internet. It combines search and large language models, LLMs, in a way that produces answers where every part of the answer has a citation to human-created sources on the web. This significantly reduces LLM hallucinations, and makes it much easier and more reliable to use for research, and general curiosity-driven late night rabbit hole explorations that I often engage in. I highly recommend you try it out. Aravind was previously a PhD student at Berkeley, where we long ago first met, and an AI researcher at DeepMind, Google, and finally, OpenAI as a research scientist. This conversation has a lot of fascinating technical details on state-of-the-art, in machine learning, and general innovation in retrieval augmented generation, AKA RAG, chain of thought reasoning, indexing the web, UX design, and much more. This is The Led Fridman Podcast. To support us, please check out our sponsors in the description. Now, dear friends, here’s Aravind Srinivas. Perplexity is part search engine, part LLM. How does it work, and what role does each part of that the search and the LLM play in serving the final result? Perplexity is best described as an answer engine. You ask it a question, you get an answer. Except the difference is, all the answers are backed by sources. This is like how an academic writes a paper. Now, that referencing part, the sourcing part is where the search engine part comes in. You combine traditional search, extract results relevant to the query the user asked. You read those links, extract the relevant paragraphs, feed it into an LLM. LLM means large language model. That LLM takes the relevant paragraphs, looks at the query, and comes up with a well-formatted answer with appropriate footnotes to every sentence it says, because it’s been instructed to do so, it’s been instructed with that one particular instruction, given a bunch of links and paragraphs, write a concise answer for the user, with the appropriate citation. The magic is all of this working together in one single orchestrated product, and that’s what we built Perplexity for.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.