Noah Weiss: commitment

Transcript context

…Shifting a little bit. I know you guys have been working on a bunch of AI stuff at Slack. I believe you've been working on AI related stuff for many years. I think at Google you worked on a lot of AI related products. I feel like a lot of people are just getting into this and trying to figure out, "How do we integrate AI and ML and LLMs into our product and how do we not just waste our time chasing things?" I want to ask you just in your time working with AI over the many years you've been doing it and share a little bit about what you've been doing there. What are some things you've learned about how to be actually effective and build valuable products and not just fall for the shiny object issue and trap? I mean, it's almost 15 years ago now that I was working at Google in search on what later became called the knowledge graph. This idea of building a canonical repository of information of people, places, things in the world and relationships between them. Back then, it was a lot of the same ideas, but obviously the techniques. I have got a lot more mature. We used natural language processing to extract all this information from the web and try to build this database of facts. An idea then was could you take queries, people have like, "What are the tallest fountains in Europe or what of the most popular beaches in Southern California?" Be able to actually give answers not just 10 blue links. I think the thing that's really changed, it's super exciting in the last 6, 12 months with LMs and chat GPT and everything else is the idea that now you can take not just knowledge about the world but actually have natural language generation where suddenly the computer can talk back to you in a way that feels extremely human. Then the creative applications of that are pretty massive and exciting. That's, I guess, the lineage there. I think from over the years back at Google at Foursquare, we did a lot of personalization and recommendations at Slack we have search and ML that's coming infused throughout the product. I think a couple things come out as ... I guess the principles that we've used over the years, back then at Google, one of the big ones, was that the promise of the UI has to match the quality of the underlying data, which is to say ... I think this is actually one of the failings of the various LMs right now is they all appear supremely confident even when they're completely hallucinating. I think that's going to be something that people are going to have to work on a lot, which is to figure out how to be not so faultless, to acknowledge when you're not sure, because otherwise, it undermines the trust people have in the system. Using a lot of transparency about where the data comes from so people can actually build credibility and the tools is really important. Then I think making sure that as you're designing the products that you have virtuous cycles that are naturally part of the product experience where you can get training data as a byproduct of people naturally using the software and then can make the model that you're building behind the scenes smarter, more accurate, more predictive. A classic example of that would be Netflix back in the day, their rating system, they actually have a feedback loop from their customers then make the system better at predicting. I think people you are still trying to figure out what does that look like in this world in LLMs? Something I hope that you're all building at Slack is a way to ask a bot questions based on all the conversations in the Slack. I've been looking for that product for a while now.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.