Julie Zhuo: prediction

Transcript context

…So your first question was how are a bunch of AI companies using data? So the funny thing, my funny answer to this is I don't actually think a lot of the fast growing companies are using data well at this point. And the main reason why is because traditionally things just didn't grow that fast. And so, you know, if you got to a hundred million users, your company has probably been around for a while. And if your company has been around for a while, you've had time to set up things like logging and you've hired a growth team at that point, and you've hired a data team and they've like done a bunch of work to log an instrument and then transform the data. And like, we've talked about like, what is the observability for our business? And you just usually had years to build and develop that because of the rate of growth. And so today we see companies that are growing insane and they're still about 10 people or two people or however many people, but they've got hundreds of millions in ARR and hundreds of millions of users. And you know what? They don't actually have all of that infrastructure, that logging and all to be able to truly do data analysis. So I would say that... these these companies are are totally getting by on just good instincts and good vibes and we see that right like you don't really need um a data analysis to sometimes make something that works but i think what data helps us do is it just it it in my mind it's it's sort of is like helping us reflect back what is really reality and so of course if ar is going awesome you know great keep doing what you're doing but what always happens is eventually things stop growing growth does not happen forever and usually when growth stop growth stops everyone has this question of like what's going on why did it happen and then you start to be able to see the power of if you've if you've instrumented everything very well and you have a very good observability model for your business it's much easier to start to get into the root cause it's easier to even predict whether growth will slow down at a certain point. It's easier to catch these trends earlier. If you don't have good observability over how your business runs and what the company's key levers are, then you will be scrambling. And at that point, that's usually when people start investing a ton in data. So I wouldn't say that a lot of these hot companies are quite there yet. But what I also think is a trend is that Every time there's a new technological shift, we actually have to change the way that we think about. Analysis has to answer the questions that we have. And if technology changes or context changes, we need new methodologies of analysis. So, for example, when mobile came to the forefront, looking at sessions or sessions per day or time spent on mobile or you know kind of length of sessions became something that was important for us to understand are people getting value in this new medium i ng at sessions or sessions per day or time spent on mobile or you know kind of length of sessions became something that was important for us to understand are people getting value in this new medium i think that's the same with what we have today conversational analytics is totally different used to be let's say in the google world right i knew you were interested in shopping if you click the shopping tab i know you're interested in maps if you click the maps tab we can measure clicks Today, it's just all conversation. And so it's actually harder for us to tease apart, what is the user intent? If I worked on any of these LLM, I would say like one of the, probably the biggest questions is, hey, what use cases are growing or what use cases are shrinking? And that's much harder to tell today because it's not just clicks on tabs or pages. It's like, we have to probably use an LLM to, or machine learning model to bucket user intent. We probably have to ask questions like, Is the flow going really well in conversations? If I just ask one question and I don't go back and forth, did the user get value? It's always trying to get back to like, we're trying to figure out if this was a good experience, but now it's like, we need to actually invent new methodologies to help us analyze that. Yeah, I think the question is always, like with conversations, do you want it to be a long conversation? Do you want to be a short conversation? Like, what's the right answer? What's better? I had the head of Chatship ET on the podcast, Nick Turley, and it turns out one of the ways they found the most common use cases early on was watching TikTok comments and things going viral on TikTok after they launched. How about that?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.