Katie Dill: belief

Transcript context

…is that they get so focused on that they forget about that piece of the journey, and how it all fits together, and not recognizing that part of their product experience is intimately tied to another. So, what we did was we set out to, number one, increase the kind of awareness and accountability of leaders to own their journeys. What we have established are we started with 15 of our most important user journeys. 15 is somewhat of an arbitrary number. It's a number that we can kind of keep track of, but also has pretty good breadth, but is certainly not comprehensive of all the most important things. But 15 of our critical user journeys, the things that we know matter so deeply to our users, and we must get right at the highest level of quality possible. Those 15 things then each have engineering, product and design leaders that are responsible for the quality of those products. They review these journeys, what we call walk the store, where they review them as if they're walking the floor of their store on a regular cadence, and they friction log what they experience, which I know David Singleton talked about on your podcast. They will write what they have seen, what's working, what's not working. They're viewing this from, they're trying to put themselves in the shoes of their user. This of course doesn't replace user research, but it substitutes it and it adds to that. So, they go through the experience and noting what's working and what's not working. And very critically, it's a journey, so a lot of times it starts from internet search, it starts on Google trying to understand something, goes to the website, they end up on Docs, they end up in the dashboard, and they're seeing it as a user might. With that, they're able to find the entailments of the experience that may or may not be working. They jot that down, they file bugs, they reach out to the teams that may own the different parts of this experience, and then they score it. Then on, again, a regular cadence, we come together in almost like a calibration, where we meet and we talk about the score of their work. It relates to performance reviews. Performance reviews, managers are assessing an individual's performance, which is hard. There's some subjectivity to it, just like understanding quality can be. But what we do as managers is we calibrate. We come together and we talk about, "Okay, how well is our interpretation of our ladders document? And how well does that performance align? And are we doing it consistently across the rest of the organization?" So, we do something very similar. We calibrate these scores because what we're really trying to do is not just the 15 essential journeys and the owners of those, we want to actually uplevel and bring more shared understanding of our quality bar across the company. These moments of calibration start that. Then having leaders do this creates this, number one, it cascades this idea of the importance of owning your journey and then also has upstream impact. ompany. These moments of calibration start that. Then having leaders do this creates this, number one, it cascades this idea of the importance of owning your journey and then also has upstream impact. Because when people see the state of products in the wild as a user would, they learn a lot about what are some of the bigger opportunities that we can make to make the product better? What are some of the things that maybe we want to change in our process to make sure that we have even better things coming into the wild? One of the best parts of this is, since then we've learned that folks have seen that like, "Oh my goodness, our SEO for this particular product or the way we're articulating it doesn't align to actually how we want people to understand it later on in the journey. So if we improve this over here, we're going to improve outcomes later on." They're seeing that and they're now able to make that happen even faster to make some of the changes there. Then my real favorite part is that we're hearing from folks that maybe at first didn't see this as necessary, that maybe in different functions that are just like, "Oh, I was so very focused on executing the technical ability of what I do on this thing, but I hadn't seen it from this lens before." Now there're actually converts of like, "Yes, this is a really important part of it." That goes back to the point of it's a group effort. You don't want just one function looking out for the quality of the product. So, having engineers and product managers and people of different disciplines walking the store, seeing the experience, feeling it firsthand, I think will lead to better care in all of the details that will aligned to better craft in the end. Oh man, what an awesome process. I have a million questions I want to ask to better understand how you operationalize this. I'll try to ask just a few. But one thing that stood out about this process is I think people kind of don't trust their own judgment when they're looking at their own product. They kind of, especially product managers, almost have to feel like they have to rely on user research or data to know a thing versus like, "I just see this and it feels bad to me." I think I've learned over time more and more that you should really trust that, because you're spending your energy trying to use this thing, you're not that different from a potential user. So, I love that this actually relies on your personal judgment trying to use a thing, which I think people undervalue.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.