Evidence receipt / recommendation
Published · transcript-backedJudd Antin: recommendation
4 Jan 2024 Lenny's Podcast The UX research reckoning is here | Judd Antin (Airbnb, Meta)
“You can't interact in a world of AB testing without knowing basic statistics. And then in the old version of this, the fifth tool was SQL, because I think good researchers need to be able to run their own queries.”
Source trail
Everything needed to verify it.
- Speaker
- Judd Antin
- Attribution
- Verified speaker
- Claim type
- recommendation
- Recorded
- 4 Jan 2024
- Publisher
- Lenny's Podcast
Transcript context
…This may be a hard question to answer, but when people hear, "If you have a great researcher, here's how you approach it." What are signals that your researchers is great versus not great? What are some things people could look for to tell them, like, "Oh, maybe I have the wrong researcher on my team." The best researchers I think are first of all, multi method. The first iteration of user research was primarily a qualitative discipline. But a strong opinion that I have is that is largely one of those models that needs to evolve. It's not that qualitative user research is no longer important. It's that the best researchers have five tools. I think they have five tools. And those five tools are number one, what we would call formative or generative user experience research. So looking ahead, innovation focused, really open-ended, maybe more ethnographic, "Let's go out into the field and talk to host and guests on Airbnb. Let's see people using our product in the field," stuff like that. So that's formative. The second type is evaluative, so more like usability testing. The third tool is a basic rigorous survey design. It's the best scaled way to get responses from communities small and large. You can get a lot out of really well crafted surveys. But to do that, you have to have the fourth tool, which is applied statistics, the best research, know a little bit of stats. You can't interact in a world of AB testing without knowing basic statistics. And then in the old version of this, the fifth tool was SQL, because I think good researchers need to be able to run their own queries. These days, so much of that is dashboarded, that the fifth tool may now be prompt engineering, which is a thing we could talk about, but I think maybe that's the fifth tool is somewhere, is it technical skills that fall in between querying your own data, understanding it very well in companies that are awash with data and then interacting with generative AI. Amazing. That's such a cool list. Okay, so just to playback, formative, generative, innovative skills to think bigger and come up with new ideas, usability.…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.