Rebecca Hinds: belief

Transcript context

…So speaking of measurement, one thing that I was kind of pondering myself as I was imagining myself leading an organization through these challenges based on all the findings in the report, and I don't know if we said this yet, but 69% of people admit to some form or some amount of bot ******** which I think has like a couple different definitions, but for me, it's like passing off AI work in, down the line somewhere where you yourself cannot defend the quality of the work. That's a very high rate. So it does strike me that, again, kind of my earlier thought, in a way, that's maybe a really optimistic thing for how well AI is working, because if you already have two-thirds of people just doing AI outputs blindly, sending them down the line, and the wheels aren't falling off entirely, that's like kind of an amazing finding. But then if I'm trying to manage that as a leader, I'm thinking, how do I detect who's doing it? And how do I detect where it's actually working? And maybe I do want to share that with with employees, maybe I do want to partially share it. I'm not sure exactly what the right level of transparency would be there. But have you seen anybody, you know, I've seen also recently, historically, my attitude or my synthesis of available information has been AI detectors don't work. So like when I go to present to a group of teachers, I'll say, don't do AI detectors, or at least if you do, you know, you certainly can't they can be wrong. So you can't trust them too much. These days, it does seem like Pangram Labs is getting a lot of praise in the general discourse around being pretty reliable. So I'm kind of wondering, should I be, if I'm a leader, adopting like something like Pangram Labs and having, you know, all these intermediate work outputs evaluated such that I can potentially both realize who's doing this, but also like maybe where which, again, this is kind of an angle on which nodes in my graph of activity that constitute my enterprise, which ones can I actually just have AI do? Like I potentially already have a lot of answer to that question in the work product that people have, the AI work product that people have passed off as their own. Do you see anything like that? Yes, and I think it's so early. And also there's not going to be a right answer for every organization. A lot of it depends on the psychological safety within the organization. I've long followed Amy Edmondson's great work and I think psychological safety has never been more important in our organization because ideally you have the people raising their hands and saying, this is bot **** or I contributed to bot **** and here is why. I think AI detectors, you know, there is for sure, and I'm seeing it from multiple different angles, you know, there will be a world where all of our AI tools will flag the level of uncertainty associated with the response as well as the likelihood of purely AI generation versus human to AI. I think what's exciting about the more context you have as well is you can feed it more data points. You can feed it not just generically is this likely to be generated by AI. If it knows enough about you as an individual, it can know, okay, Rebecca's default writing style as a human is this versus doing something generic. And I think I think that's why we're seeing a lot of detectors not work. Why I think this isn't necessarily and certainly isn't strictly a technology issue or solution is organizations need to understand the why. The why behind the bot ******** is just as important as the bot ********. In particular, when we think about using shadow AI tools, right? That is a form of bot ********. You're injecting risk into the organization by using an unapproved tool or a sanction in an unapproved way, we're seeing a cohort of organizations crack down and punish employees for doing that. And then we're seeing a portion that definitely have guardrails and repercussions for unsanctioned use of the technology, but not strictly that. It's understanding why are employees using unapproved tools or why are they coloring outside the lines because usually it's the highest performers that are doing that. And usually they're doing that because the existing tech stack isn't working for them. They see so much potential in the technology and they're making that calculus in their head. Hey, I would rather get the productivity gains. In the best cases, hopefully get the gains for the organization. And they're doing this trade off. Ideally, organizations recognize that and make the safe path, the more efficient path. That is the gold standard. I think part of it is a technology piece, but part of it is deeply human. Another stat that jumped out at me was, I believe both the people that do the most bot sitting. So doing, first thing that should be said is doing more bot sitting is associated with being more successful with AI. So both probably because You need to do it to get good results and because you are going to naturally do more of it the more you use AI in the simplest analysis. But then also the more bot sitting seems to be associated with higher likelihood of being on the job market and actively looking for your next phase of your career. And the same is also true on the bot ******** side. People that are doing more of that are again more likely to be looking for a job. That sounds scary and it might be scary. I guess another way I might interpret it if I think about the person who was in the traditional sort of customer service or customer success role and is now being asked to babysit bots, is maybe some of this turnover could in fact be healthy. If people are like doing something they don't want to do, I guess a big question in general with AI is like, Obviously, is it going to create more jobs than it destroys? But even more locally and focused on specific organizations and teams and individuals, is the new job that gets created, whether it's greater or less than one per job that gets eliminated, is it something that the person that had the original job can pivot into or would want to pivot into? And I guess my read on some of this stuff is like, it sort of suggests that in a lot of cases, the answer is no. People that are doing a lot of this AI stuff, good and bad, it sounds like are kind of voting with their feet that this is not really what they want. Maybe they don't see themselves being successful in this new world. And so I guess I wonder, do you see that as a sign, do you see that as something that like leadership is doing wrong? Or do you see it as sort of an actual signal of just change is often not what people want. And so they're kind of reacting in a ultimately what might be a perfectly sensible way.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.