Rebecca Hinds: evaluation

Transcript context

…Cool, interesting. The mix of company context and individual context is definitely a pretty live discussion. I'm thinking of Dan Shipper and every team and how they've kind of experimented with various versions of this and already evolved their approach in a meaningful way. I have my own pretty elaborate personal AI infrastructure at this point that is kind of an extension of me in one lane and then also increasingly like trying to get it to be sort of an employee that can do its own thing with more and more autonomy as we go over time as well. Maybe let's again come back to that. I want to make sure we don't bury the lead on the headline of the report, which I do think is interesting food for thought at a minimum. So I'll give you the headline and you give me the expanded version of it. 87% of people that you surveyed are using AI. 73% say it makes them more productive. And the one that really blew me away, on average, they say they save 13 hours a week thanks to their use of AI. That's 1/3 of a work week. Some caveats to come, obviously, but that's a lot of time, like a really strikingly large number. But then the flip side of this is only 13% of the survey respondents say that their organization as an organization is performing much better than it used to be based on all this AI that's happening. Those are the big headlines that jumped out to me. Give me the double click on how we should start to understand that. Sure, and I think you've hit the nail on the head with your previous set up to the question and that, we're seeing this massive disconnect between individual productivity gains and organizational performance. Now, this is survey data, so it's, inherently biased in terms of how people are reporting. Depending on how we ask multiple questions around time savings and, depending on the question, we saw ranges from 10 hours per week to 14 hours per week. On average, what we're seeing is people are reporting that based on the work output, they have fully automated with AI. It's about 11 hours of time savings. But when you ask them about the organizational performance, do they see their organization perform significantly better as a result of the technology? We see this gap, just 13%. The fact that 13% of employees only 13% are saying that their organizations are significantly performing better or performing better significantly because of the technology. And it's a big disconnect that we've seen in multiple different studies. We've certainly seen in the headlines the budgets that don't pay off, the ROI that doesn't pay off. What's novel about this report, this Work AI Index is where we're theorizing the time savings is going. And we're coining this phrase bot sitting as the hidden human labor that is required to make the technology usable. I think we all feel it on some level. It's the feeding AI context. It's the overseeing the AI. It's the debugging the AI. It's the cleaning up after the AI. And it's a massive chunk of labor, on average, upwards of six hours per week workers are spending on this bot sitting activity that is often tedious. You make a point in the report to point out not all bot sitting is negative, certainly. There are healthy forms of bot sitting, but more often than not, it is negative, it is tedious, it is exhausting, it is not rewarded, and it's not appreciated or tracked or measured and certainly not incentivized within the organization. And I think that is feeding a lot of this failed expectations, the an execution gap that we see across the board, not recognizing the human labor and recognizing that this isn't in most cases a problem in terms of the individual employee. This is a systematic issue where we've invested in tools that don't have that context. We've invested in tools in silos that now we're seeing AI sprawl and agent sprawl. And it's a whole host of different factors that are coming from multiple different directions to feed this bot sitting which then turns into what we're calling bot ********. Bot ******** is the dissipation of work that hasn't been checked, that workers can't explain. We see 40 to 41% of employees saying they ship AI work that they couldn't explain if asked. This is bot ******** and I think it's pervasive. We often see the polished nonsense come where it looks polished, it looks finished, it looks like we've done the assignment. When we dive a little bit deeper, it's actually quite hollow. 's pervasive. We often see the polished nonsense come where it looks polished, it looks finished, it looks like we've done the assignment. When we dive a little bit deeper, it's actually quite hollow. There's very little substance. That's part of it. That's the most visible form. But it's also, it's the shadow AI. It's the using AI that you can't explain in terms of the output. It's a whole host of different dimensions as well.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.