Evidence receipt / belief
Published · transcript-backedAlessio Fanelli: belief
1 Feb 2024 Latent Space Why StackOverflow usage is down 50% — with David Hsu of Retool
“Obviously, there's like all these different things like privacy, you know, if an internal tool hallucinates, that's fine, because you're paying people to use it basically, versus if it hallucinates to your customer, there's a different bar. Because for you, if people build internal tool with retool, there are external customers to you, you know, so I think you're on the flip side of it.”
Source trail
Everything needed to verify it.
- Speaker
- Alessio Fanelli
- Attribution
- Verified speaker
- Claim type
- belief
- Recorded
- 1 Feb 2024
- Publisher
- Latent Space
Transcript context
…Our sense, basically, the survey, and this is true, we believe, too, is we are most when we do engineering interviews, we are most interested in assessing like critical thinking or thinking, you know, on the spot. And I guess, you know, when you hire the employee, you know, in the end, the job of employees to be productive, which they choose whatever tools they want to be productive. So, you know, that's kind of our thinking, too. However, we do think that, you know, if you think about it from a first person's way, if your only method of like coding is literally copy pasting, you know, off of chat GPT, or like, you know, it's pressing tab and copilot, I think that would be concerning. And so, for that reason, we still do want to test for like, you know, fundamentals understanding of comp sci. Now, that said, I think if you're able to use chat GPT or copilot, let's say competently, we do view that as a plus, we don't view it as a minus. But if you only use copilot, and you aren't able to reason about like, you know, how to write a for loop, for example, or how to write fizzbuzz, that would be highly problematic. And so, for us, we do today is we'll base a screen share, or a rest is a hackpad, actually. So it's, sorry, this is no copilot there to sort of see what they're doing, or see what they're thinking. And we really want to test for thinking, basically. But yeah, I mean, we ourselves internally have embraced copilot, and we would encourage engineers to go over this copilot too. But we do want to test for understanding of what you're doing, rather than just copy pasting a copilot. The other one was AI adoption rate, only 27% are in production. Of that 27%, 66% are internal use cases. Shout out to retool, you know, do you have a mental model as to how people are gonna make the jump from like, using it internally to externally? Obviously, there's like all these different things like privacy, you know, if an internal tool hallucinates, that's fine, because you're paying people to use it basically, versus if it hallucinates to your customer, there's a different bar. Because for you, if people build internal tool with retool, there are external customers to you, you know, so I think you're on the flip side of it. Yeah, I think it's hard to say, maybe a core retool belief was actually that most software built in the world is internal facing, actually, which actually sounds may sound kind of surprising, you know, for some of you hearing this, but effectively, like, you know, we all work at Silicon Valley, right? We all work at businesses, basically, that sell software as, you know, as sort of a business. And that's why all the software engineers that we hire basically work on external facing software, which makes sense with most software companies. But if you look at most companies in the world, most companies in the world are actually not software companies. If you look at like, you know, the clothing manufacturer that I was talking about, they're not a software company, like they don't sell software, you have to make money, they sell clothing to make money. And most companies in the world are not software companies, actually. And so most of the engineers in the world, in fact, don't work at Silicon Valley companies, they work outside of Silicon Valley, they work in these sort of more traditional companies. So if you look at the Fortune 100, for example, probably like 20 of them are software companies, you know, 480 of them are not software companies. That's the employable software engineers. And so most of the software engineers in the world, and most of the code engineers in the world actually goes towards these internal facing applications. And so, for all the reasons you said there, like, I think hallucination matters less, for example, because they have someone checking the output, and consumer, so hallucination is more okay, it's more acceptable as well. Yeah, it can be unreliable, because it's probabilistic, and that's also okay. So I think it's kind of hard to imagine AI being adopted in a consumer way without the consumer like opting in, like, Chachapiti is very obviously a consumer, the consumer knows that it's Chachapiti, they're using it. I don't know if it's going to make its way to like the banking app anytime soon. Maybe for like, even for support, it's hard. Because if it hallucinates, then, you know, it's actually quite bad for support if you're hallucinating, right? So it's, yeah, it's hard to say. I'm not sure.…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.