David Rosenthal: evaluation

Transcript context

…If I’m remembering from our episode, basically what happened is the AlexNet team did way better than anybody else had ever done. The complete step changed for the better. I think the error rate went from mislabeling images 25% of the time to suddenly only mislabeling them 15% of the time. That was a huge leap over the tiny, incremental progress that had been made along the way. You are spot on. The way that they did it, and what completely changed the fortunes of the Internet of Google, of Facebook, and certainly of Nvidia, was they actually used old algorithms, a branch of computer science and artificial intelligence called neural networks, specifically convolutional neural networks, which had been around since the 60s. But they were really computationally intensive to train. Nobody thought it would be practical to actually train and use these things, at least not anytime soon or in our lifetimes. What these guys from Toronto did is they went out probably to their local Best Buy or equivalent in Canada. They bought two GeForce GTX 580s, which were the top-of-the-line cards at the time, and they wrote their algorithm, their convolutional neural network in CUDA, in NVIDIA’s software development platform for GPUs, and by God they trained this thing on $1000 worth of consumer-grade hardware. And basically the algorithm that other people had been trying over the years just wasn’t massively parallel the way that a graphics card enables. If you actually can consume the full compute of a graphics card, then perhaps you could run some unique novel algorithm and do it in a fraction of the time and expense that it would take in these supercomputer laboratories.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.