Ben Gilbert: evaluation

Transcript context

…Just generational talent. But this is that early nucleus of engineers that Google recruited. It’s amazing that they attracted them because prospects were not good that all of this would work in scale. It was only because of these guys that it did. Here’s the crazy thing. Later—there’s an easy point to make—Google got to hoover up all the best talent because they were a solid business after the dot-com crash. But this in 98–99, we’re in the go-go times. The dot-com bubble hadn’t burst yet, and Larry and Sergey managed to recruit this talent. I think this is like a ‘history turns on a knife point’ or a ‘make or break the company’ thing. The fact that they were able to get these guys in a hot talent market really speaks to Larry and Sergey’s vision, the excitement around the idea, how novel their approach was, everything. And part of the reason why this talent was attracted to Google, sure, some of it was like, oh, the product’s really good and people are using it, so that makes the company interesting. The other part of it, though, is that the technical challenges and the architecture coming out of Stanford was super unique and novel. This was a really, really interesting thing to work on. Why was that? The Google index that they needed to build and operate on for the search engine, for PageRank to work, was so much bigger than any other index out there. Google needed the entire page to compute all the rankings and find the links, find the backlinks. They needed to architect Google with this huge distributed computing system. The index was so big that it wouldn’t fit on a single machine or a single server no matter how big or how expensive. What they do to store the index and to operate on it with this distributed file system, is they break the giant index into tons and tons and tons of little chunks, they’re called, of individual 64 megabyte files. Small files. Tractable files. They get stored on lots and lots of different disks, lots of different machines, lots of different servers, and ultimately different data centers all over the world. Then there’s a separate server that keeps a master mapping of all the chunks, where the chunks physically are. When a query comes in and needs to operate on the index data, the master server just returns only the chunks that it needs, not the whole index. That makes the whole thing possible.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.