Reiner Pope: evaluation

Transcript context

…But it includes the initialization of zero. Yeah. We want to have quadratically more compute. We have x times y as much compute as we had before. But we want to aim for having only x times as much communication. The intention is to get this advantage term going as y. We’ve laid down the multiplications. We want to bring in a vector of size two, and that is already in line with our columns target. That’s fine. However, we need to manage the communication of this matrix, which exceeds our budget of x. The idea is that in an AI context, this matrix is going to stay fixed for a long period of time. We’ve got some register files sitting over here. The amount of stuff coming out of this register file… this is the term that we want to go as x, in some sense. We don’t want to bring this full matrix in from the register file every cycle, because that would cost too much in terms of wiring from the register file. Our key trick is that this matrix can be stored locally to the systolic array. We’ll store these numbers 0, 1, 2, and 3 in a gate called a register that physically stores these numbers, and we’re going to reuse these numbers over and over again for a large number of different vectors. The optimization here is that the nature of matrix multiplication is that you can store this square quadratic thing directly where the logic is happening, which has an extra dimension compared to the inputs that you keep swapping in and out.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.