Public evidence record
Published podcast speaker
Claim ledger
5 transcript-backed records
01 / prediction
“The current implementation ended up using NVIDIA's library or primitives, but I expect these ideas to be broadly applicable to different hardware.”
02 / prediction
“I think just this change will have a very significant impact in the kind of landscape of open-source AI, where now lots of businesses, lots of companies will be using, I expect will be using things like LLAMA2.”
03 / prediction
“On the machine learning side, people have been thinking more algorithmically. They say, okay, either we can approximate attention, or there's this trick called the online softmax trick, which says that because of softmax, the way it's written mathematically, you can actually break it up into smaller pieces, do some rescaling, and still get the right answer.”
04 / prediction
“Now, this, I don't think, has been validated at scale. So as a researcher, I'm bullish on this stuff because I think in the next couple of years, these are use cases where these alternatives could have an advantage.”
05 / prediction
“We have some ideas. And in the future, I think we will need to design architecture that explicitly has some kind of reasoning module in it if we want to have much more capable models.”