01 / uncertainty
I think a lot of people I've had this exact sort of argument with a lot of people and 1 counterargument that people have towards this idea, I don't know if I fully agree with it, but it's interesting, is that attention heads really are doing something more magical than we give them credit for, which is they are kind of dynamically rerouting and effectively resetting the network based on the context that the prompt is getting.
“I think a lot of people I've had this exact sort of argument with a lot of people and 1 counterargument that people have towards this idea, I don't know if I fully agree with it, but it's interesting, is that attention heads really are doing something more magical than we give them credit for, which is they are kind of dynamically rerouting and effectively resetting the network based on the context that the prompt is getting.”
- Speaker
- Max Bennett
- Publisher
- Machine Learning Street Talk