Andrej Karpathy observes that humans have a process for distilling knowledge into weights, which current large language models lack.
But I still think that humans obviously have some process for distilling some of that knowledge into the weights. We're missing it.