Jacob Kimmel: uncertainty

Transcript context

…You're flattering our listeners. “Only cringe listeners will appreciate it, but your audience will love this.” I don't know about your audience, but you will. You can think about it like this. If you think about how attention works—queries, keys, values—TFs are like the queries. The genome sequences they bind to are like the keys. Genes are like the values. It turns out that that structure then allows you to very efficiently, in terms of editing space, change just one of those embedding vectors, in this case one of those sequences, and get dramatically different performances or total outputs. So I do think it's interesting how these structures recur throughout biology, in the same way that the attention mechanism seems to exist in some neural structures. It's interesting that you can very easily see how that same sort of querying and information storage might exist in the genome. Interesting. A previous guest and a mutual friend, Trenton Bricken, had a paper in grad school about how the brain implements attention.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.