Jacob Kimmel: evaluation

Transcript context

…So we can think of these transcription factors as these basis directions, and you can get a little bit of this thing, a little bit of that thing and some combination. And evolution has designed these transcription factors to…Is that your claim? They have relatively modular, self-contained effects that work in predictable ways with other transcription factors and so we can use that same handle to our own ends? That would be very much my contention. One piece of evidence for this is that's the way development works. It's a crazy thing to think about, but you and I were both just a single cell. Then we were a bag of undifferentiated cells that were all exactly alike. Somehow we became humans with hundreds of different cell types all doing very different things. When you look at how development specifies those unique fates of cells, it is through groups of these transcription factors that each identify a unique type. In many cases, the groups of transcription factors, the sets that specify very different fates, are actually pretty similar to one another. Evolution has optimized to just swap one TF in or swap one TF out of a combination and get pretty different effects. You have this sort of local change in sequence or gene set space leading to a pretty large global change in output. Likewise, many of these TFs are duplicated in the genome. Because mutations are going to be random and they're inherently small changes at the level of sequence at a given time, evolution needs a substrate where, in order to function effectively, these small changes can give you relatively large changes in phenotype. Otherwise it would just take a very long time across evolutionary history for enough mutations to accumulate in some duplicated copy of the gene for you to evolve a new TF that does something interesting. I think we're actually in most cases in biology—due to that evolution constraint, small edits need to lead to meaningful phenotypic changes—in a relatively favorable regime for generic, gradient-like optimizers. It would be a little bit overstating to say evolution is using the gradient, but there is a system. If you've heard of evolution strategies, where basically the way you optimize parameters is you can't take a gradient on your loss. So you make a bunch of copies of your parameters, you randomly modify them, and then you compute a gradient on your parameters against your loss, and so you can take a gradient in that space. That's how I imagine evolution is working. So you need lots of those little edits to actually lead you to have meaningful step sizes in terms of the ultimate output that you have. Interesting. You're just like designing a little LoRA that goes on top.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.