Jacob Kimmel: belief

Transcript context

…You basically described one of the models you guys are working on at NewLimit. You're training this model based on this data where you're taking the entire transcriptome and just labeling it based on how old that cell actually is. If you've got all this data you're collecting on how different perturbations are having different phenotypic effects on a cell, why only record whether that effect correlates with more or less aging? Why can't you also label it with all the other effects that we might eventually care about and eventually get the full virtual cell? That's a more general purpose model, not just the one that predicts whether a cell looks old or not. Absolutely, we actually do both today. We can train these models where the inputs are a notion of what that cell looked like at the starting place, here's what a generic old cell looked like, and then representations of the transcription factors themselves. We derive those from protein foundation models. They're language models trained on protein sequences. It turns out that gives you a really good base level understanding of biology. The model's starting from a pretty smart place. Then you can predict a number of different targets from some learned embedding, the same way you could have multiple heads on a language model. One of those for us is actually just predicting every gene the cell is expressing. Can I just recapitulate the entire state and guess what effect these transcription factors will have on every given gene? You can think about that as an objective rather than a value judgment on the cell. I'm not asking whether or not I want this particular transcriptome. I'm just asking what it will look like. Then we also have something more like value judgments. I believe that that transcriptome looks like a younger cell. I'm going to select on that and train ahead to predict it where I can denoise across genes and then select for younger cells. But you could do that for arbitrary numbers of additional heads. What are some other states you might want? Do I want to polarize T cells to a less inflammatory state in somebody with an autoimmune disease? Do I want to make liver cells more functional in a patient who's suffering from certain types of metabolic syndrome, maybe even orthogonal to the way that they age? Do I want to go in and change the way a neuron is functioning to a different state to treat a particular type of neurodegenerative disease? These are all questions you can ask. They're not the ones we're going after, but that is the more general, broader vision. This is so similar to, in LLMs, you first have imitation learning with pre-training that builds a general-purpose representation of the world. Then you do RL about a particular objective in math or coding or whatever that you care about. You are describing an extremely similar procedure where first you just learn to predict perturbations in genes to broad effects on the cell. That's the pre-training, just learning how cells work. Then there's another afterward layer of these value judgments of, “How would we have to perturb it to have effect X?” That actually seems very similar to “How do we get the base model to answer this math problem or answer this coding problem?” I don't know if people usually put it this way, but it actually just seems extremely similar. That makes me more optimistic on this. LLMs work and RL works.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.