George Church: evaluation

Transcript context

…Are you more excited about AI which thinks in protein space, or capsid space, just predicting some biological or DNA sequences? Or are you more optimistic about LLMs just trained on language, which can write in English and tell you, “Here's the experiment you should run” in English. Which of those two approaches, or is it some combination, when you think about AI and bio is more promising? I'm much more excited about scientific AI than I am about language AI. With languages, we're in pretty good shape already. What worries me is that to get to the next level of language requires AGI or ASI. That's very dangerous. I don't think we have quite figured out how to handle that. There's a lot of safety organizations and a lot of safety rules and so forth. What typically happens when there's an intense competition is those safety rules get undermined and pushed aside. Even if they weren't, I don't think we understand our own ethics well enough to educate a completely foreign type of intelligence. We barely know how to pass it onto the next generation of humans. So we need time to sort that out. There's no rush. This is a completely artificial emergency. This is not like COVID-19, where millions of people were dying if we delayed the science. This is something where, if there ever is a crisis, it's because we created it, it's not because we're trying to solve it. So I think we need to go very slowly on AGI and ASI, and double down on slightly narrower scientific goals. With even that, we need to be very cautious. We need to have kind of an international consensus on what constitutes safe AI. Suppose we did build safe superintelligence. How much would that speed up bio progress? There's a million George Churches in data centers just thinking all the time, is it a 10x speed-up?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.