Eric Jang: belief

Transcript context

…I think automated scientific research is one of the most exciting skills that the frontier labs are developing right now. It’s important for everyone who’s doing any kind of research to get a good intuition of what it can do now and what it can’t, and how the science process might work in the future once we have AIs automating a lot of this investigation. In brief, I mostly used Opus 4.6 and 4.7 while working on this. What works is that the models can do a very good job of hyperparameter optimization. In the past, people would come up with a search space of hyperparameters like learning rate, weight decay, and maybe how many layers are in your network. They would do a grid search or a Bayesian hyperparameter optimization approach, and it would find some tuned parameters. The really cool thing that automated coding can do now is search a much more open-ended set of problems. It can say, “I’ve identified that the gradients are small in this layer, so let me change it up here. Let me rewrite the code so the data loader has a new augmentation I came up with. Let’s try to find the best way to fit the constraints of the optimization problem.” You end up with this much more flexible, high-level, almost grad-student-like ability to just grind a performance metric. This can squeeze out quite a lot of performance. On a fixed data set with a fixed time budget, you can improve perplexity by quite a lot on a classification problem like LLMs or Go. It is also fantastic now at basically executing any experiment. I have a Claude Skill that I wrote called Experiment where I give it a description of what I want it to plot. I just describe, “Here’s the x-axis I want, here’s the y-axis. Answer this question for me.” It’ll run off and do all the experiments, compile the plot, make a report, and suggest what might have caused it and so forth. That’s what works quite well today, and I think we can expect these abilities to get better in the future. But it’s also useful to know what it’s not doing so well today. In the blog version of this tutorial, I have a plot of all the experiments I did grouped in a tree, where every node represents a failed, successful, or mixed experimental result. From there, it branches off into a child representing the follow-on experiment. Occasionally, I’ll rabbit-hole down a track like off-policy MCTS relabeling, do a few experiments, and then realize it’s probably not worth it. So then I’ll jump to a completely different track. I call these things rows. What I find is that the current closed models the public can access today don’t seem to be that great at selecting what the next experiment should be in a given track. They don’t seem to be able to step back and do the lateral thinking of, “Wait a minute, this track doesn’t really make sense. at great at selecting what the next experiment should be in a given track. They don’t seem to be able to step back and do the lateral thinking of, “Wait a minute, this track doesn’t really make sense. Let’s go back to first principles and think about what the bottleneck might be, or what we are trying to achieve.” Often I had to catch infra bugs myself by prompting the right question to Claude to investigate what’s causing the discrepancy, and then it’ll answer the question. With Mythos-class models or Mythos++ models coming online, maybe this just completely changes and these problems fall to improved scaling. But at the same time, I think there’s a rich opportunity to develop RL environments that might incentivize this kind of lateral thinking. One of the motivations for setting up this Go environment was that Go captures a lot of very interesting research problems, often overlapping with LLMs or robotics. Yet it’s very quick to verify. The outer loop is ultimately: does the agent do what I think it does? You can check the outcome of a Go game quite easily. The inner loop involves all this research engineering around distributed systems, predicting whether an idea is going to work or not, and predicting the difference a particular modification to your training algorithm might make. I think there’s a rich library of subtasks and sub-environments that you can train an automated scientist to work on, with Go as a sort of outer verification loop. Once you acquire these skills, maybe you can apply them to other domains like biosciences or robotics. Or automating AI research.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.