Public evidence record
Published podcast speaker
Books, apps, and tools
book / recommends
“Yes, so three books I often recommend are, first, Story of Real Life by Ted Chang. It's my all time favorite short story and it's about a linguist learning and alien language, and I basically reread it every couple years.”
Claim ledger
19 transcript-backed records
01 / recommendation
“Yes, so three books I often recommend are, first, Story of Real Life by Ted Chang. It's my all time favorite short story and it's about a linguist learning and alien language, and I basically reread it every couple years.”
02 / belief
“I think there is a question of what products they're building and whether those products themselves are something that kind of help or hurt humanity.”
03 / belief
“I think Anthropic takes a very principled view about what they do and don't care about and how they want their models to behave in a way that feels a lot more principle to me.”
04 / belief
“Almost like it's this destiny that their entire life, and experiences, and interests shape them towards. And so I think that principle applies pretty broadly, not just the founders, but the people creating, I think.”
05 / belief
“I think people don't realize that there's a big difference between moving from 80% performance to 90% performance to 99% performance to 99.”
06 / belief
“Like labeling cat photos and drawing bounding box around cars. And so I've actually always hated the word data labeling because it just paints this very simplistic picture when I think what we're doing is completely different.”
07 / belief
“Again, going back to the example I said earlier, certain companies, if you ask them what is good poem, they will simply robotically check off all of these instructions on our list. But again, I don't think that makes for good poetry, so certain frontier labs, the ones with more taste in sophistication, they will realize that it doesn't reduce to this six set of checkboxes and they'll consider all of these kind of implicit, very subtle qualities instead, and I think that's what makes them better at this at the end of the day.”
08 / commitment
“You say pop and I'll wonder where you came from, but I won't score on you too much.”
09 / belief
“I have an idea in my head, but I think the answer to that question maybe reveals certain things about what kinds of AI models those companies want to build and what direction and what future they want to achieve, yeah, so I think about that a lot.”
10 / prediction
“I think one of the things that's going to happen in the next few years is that the models are actually going to become increasingly differentiated because of the personalities and behaviors that the different labs have and the kind of objective functions that they are optimizing their models for.”
11 / commitment
“One is our forward-deployed researchers who are often working hand in hand with our customers to help them understand their models. So we will work very closely with the customers to help them understand, "Okay, this is where your model is today.”
12 / recommendation
“At least you took a swing at something deep, and novel, and hard instead of pivoting into another LLM wrapper company. So yeah, I think the only way you build something that matters that's going to change the world is if you find a big idea you believe in and you say no to everything else.”
13 / evaluation
“It can completely hallucinate. But it will look impressive because it has crazy emojis, and boating, and markdown headers, and all these superficial things that don't matter at all, but it catch your attention.”
14 / preference
“Yes, so the way we really care about measuring model progress is by running all these human evaluations. So for example, what we do is, yeah, we will take Gore human annotators, and we'll ask them, "Okay, go have a conversational model.”
15 / prediction
“I think people don't realize how much it's going to make your systems unmaintainable in the long-term and they simply dump this code into their code bases if this seems to work out right now, so I kind of worry about the future of coding.”
16 / prediction
“I used to work at a bunch of the big tech companies and I always felt that we could fire 90% of people and we would move faster because the best people wouldn't have all these distractions.”
17 / evaluation
“" Some companies, they see these benchmarks and they're like, "Okay, for PR purposes, even though we don't think that these academic benchmarks matter all that much, maybe we just need to optimize for them anyways because our marketing team needs to show certain progress on certain standard evaluations that every other company talks about, and if we don't show good performance here, it's going to be bad for us even if ignoring these academic benchmarks makes us better at the real tasks.”
18 / prediction
“I used to work at a bunch of the big tech companies and I always felt that we could fire 90% of the people and we would move faster because the best people wouldn't have all these distractions.”
19 / preference
“I always thought it was really important for us to have early customers who were really aligned with what we were building, and who really cared about having really high quality data, and really understood how that data would make their AI models so much better because they were the ones helping us.”