Evidence receipt / evaluation
Published · transcript-backedNicholas Carlini: evaluation
29 Aug 2024 Latent Space Why you should write your own LLM benchmarks — with Nicholas Carlini, Google DeepMind
“I feel like lots of people, there are far too many X plus AI, where X is like arbitrary thing in the world that has nothing to do with it and could not be benefited at all. And they're just doing it because they want to use the word.”
Source trail
Everything needed to verify it.
- Speaker
- Nicholas Carlini
- Attribution
- Verified speaker
- Claim type
- evaluation
- Recorded
- 29 Aug 2024
- Publisher
- Latent Space
Transcript context
…Right? They don't maybe do a good job. Unless you watch Google I.O., there was no other opportunity to learn that Gemini is now in your Google Sheets. And so I just don't write formulas manually anymore. It just prompts Gemini to do it for me. And it does it. One of the problems that these machine learning models have is a discoverability problem. I think this will be figured out. I mean, it's the same problem that you have with any assistant. You're given a blank box and you're like, what do I do with it? I think this is great. More of these things, it would be good for them to exist. I want them to exist in ways that we can actually make sure that they're done correctly. I don't want to just have them be pushed into more and more things just blindly. I feel like lots of people, there are far too many X plus AI, where X is like arbitrary thing in the world that has nothing to do with it and could not be benefited at all. And they're just doing it because they want to use the word. And I don't want that to happen. You don't want an AI fridge?…
Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.