01 / observation
We don’t actually publish anything on this right now, but have tracked it a bunch internally in our internal analytics on evals across all the models that we run, where we look at the difficulty to questions and the correlation between token usage and difficulty and net net, surprise, surprise, like models have got.
“We don’t actually publish anything on this right now, but have tracked it a bunch internally in our internal analytics on evals across all the models that we run, where we look at the difficulty to questions and the correlation between token usage and difficulty and net net, surprise, surprise, like models have got.”
- Speaker
- Micah-Hill Smith
- Episode
- Artificial Analysis: Independent LLM Evals as a Service — with George Cameron and Micah-Hill Smith
- Publisher
- Latent Space