Public evidence record
Published podcast speaker
Claim ledger
15 transcript-backed records
01 / evaluation
“Yeah. So the tricky thing about vision is I think the models are getting significantly better, especially if you look at the last six months, natively being able to do like VQA stuff, and so on.”
02 / uncertainty
“In my opinion, the places you'll see a real difference is like, I don't know, a small part of the tail, especially in like this kind of an open domain setting.”
03 / belief
“The other point about the more number of websites, I think, again, it comes down to the number of websites.”
04 / belief
“I think the markdown step definitely helps in terms of, you know, there's a lot of noise, like as you can imagine with the pure HTML.”
05 / uncertainty
“Like, like the problem I described might be obsolete six months from now. And I don't know.”
06 / belief
“I think the, for us, our primary goal is like solving the deep research user value for the user use case.”
07 / belief
“Do we, do we have like a, do we have like a, do we have like a, do we have like a, do we have like a new model is like a new model, produce really longer, many more steps, number of characters, like number of steps in case of the plan in the plans, it could be like, like we spoke about how it iteratively plans based on like previous searches, how many steps does that go on an average or some dev set. So there are some things like this you can automate, but beyond that, there are all generators, but we definitely do a lot of human evals and that we have defined with product about certain things we care about.”
08 / belief
“I think for us, like it's we often like live and breathe within Google and which is like a really big place.”
09 / belief
“Like some of these things, depending on how deep you want to go, you might just want to quite g thermometer versus like kick off another deeper search. And even from a UX perspective, I think the, the panel allows for this notion of, you know, not every fall up is going to take you.”
10 / commitment
“You'd actually want the model to be like, okay, this is sufficiently different that I want to go do more deep research to answer this question. I won't find this information in what I've already browsed.”
11 / commitment
“When we built deep research, I think there was a few things that we took a few different bets, uh, around how this, how it should work.”
12 / preference
“I think when deep research really shines is there like multiple facets to your question and you spend like a weekend, you know, just opening like 50, 60 tabs and many times I just give up and we wanted to solve that problem and, and give a great starting point for those kinds of journeys.”
13 / preference
“First, I think the thinking style models definitely help here because they are significantly better on how they reason natively and being able to draw these second order insights, which is like very premise.”
14 / evaluation
“Like, there are things that you might not optimally spend time verifying, even though the models like, like, this is a very common fact the model already knows and it's able to like reason over and balancing that out between trying to leverage the model memory versus being able to ground this in, is in, you know, some kind of a source is the challenging part. And I think as, as like you rightly called out with the thinking models, this is even more pronounced because the models know more, they're able to like draw second order insights more just by reasoning over.”
15 / recommendation
“I think there's definitely some things we can do on the UX side to basically invite the user to be like, Hey, this is the starting point.”