Andrew Gordon Wilson: belief

Transcript context

…But maybe maybe just to put this into perhaps more familiar territory for other Bayesians out there, you know, like the assignment of priors just for parameters. It's always the goal to try and find a prior that's relatively ignorant, but encodes some very soft, you know, type of constraint. Like, maybe it's a a scale invariant parameter or a locate or a scale invariant prior or location invariant, you know, kinda prior. And overall, a lot of times these priors are maybe worth, like, 1 data point or 2 data points, but they're small enough that they can be overridden quite easily by enough data. But even that small amount is enough to avoid sort of stupid answers. Like, you know, that that like the chance of a head is infinity or something like that. Is it kind of analogous to that? Or I think that's a reasonable analogy. I would also add that we can't get away from making assumptions. So even though I'm in favor of embracing expressiveness and having relatively soft biases for certain types of solutions as opposed to hard constraints, Machine learning means learning by example, and we can't do that without making assumptions. The question is just what assumptions should we be making and at what level of abstraction. And perhaps it's enough in a surprisingly large array of different problems to embrace expressiveness in combination with some sort of simplicity, some Occam's razor bias that can be formalized in terms of compression. Empirically, simple models work better.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.