Ali Behrouz: recommendation

Transcript context

…To. Avoid such cases because when we are in this context, let's say that, for example, let's say that I, I don't know anything about one specific task. And for example, I, I don't know like how to, I, I don't know, like how to paint something like that. And then I want to learn it. So that's, that's a context. That's my context of learning how to paint and the teacher and you know, anyone that is trying to teach me how to like do painting, they can teach me in a very wrong way. And what would happen is that I could simply just learn that because I have no idea about like how to paint or you know, any task I did painting. Here is just one example, but you know, I have no idea how to do it. That's the only source of information that I have. And they are saying that you should do it in this way. So I can simply learn that. But that's only in my context right now. If I want to truly learn, then I will like practice. I will like get feedback from others. I will search about it. Generally, I will gather some some information about like how to do paintings and then I would realize that this is not the best way of learning how to paint. And that's the time I gather all the information compressed, understanding the underlying pattern and so on and so forth. And now that's the time I need to transfer this knowledge to upper levels of, of knowledge abstraction. I mean, it's lower networks. So that's the part I that the model needs to understand how to filter all those adversarial examples, all those examples that are not needed anymore. But yeah, I think that's, I mean, if you want to like think about a continual learner, potentially the part that is responsible for this take for these cases could be the process of knowledge transfer. But also you mentioned like methods like Titan about like adversarial process. There are some like micro methods that we can use. They are not super effective in a in a severe adversarial environment. But on the other hand, at some level at least they can be effective. For example in Titan and also like self modifying Titan and more recent recurrent models, we can see that the learning rate is like learnable parameters and it's input depended when the learning rate in the inner loop of the model in the context, in the process of in context learning of the model, when the learning rate is learnable and we see something that is just a noise, it is adversarial. Example, the gradient or the surprise metric can show a high level of surprise because you know, that's just a noise. It's very surprising we have not seen that. And so potentially it can affect the memory, but that's the responsibility of the learning rate to understand that the surprise metric is high. But this concept is irrelevant and I need to filter it. So gating here acts as a form of gating. So learning rate here act as a form of gating and reinter data specific data sample we have. c is high. But this concept is irrelevant and I need to filter it. So gating here acts as a form of gating. So learning rate here act as a form of gating and reinter data specific data sample we have. It's just a simple way of mitigating adversarial examples that we might feed them in the, you know, in the training process. But still it's not the best way. As I mentioned, potentially the knowledge transfer is the part that we should avoid these cases.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.