Andrej Karpathy notes that solving the problem of basic perception that's robust, building representations, and having a model with common sense is a significant part of the time taken for self-driving.
If somebody's waving down the road this way, you don't need to train for it. The thing will have some understanding of how to respond to something like that.