Self-supervised algorithms learn more concepts when trained on redundant data, as redundancy reveals internal structure.
those self supervised algorithms that learn by prediction, even in representation space, they learn more concept if the input data you feed them is more redundant. The more redundancy there is in the data, the more they're able to capture some internal structure of it.