Look at Igor Doncov’s recent post Abandonado
The house is the brightest thing here, and since it is the subject, thats where he wants the viewer to look . If there was a big bright white rock at the horizon halfway up the right frame edge, well then that bright rock would have created a distraction.
It’s the luminosity of the sky that can create distractions in an ICM, since I assume you wan the viewer looking at the rest of the image. Small patches of blue sky might work if they fit harmoniously with the rest of the patterns in the image however. It depends…