Interesting question. In my experience, playing stereo recordings w/ mono switch on = definite loss of sound quality, not only of the stereo separation... it also creates a dull and less present sound. I'm not completely sure how this happens BUT I assume it has to do with the stereo mix not being easily put back together by simply summing L and R.
In playing Mono records with the mono switch OFF... I'm less sure there is a noticeable difference.