In your 'line out' settings on the Port, do you use 'fixed or variable'? It matters even if you use the digital out with an external DAC. Fixed is best. If you use the variable, put the level at around 75 to 85%. Your are streaming FLAC correct?
The internal DAC in the Port is adequate for...