It's hard for me to start.
Well, all is important.
Let's start with two sentences as references.
Forgot the sources to these quotes:
"American wise, speaker is the most important component."
"English wise, source is the most important component."
By adapting these two ideas, one could build a system using the best American speakers component and the best English source component. Congratulations, best stereo/mono system ever on earth. (maybe)
If for example, one is upgrading the source first and the speaker latter, then he/she would not realise how different a source could "sound" from one to another.
But if one is upgrading the speaker first, then the amplifier, then the preamp, then the cartridge, one would experience the "different taste" of phono carts and preamps, assuming that the speaker, amplifier and line level preamp are "honest".
Because, it is harder for one to tell how does a source "sound" if the speaker is the limiting factor. For example, changing from Tripath amplifier to SingleEnded Tube amplifier through B&W Matrix 804 will not produce more significant difference in sound compared to changing from Tripath amplifier to SingleEnded Tube amplifier through Magnepan MG IIIa. I'm not implying that Magnepan is better than B&W. I am also not implying that Tripath amplifier is the same as SingleEnded Tube amplifier. These are the only particular types of brand and technology that are accidentally available for me to play with for the purpose of real life experience(s). One must be open minded and not limit oneself to only specific brands if the end goal is to have the "ultimate" system.
If one is using "best possible transparent" speaker and amplifier, then one could really appreciate different kinds of "sound" produced from various source components* limited by technology, design, build quality, and price.
Source components* = phono cart, phono pre, step up transformer, line level preamp