Progress!
Let's do some math.
We know that damping factor is a ratio, specifically, it's the load impedance divided by the output impedance of the amplifier. For now, we'll ignore how amplifiers attain low output impedance just for the sake of simplicity.
So, let's say we have an amplifier with a damping factor of 2000 and an impedance of 8 ohms.
DF = Load Impedance / Output Impedance
2000 = 8 / Output Impedance
Output Impedance = .004
All is well in the world, right? We have our staggeringly high damping factor but we've ignored something important.
We forgot to include the series resistance of the speaker wire between the amp and speakers. Let's redo the calculation now.
Again, we know that DF = Load Impedance / Output Impedance, but now we are going to add in the effects of the speaker wire into our equation to figure out what the "real" or "effective" damping factor is of the real world situation. For simplicity sake, we'll err on the side of caution and say that it's very low resistance wire and only presents a .1 ohm load.
DF = 8 ohm / (.004 ohm + .100 ohm)
DF= 8 ohm / .104
DF= 76.9
The advertised damping factor of 2000 is now only 77 at the terminals of the speakers.
Now, let's look at what is going on at the terminals of the woofer in a speaker system with a passive crossover.
It's not unusual to have a series resistance of 1 ohm in a passive crossover, but just for a best case scenario, let's say that it's only .25 ohm.
DF = 8 / (.004 + .100 + .250)
DF = 8 / (.354)
DF = 22.6 at the woofer.
Now, let's say the damping factor of the amp is 20000! Surely that must make a huge difference being an order of magnitude higher than 2000!
Unfortunately, the reality is:
DF = 8 / (.0004 + .25 + .1)
DF = 22.8
It's essentially the same.
Now, let's say that the amplifier in the same scenario is only at 50.
DF = 8 / Output impedance
50 = 8 / Output Impedance
Output Impedance = .16
Now that we have the output impedance of the amp with a DF of 50, let's subsitute into our scenario to see the DF at the woofer:
DF = 8 / (.16 + 1 + .25)
DF = 8 / (.51)
DF = 15.7
Do you see how most people just don't find DF that important? In the real world, the difference between a low DF amp and a ridiculously high DF amp is essentially moot at the woofer.