The stealth buff doesn’t change your movement speed at all; it’s just the illusion of moving slower due to the animation.
I actually find the situation pretty interesting, as it’s like moving through a city of phantoms on a populated server when people are just standing around stealthed if they’re not mounted. I can definitely put up with this for one week out of every four on the rare occasions where the buff is voted up.