It could, but only if you use a lottery system and forcibly change people. Then, those people will either accept the change and play, or get pissed off and quit. Either way, less imbalance.
If you leave it to the players, no, it will not fix anything. In fact, it’s more likely that there would be a ton of swapping to Horde.