So the only thing stealing your SR is the enemy swapping and out playing you?
Nothing stopping you from doing the same.