The deserter debuff is there to discourage folks from abandoning a group immediately after entering a dungeon, and unfortunately, because there are sometimes individuals who would, usually through bad behavior, encourage the group to use the vote to kick system to help avoid that debuff, it is still a factor in those situations.