How to Lock Action Bars to a Spec?

Trying to get Vengeance sorted out for dungeons, but when I move things around on the action bar it changes the Havoc action bars too for some reason. And if I go Havoc then back to Veng without touching anything it will remove abilities from the bar. Very strange. Is there a setting I’m missing or is it just scuffed?