Spirits of Auchindoun Towers Layering Issues

I’ve been trying to complete the Spirits of Auchindoun tower quest for a couple of days now. Every time it gets within a couple minutes of capture time, I get pushed to another layer. If I’m in a raid group and someone gets invited it is switching the whole groups layer to another layer. Please fix this. I want to earn some honor. There was a time I could complete this quest with more than one character at a time. Now I can’t even do it on one. Anytime I log or disconnect, I get thrown into another layer. In my opinion, if I’m on a layer there should be nothing that should swap me to another layer other than being invited to a group by someone on that layer, and if someone joins or leaves my group I should retain my current layer. Also if I’m standing on a tower point for 2 hours, it’s absolutely unacceptable to push me another layer 2 minutes before the tower is capturable. This is so infuriating. Please please please let me do this quest and quit messing with the layering and making it worse.