There are a variety of reason, broken down into 2 categories: map and players.
The biggest issue is that the map is unbalanced. Generally, the center of the map is the key to who wins and loses, specifically the triangle of stonehearth, iceblood, and snowfall graveyards.
We’ll start with the farthest foward horde graveyard iceblood graveyard: if alliance assault IBGY, horde respawn 20 at a time 22 seconds away. Alliance will respawn 10 at a time 57 seconds away at stonehearth graveyard. Holding IBGY for the 4 minutes to “hard cap” so that it can be used for alliance to respawn at is extremely difficult in a battle of attrition with such a disparity in reinforcements. This means that it is very easy for horde to retake IBGY if alliance captures it, which generally means that if it is not being immediately threatened, they aren’t required to hold people back to defend it.
The farthest forward alliance graveyard is stonehearth graveyard. Immediately to the north of SHGY (so, behind it) is a choke point (Icewing bunker choke) that is extremely easy to defend against assault when defended from the south. There is no way around it and the choke point at its narrowest is about 2-3 times the width of the road. If alliance lose SHGY and have not captured any other graveyard south of that point, horde can easily block off IWB choke and prevent any non stealth alliance from getting south. Further, once horde has hard capped SHGY, the graveyard is immediately behind the choke whereas the alliance graveyard is about a minute away. So again, in this particular instance, horde can reinforce faster. Basically, if horde captures SHGY, any chance alliance has of winning the game is gone. They can stall the game out for a long time with the strong defensive positions of stormpike graveyard or the bridge to Dun Baldar (although generally not both) but they have no actual hope of winning at this point. The importance of not losing SHGY, and the ease with which its loss allows horde to stop all alliance from going south of IWB if it is lost, means that alliance can not lose SHGY before capturing a graveyard south of IWB choke. Which means it must be defended.
The “neutral” graveyard that starts off not controlled by either side is snowfall graveyard. Due to the relative positions of the starting caves, horde are able to get to SFGY and soft cap it long before alliance get anywhere close to it. That is only a minor benefit, but a benefit none the less. A bigger issue is that any alliance players fighting at SFGY who die respawn (for whatever reason) all the way back at the aid station in Dun Baldar, which means they have been completely taken out of the fight for 2 to 2 1/2 minutes, which is huge. Additionally, since alliance have to maintain some sort of defensive force to guard SHGY, they have a smaller force to attempt an assault of SFGY.
Alliance has several potential options at the start of a game: attempt to run the majority of their forces around incoming horde all the way down to IBGY and attempt to capture and hold it for the 4 minutes necessary for it to hard cap, run the majority of their forces to SFGY to attempt the same thing, sit at SHGY and “turtle”, or some combination of the 3, which splits forces up into smaller groups. For any of the offensively oriented strategies to be successful, alliance have to have solid coordination, but most importantly they have to win all of their fights by substantial margins since they are operating from a reinforcement disadvantage. Ultimately, horde has a single strategy that is essentially foolproof: immediately assault SHGY with everything they have and take it as quickly as they can. If a substantial portion of the alliance forces have gone on the offensive, they can take SHGY with ease, close off IWB choke and send a portion of their force to mop up any pockets of alliance south of IWB choke, who can’t be reinforced. They don’t have to “win” these fights south of the IWB choke, they just need to kill some alliance each fight to whittle them down, respawn close by and attack again, until eventually all alliance south of IWB choke are dead. If alliance instead choose the defensive option of defending SHGY, horde can simply throw their entire force at it over and over again trying to take it. Eventually they will get through and can soft cap it. They don’t need to hard cap it because as soon as it is soft capped, all alliance now respawn north of IWB choke a minute away. Horde don’t have to fear some small force running around them and capturing another graveyard because a small portion of their force can simply turn around and deal with it due to reinforcement advantages.
All of that means that the alliance team has to be substantially better than the horde team in terms of player skill, gear, class composition, and coordination to win.
All of that detailed above has had effects on the alliance players who queue for AV. Alliance pvp rankers generally aren’t going to queue because AV is only good honor if you are winning a decent amount of your matches. It is really good honor if you win. It is terrible honor if you lose. Alliance players who pvp mainly for enjoyment aren’t going to queue for a match where they do not have a realistic chance of winning. That’s simply not fun. What’s left are largely, though not entirely, people who queue for AV matches because they want the item rewards that reputation with stormpike can give. The average gear level of alliance players is generally not comparable to that of the horde players who queue for AV. In addition, there are some people who want the reputation gains but don’t want to actually sit through the frustration that is playing AV, so they find a cave somewhere and simply go AFK. The lower average gear level and the general demoralization of alliance players combine with the map disadvantages to make for an absolutely toxic situation where the alliance side essentially never wins. Alliance players who finish the rep grind with stormpike for the most part never come back to AV again.
EDIT: I’d also add that in the context of taking and holding SFGY, horde have what are called “interior lines” providing a substantial benefit. If horde has some people assaulting SHGY and some people at SFGY, they can move reinforcements back and forth between the two more quickly than alliance can, giving them local numerical superiority at the place they need it most quite quickly. The east facing ramp to to SFGY is about 4 or 5 bow shots from the south facing ramp leading up to SHGY. Alliance, assuming they want to disengage from horde and move around, have to move all the way north around Balinda’s bunker and then come south down to the ramp to SFGY. Combined with the fact that alliance assaulting SFGY respawn at DB, it makes it very difficult to take and hold SFGY.