Sanctum of Domination Lag Update

For the past several weeks, we’ve received and acted on reports of lag in the Sanctum of Domination raid, particularly during the Sylvanas encounter. We’ve been working hard to understand and resolve the issues, but we overlooked telling you that. We apologize for that oversight, and we will strive to do better in the future.

The most common complaint we received was that there was a burst of lag when Sylvanas is first engaged. Our server performance group dug into those reports, and we found the cause – there are a higher-than-usual number of calculations being done at the start of the fight to facilitate the complex mechanics of the encounter. We’re looking into ways we can potentially improve this, but we have to be careful to not change the design or mechanics of the encounter. While this is something that we think we can improve over time, there will still be a few seconds of server lag on pull for now.

Outside of that moment, our main challenge in diagnosing these lag reports came from the fact that lag was reported in almost every phase of the fight, and we were unable to reproduce that lag in our test cases. In addition to examining our own internal testing, when we utilized our performance analysis tools on actual live raids that reported they were experiencing lag, the server always appeared to be functioning smoothly. This guided our focus elsewhere.

What we eventually discovered is that the Sylvanas fight in particular is very messaging-heavy, the mechanics involved require sending many frequent updates to the player’s game clients, which in itself isn’t a problem. However, we discovered that an unrelated change to how we organize and distribute server processes across our server hardware, combined with accumulated changes over time to how our hardware server infrastructure is organized, resulted in our server processes being more concentrated across our hardware.

We believe all of these factors combined to make the messaging overload happen more commonly and cause lag.

With our most recent weekly maintenance, we’ve deployed several changes to how we distribute server processes across our infrastructure, which should alleviate this potential source of lag. Please let us know about your experiences in Sanctum of Domination going forward, especially if you can capture video of any new misbehavior.

Thank you very much for your valuable feedback!

33 Likes

thank you.

edit:
wouldve liked to have known that you guys were aware of the issues earlier and working on it however is my one complaint.

5 Likes

This is the kind of communication we want! Thanks for the update!

8 Likes

What about insane outdoor world lag that has been happening for years and still not fixed?

7 Likes

Thanks for the update, but this also seems like the extent of the problems are still not fully understood by the team there. Almost every fight experiences lag and delay on peak nights when pulling bosses or going into specific phases (hello Taragrue dark phase). It’s a recurring and persistent issue that is far more prevalent on the first raid night of the week.

To my semi-ignorant observations and leveraging the small sample sizes/discussion I have had with other players on my server, this seems like it’s a mix of load at the start of encounters (as indicated above) as well as not scaling up/having enough resources to meet peak demand. This is also true when we see a large number of players on a single shard, everything just chokes…

1 Like

THANK YOU! This is the sort of communication I’m genuinely happy to see.

6 Likes

Ah, wish it had attention 2 weeks ago. Sylvanas would’ve been dead instead of wasting hours to dying in intermission due to lagging for 5 seconds at a time.

1 Like

Thank you for the feedback.

Could we please take a look at Oceanic forums as there is a thread regarding just general server stable/connection issues in raids. This is magnified obviously on Sylvanas, but it is just really bad on several bosses, especially on Reset night week where the server seems to crumble.

We have tried with disabled addons (Delete/WTF/Interface), but even bare stock, the fps crumbles.

On our Sunday (Sat USA), time, this seems to be considerably lower, to me it feels like reset night when everyoine starts the first raid, during peak time Frostmourne server just struggles.

Please could we have someone look a bit deeper into Oceanic servers as well.

4 Likes

Thanks for the update!

Great, now explain why it lags in korthia, why it didn’t lag for every guild, and why it doesn’t lag for every player. Why there are still desyncs on painsmith too for that matter.

3 Likes

@ Elocin i thought it was personal / pc lag thooo?

2 Likes

This is how you communicate. Even if you dont know how to fix something yet, acknowledge it & tell us youre working on it. Please do this more.

2 Likes

You sir need a pay rise. Keep up the good work, communication is what keeps customers happy even if it’s bad news, at least we know what is happening and how things are being worked on, fixed and not ignored.

Thank you.

1 Like

I am very impressed with the attitude shown in this post. The use of things like

gives me hope that the teams are starting to get a grip on one of the things that really annoys so many people - the lack of response to problems and the apparent disinterest in how players feel about apparently being ignored.

Then to provide a concise technical explanation (even if I am completely unable to understand it) shows that you want us to know what the actual problems are. Rather than just, ‘its a technical problem, we’re working on it’ it gives the feeling that you are talking to people rather than numbers.

Well done, its a good step forward.

3 Likes

I haven’t yet had the pleasure of pulling Sylvanas, but I nevertheless applaud the humble transparency!

Hey Kaivax!

Thanks so much for updating us! I think it’s really nice if you let us know that you’re aware of a problem, even if you haven’t fixed it or aren’t fixing it. Better than radio silence. Thanks!

YES! Acknowledge this too finally please!

YES! please!
the world lag has been a serious problem for me this expansion – 2-5fps is really not playable, even when i’m at my most patient. worse yet, it comes and goes, so while i may be fine for awhile, headed out on a quest, suddenly it will drop to that low fps, be that way for minutes at a time, and then suddenly “clear up” and everything will be fine again. this does not seem to be anything on my end, because i’ve done everything recommended by blizz in other threads to mitigate the stress and load on my end.

Perfect kind of update/communication I would love to see more of. Thanks

lol they are deleting discordant posts on this thread , i bet this one is gonna be deleted too