Ability/Loot Lag on Oceanic Servers Despite Normal Latency

Edit: We've requested the issue be investigated further. Please check out the newest post here.

Hey all,

I've been keeping my eye on the situation here and with the investigation while I was away, and wanted to follow up with you all on what we’ve found. Because these problems tend to be clustered around certain times, it took us some time to get all the testing done.

We took a look over a lot of these winMTRs and had our Quality Assurance team try to reproduce the issue in real-time on your servers. We haven’t seen any server instability which would be causing this on our end, and when we went in to test, we didn’t find the same issue you all have been reporting, even when we actively joined the same shard some of you were on while reporting issues.

Based on the more recent MTRs that have been posted, we're not seeing much of a pattern in routing outside of some local connectivity issues. We're seeing some packet loss and increases in latency along the route in some examples, but nothing that is consistent between each report. I'm disappointed that I don't have better news here guys, but we just weren't able to reproduce this. It may be best to try troubleshooting these individually.

-Lantrasau
09/03/2018 07:48 PMPosted by Lantrasau
Hey all,

I've been keeping my eye on the situation here and with the investigation while I was away, and wanted to follow up with you all on what we’ve found. Because these problems tend to be clustered around certain times, it took us some time to get all the testing done.

We took a look over a lot of these winMTRs and had our Quality Assurance team try to reproduce the issue in real-time on your servers. We haven’t seen any server instability which would be causing this on our end, and when we went in to test, we didn’t find the same issue you all have been reporting, even when we actively joined the same shard some of you were on while reporting issues.

Based on the more recent MTRs that have been posted, we're not seeing much of a pattern in routing outside of some local connectivity issues. We're seeing some packet loss and increases in latency along the route in some examples, but nothing that is consistent between each report. I'm disappointed that I don't have better news here guys, but we just weren't able to reproduce this. It may be best to try troubleshooting these individually.

-Lantrasau


That's unfortunate as a lot of us are still experiencing the lag at certain times and certain places.

I was on last night and the lag in Borulas was not as bad as it's been which I'm guessing uis because less people are playing due to the lag, but it still took me a good 5 seconds to sell stuff to a vendor.

I was also doing world quests in Voldun and looting of enemies was taking substantial amounts of time :(

What's the next step? :)
1pm here where I am in Australia.
All of a sudden crippling lag to game play - home and world MS remain fine (25ms).
Happens every damn day. MOST frequently in peak play hours for Australia but not all the time.. as I'm experiencing it right now.

I am in general chat in Drustvar and the chat is *SWAMPED* with people complaining of the exact same lag experience. It is not an individual problem. Every single Oce player on my server, with warmode off, is experiencing crippling lag at exactly the same time.
Oh look the run of the mill Blizzard response basically saying it's not on our end guiz it's on your end -.-
Lantrasau,

Thank you for the reply, from what ive seen and experienced personally from the lag is that it gets worse depending on time and if theres an elite quest in the zone.

Back in legion if there was an elite world quest you would join a group, zone in to the leaders server complete the quest, then leave group which would put you back to your own server.

In BFA when you join a group and complete the quest you are not returned to your own sever. Couple this with the fact some elite groups stay formed for hours upon hours bringing hundreds if not thousands of people to the sever.

This isssue is caused by to many people in the area on one sever.

This is my opinion, and I personally believe that fixing that issue so that you return to your own server like it was in legion will fix this problem.

Hope this helps.
09/03/2018 08:00 PMPosted by Helruna
I am in general chat in Drustvar and the chat is *SWAMPED* with people complaining of the exact same lag experience. It is not an individual problem. Every single Oce player on my server, with warmode off, is experiencing crippling lag at exactly the same time.

This is what bothers me about the 'troubleshoot individually' response we've just received. Don't get me wrong, I appreciate the update but... the number of players actually complaining on the forums is miniscule compared to the number experiencing it. It is simply not feasible to troubleshoot individually on that scale.
09/03/2018 07:48 PMPosted by Lantrasau
Hey all,

I've been keeping my eye on the situation here and with the investigation while I was away, and wanted to follow up with you all on what we’ve found. Because these problems tend to be clustered around certain times, it took us some time to get all the testing done.

We took a look over a lot of these winMTRs and had our Quality Assurance team try to reproduce the issue in real-time on your servers. We haven’t seen any server instability which would be causing this on our end, and when we went in to test, we didn’t find the same issue you all have been reporting, even when we actively joined the same shard some of you were on while reporting issues.

Based on the more recent MTRs that have been posted, we're not seeing much of a pattern in routing outside of some local connectivity issues. We're seeing some packet loss and increases in latency along the route in some examples, but nothing that is consistent between each report. I'm disappointed that I don't have better news here guys, but we just weren't able to reproduce this. It may be best to try troubleshooting these individually.

-Lantrasau

I guess what you are saying is that the people are wrong and that the issues are all on their end.
During peak times this is all people on Frostmourne are talking about.
Your comments are really a big slap in the face but worded in a nice way.
09/03/2018 07:48 PMPosted by Lantrasau
Hey all,

I've been keeping my eye on the situation here and with the investigation while I was away, and wanted to follow up with you all on what we’ve found. Because these problems tend to be clustered around certain times, it took us some time to get all the testing done.

We took a look over a lot of these winMTRs and had our Quality Assurance team try to reproduce the issue in real-time on your servers. We haven’t seen any server instability which would be causing this on our end, and when we went in to test, we didn’t find the same issue you all have been reporting, even when we actively joined the same shard some of you were on while reporting issues.

Based on the more recent MTRs that have been posted, we're not seeing much of a pattern in routing outside of some local connectivity issues. We're seeing some packet loss and increases in latency along the route in some examples, but nothing that is consistent between each report. I'm disappointed that I don't have better news here guys, but we just weren't able to reproduce this. It may be best to try troubleshooting these individually.

-Lantrasau


I bet you tested in your time zone and not actually in Australia where the actual servers are housed.

You CANNOT blame our connections for multiple isps (numerous WINMTR results) on this issue at all, its a capacity issue that is server side.

It has to be tested especially in our peak which is in ~6 hours from this post.

During peak we enjoy mobs moonwalking, going from 100-0 health, constant up to 5 second loot lag and other great benefits which greatly imapct gameplay.

This does not happen in non BFA areas so the issue is narrowed down to capacity of where we are being placed as we play.
09/03/2018 07:48 PMPosted by Lantrasau
I'm disappointed that I don't have better news here guys, but we just weren't able to reproduce this. It may be best to try troubleshooting these individually.

-Lantrasau


I'm sorry, but there's simply no way that's possible. Every night, chat is filled with people saying things like "I see the lag still hasn't been fixed" and "Anyone else taking 5 seconds to loot?"

What are the chances that all of those people, as well as all of the people who have posted on this thread, have individual issues which amount to the same symptoms?

I worked as a retail manager for over a decade, so I'm aware of how difficult customer service can be and I choose my words very carefully.

As a paying customer, this isn't good enough. Go back and try again.
09/03/2018 08:09 PMPosted by Stitchiz
Lantrasau,

Thank you for the reply, from what ive seen and experienced personally from the lag is that it gets worse depending on time and if theres an elite quest in the zone.

Back in legion if there was an elite world quest you would join a group, zone in to the leaders server complete the quest, then leave group which would put you back to your own server.

In BFA when you join a group and complete the quest you are not returned to your own sever. Couple this with the fact some elite groups stay formed for hours upon hours bringing hundreds if not thousands of people to the sever.

This isssue is caused by to many people in the area on one sever.

This is my opinion, and I personally believe that fixing that issue so that you return to your own server like it was in legion will fix this problem.

Hope this helps.


+1

The issue is phasing into a different realm/shard for a worldquest and then the game doesn't phase you back when you are done. All of my input delay issues in the world all happen after attempting to do a WQ in a highly populated area and/or join a custom group for an elite WQ. Home/World latency shows 80/80 but the 1-1.5 sec input delay disagrees with that.
Why are you all thanking this douche? I bet he hasn't even looked into it. Typical response blaming everyone else
I appreciate the response, even if it’s not great news. Thanks.

Hope it’s not going in the forgotten pile, though.

Something’s not quite right...
Hey guys, while I can't promise anything at this time, we've once again pushed this to the quality assurance team. It seemed really weird to us, and we agree that the behavior doesn't add up for individual user or ISP issues, so we're trying some different reproduction steps. I know we've been asking for a lot of tests, but we'd like to have as much information as possible for users having issues after the time of this post. The next time you see the issue, please post another winMTR so we can have more examples of the issue being ongoing.

I know this has been a really frustrating process so far, and apologize for all the confusion on the way. We hope to have some more useful updates going forward.

-Lantrasau
Lantrasau

One question

How many of your testers were in the Oceanic region testing from Oceanic cities like Sydney?

Or were they all in the US? If they were...I know why you cant replicate it.

I'm disappointed that I don't have better news here guys, but we just weren't able to reproduce this. It may be best to try troubleshooting these individually.


I understand your request, but your statement of "we cant replicate this" flies directly in the face of this issue being reported across every single Oceanic server, every single city with thousands of players.
09/03/2018 08:59 PMPosted by Lantrasau
Hey guys, while I can't promise anything at this time, we've once again pushed this to the quality assurance team. It seemed really weird to us, and we agree that the behavior doesn't add up for individual user or ISP issues, so we're trying some different reproduction steps. I know we've been asking for a lot of tests, but we'd like to have as much information as possible for users having issues after the time of this post. The next time you see the issue, please post another winMTR so we can have more examples of the issue being ongoing.

I know this has been a really frustrating process so far, and apologize for all the confusion on the way. We hope to have some more useful updates going forward.

-Lantrasau


Lantrasau, can you explain why we don't lag in old world content. Lag only happens in BFA zones and not in dungeons?

If it was our connections, we would lag everywhere correct? Just with that little bit of information you can tell it's not on our end.
09/03/2018 07:48 PMPosted by Lantrasau
Hey all,

I've been keeping my eye on the situation here and with the investigation while I was away, and wanted to follow up with you all on what we’ve found. Because these problems tend to be clustered around certain times, it took us some time to get all the testing done.

We took a look over a lot of these winMTRs and had our Quality Assurance team try to reproduce the issue in real-time on your servers. We haven’t seen any server instability which would be causing this on our end, and when we went in to test, we didn’t find the same issue you all have been reporting, even when we actively joined the same shard some of you were on while reporting issues.

Based on the more recent MTRs that have been posted, we're not seeing much of a pattern in routing outside of some local connectivity issues. We're seeing some packet loss and increases in latency along the route in some examples, but nothing that is consistent between each report. I'm disappointed that I don't have better news here guys, but we just weren't able to reproduce this. It may be best to try troubleshooting these individually.

-Lantrasau


Lantrasau,
The issue is intermittent and the lag tends to come in spikes. There was one major lag spike last night whilst questing in Drustvar that resulted in the entire General Chat complaining! So the issue is definitely not isolated to local connections. Every evening since launch I have experienced this lag at some stage between 5:00pm & 10:00pm weekdays server time/AEST.

Perhaps you may need to increase your sampling time/size Window in order to physically witness our grievances if you are not seeing it?
I would have thought a company as big as Blizzard would have analytical logging and information to refer to rather than physically witnessing our complaints though. What is the best method for trouble shooting these individually?
For issues experienced by hundreds, if not thousands of Oceanic players, it has to be something on your end. It just doesn't add up.

This weeks old issue has unfortunately ruined a lot of game time since release making it one of the worst expansion launches since vanilla and I've been through them all. The expansion is great, but this issue has soured it.

It needed priority escalation and a resolution 2 weeks ago.
09/03/2018 08:59 PMPosted by Lantrasau
Hey guys, while I can't promise anything at this time, we've once again pushed this to the quality assurance team. It seemed really weird to us, and we agree that the behavior doesn't add up for individual user or ISP issues, so we're trying some different reproduction steps. I know we've been asking for a lot of tests, but we'd like to have as much information as possible for users having issues after the time of this post. The next time you see the issue, please post another winMTR so we can have more examples of the issue being ongoing.

I know this has been a really frustrating process so far, and apologize for all the confusion on the way. We hope to have some more useful updates going forward.

-Lantrasau


This was an appropriate and professional response. As customers, we just want an acknowledgement that our concerns are being heard and taken seriously, so thank you for doing that, Lantrasau
I know we've been asking for a lot of tests, but we'd like to have as much information as possible for users having issues after the time of this post.


How many of your QA staff and testers are in Ocean cities? If there arent any, thats the next step. You need LOCAL testing done.

Not long distance.
It seemed really weird to us, and we agree that the behavior doesn't add up for individual user or ISP issues, so we're trying some different reproduction steps.
inb4 they forgot about timezones /nervouslaughter

For real though, as others have pointed out - the issue occurs in spikes, and any given shard can be experiencing it one second and then be fine the next, then right back at it at full throttle for hours. You're going to need to spend quite a bit of time hanging around during oceanic peak times to notice it. And, y'know, actually try doing things. I certainly hope QA already were, but at this point I'm not going to assume anything.

I highly recommend Zuldazar and Tiragarde for testing. While it can go off in any BFA zone, these two are the worst offenders by far. Probably due to the whole having-capital-cities-in-them factor.