Australia/New Zealand - High Latency or Connecting to Wrong Servers since 2/Jul/2019

i am the same isp/location/region as Giantegg. Issue appears resolved for me. All games tonight (about 10) matched me to SYD2 and pings were stable 40’ish. No lag spikes and playing was smooth in competitive. I’m aasuming that we are on the same cable network in Geelong Victoria Australia connecting to Blizz servers via iinet.

Region: Americas
ISP: iinet
Timezone: GMT +10 (Geelong)

Region: Americas
ISP: iinet
Timezone: GMT +10 (Geelong)

Just run another WinMTR test on AI create and no pings over 40ms for me. Hope this helps the others. I’m correctly matching region servers 20+/0 and no massive lag spikes at any stage.

|------------------------------------------------------------------------------------------|
|                                      WinMTR statistics                                   |
|                       Host              -   %  | Sent | Recv | Best | Avrg | Wrst | Last |
|------------------------------------------------|------|------|------|------|------|------|
|                             192.168.0.1 -    0 |  569 |  569 |    0 |    0 |    5 |    0 |
|                              172.22.0.1 -    0 |  569 |  569 |    2 |    6 |   17 |    6 |
|            be21-450.cr2.mel10.on.ii.net -    0 |  569 |  569 |    4 |    7 |   24 |    7 |
|                be51.cr2.mel11.on.ii.net -    0 |  569 |  569 |   14 |   17 |   36 |   18 |
|                 be55.cr3.syd2.on.ii.net -    0 |  569 |  569 |   13 |   17 |   30 |   18 |
|                 ae15.cr1.syd4.on.ii.net -    0 |  569 |  569 |   13 |   17 |   34 |   17 |
|                  ae3.cr2.syd4.on.ii.net -    0 |  569 |  569 |   16 |   17 |   41 |   17 |
|                   57976.syd.equinix.com -    0 |  569 |  569 |   14 |   21 |   84 |   17 |
|        et-0-0-49-br02-eqsy4.as57976.net -    0 |  569 |  569 |   15 |   23 |  159 |   17 |
|         et-0-0-1-pe01-eqsy4.as57976.net -    0 |  569 |  569 |   16 |   20 |   82 |   18 |
|                          137.221.66.133 -    0 |  569 |  569 |   16 |   18 |   36 |   18 |
|                            37.244.40.29 -    0 |  569 |  569 |   16 |   17 |   39 |   18 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |  114 |    0 |    0 |    0 |    0 |    0 |
|________________________________________________|______|______|______|______|______|______|
   WinMTR v0.92 GPL V2 by Appnor MSP - Fully Managed Hosting & Cloud Provider
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Edit: We are requesting some new information on this issue. For posterity, the old post is being left as it was. However, please see this post for more information on what we need to investigate the problem in more depth.

Hey all,

Thank you all for the continued reports. These do help us greatly in identifying the source of issues like these.

After a thorough investigation, it looks like some ISPs are encountering what’s called “Packet Loss” during peak hours when there are large numbers of players logged in. To reduce stress on their servers (and subsequently resolve the packet loss) ISPs are rerouting their customers through different nodes which results in being placed on a different game server.

This is, in some ways, a good thing for them to do since it results in better network stability for their users. The only downside is the increase in latency caused by the routing change. So it definitely seems like the lesser of two evils.

Since the problem is happening outside of our peer network our ability to assist is rather limited. The best solution in this case would be to contact the affected ISP directly to report the issue. A Tier 3 agent may need to be requested since they are generally the ones who work with their network engineers for routing issues.

These types of reports are best submitted by customers who pay the ISP to provide service.

Thanks for the info. Still getting the problem unfortunately (ISP is Telstra)

Any suggestions on what to say to the ISP when logging the ticket? I’m almost certain the response will be “not our problem, please contact technical support for the game” :frowning:

  • Edit - just tried the Tech Support Chat now service, and they want me to contact Telstra Platinum, which is a fee based support service.

I’m sorry, what?
All ISP’s in Australia and NZ? All at once?
Nice try.
I guess you really, really just don’t care do you.
“Packet loss”. Please. Who do you think you’re fooling here but yourselves?

2 Likes

OMFG this is hilarious. This problem has been reported for ages and I think you’re right that they just don’t care.

I actually asked a GM about this and he told me that putting players in different geographic region servers is their intention. I posted the chat log at the below thread. The GM blames the amount of players, I think this also allows them to rig the comp games and ensure you stay exactly where they want you to stay. Think of this game as competitive is a joke, it’s no wonder everyone treats comp like poo, Blizzard don’t even care about the integrity of the competition so why would any of the players.

I mean a simple ping test on their end and matches us to a low ping server would fix all this but it doesn’t seem like Blizzard would even entertain that thought, they’ve been blaming ISP’s since this started and think we’re all stupid enough to believe that to be a valid reason. No other game does this and it’s super easy to code around.

I noticed this at the start of the year and have made, read and contributed multiple posts on the issue.

I basically gave up and now just play comp for the giggles, Blizz have kind of forced me to be one of those players I hate, I don’t care too much about teams now I just trying to get some enjoyment out of it when I play even if that means playing the 5th dps and having no chance at winning.

3 Likes

Kaldraydis, you probably can’t comment on this… but would it be possible to be able to implement a sever selector in the Overwatch client?

That way OCE players could force a connection to the Sydney servers, rather than letting the client choose for us.

In the US, ISPs in one region will sometimes share a common peer. When the peer has problems, 10-15+ ISPs can also have a problem. (Note: I am not talking about a local peer, I am talking about something larger scale connecting Australia and New Zealand traffic to Blizzard servers.) I don’t have access to the info Blizz used to make the above statement, but my guess would be that something similar is happening for the OCE players.

Edit: Removed link that discussed overburdened internet infrastructure, but was focused on another ISP not discussed in the thread.


That’s how it works already. It looks at the last hop where your ping came from. So let’s say your ISP routed you into the Netherlands, it would connect you to Europe because the last hop is close to the European server.


A good suggestion in my opinion – I don’t know how it would affect things, but would be interesting to see in action. They’ll probably tell you to suggest it in General, since devs peruse that forum.

2 Likes

Hey Arkfox congrats on your insanely fast cable internet that only ever struggles with Overwatch! Sadly however the problem is the ISP so time to switch it up :frowning:

Kaldraydis you are a idiot that has no idea what you are talking about. As someone with 10 years of Network Operations experience I can tell you right now, nothing you have said about how a network works is correct. But on your advice, i did contact my ISP’s Teir 3 agent (which is a American term, we don’t have teir 3 agents here), and the poor dude I spoke to almost wet himself with laughter with your excuse. Blizzard support has just become the laughing stock of the Aussie ISP tech support scene. Keep up the great work!

As for Nicole, “A survey found 36 per cent of respondents [in Australia] have reported experiencing some connection or streaming issues” because they are not hosted locally. Local load balancing (the common peer you talk about) will only add a few ms to any route, and any problem within that peer will be detected instantly by Network Operations and fixed as it will affect all traffic.

It’s cute that both Nicole and Kaldraydis failed to mention the glaring issue…This started after the last patch. You don’t need to have pretty Green or Blue text to realize that YOU changed something with your internal load sharing with the last patch that caused this problem.

But keep blaming our ISP’s, it just shows how clueless your support ‘experts’ are.

3 Likes

Come on Blizzard.

I hate to play this card but as someone with over 6 years of experience in the IT industry looking after extremely complex networks and systems, I need to tell you that you shouldn’t try to explain how this works by reading news articles and pasting them into forum posts. It’s erroneous information and doesn’t really help the cause at all.
Let me lay some facts out for you.

The chances of EVERY SINGLE ISP in NZ/AUS having problems at once are EXTREMELY slim. You know that. Everyone here knows that.

When im matched into an OCE server:

My wife and I play together on the same Overwatch games. She sits behind me. My ping will be 40 and hers will be 80 (according to the in-game net-graph). It could be me 80 and hers 40, it tends to swap. Sometimes both 80 sometimes both 40.

  1. During this time, I have constant pings running from my router, both of our PC’s and my home server. They all sit between 27 and 30ms and are all going to the exact IP that the in-game net-graph is showing. The pings NEVER CHANGE REGARDLESS OF WHAT THE GAME REPORTS

Now, I ask you, how can this be a “problem with the ISP” ?

When I’m NOT matched into an OCE server
I am still continually running pings AND Traceroutes/MTR to the SYD1 and SYD2 servers that I would prefer to be in.

GUESS WHAT, THE PATH THE TRAFFIC TAKES AND LATENCY RTT NEVER CHANGE

Now, I ask you, how can this be a “problem with the ISP” ?

Blue team, please respond.

(edit: I was a bit toxic here and realized the error of my ways, so tidied this up a little, sorry if I offended anyone).

All righty, I’m going to make two separate posts in a row on this, because this particular post is aimed at a few particular posters and it’s not relevant to the issue.

I completely understand that this has been going on for a while, now, and as a result it’s starting to get understandably irritating to deal with. There have also been some communications made with incomplete data, which I will be going back and correcting shortly. However, we’re all trying to work together to get this problem solved. Disagreements are entirely allowed, and we can try to address them. However, we’re going to treat toxicity and personal attacks in this thread like we do in game going forward.

That is to say that unless you’re communicating in a level headed/professional manner and trying to work with us, your posts may be hidden and if the behavior does not improve you may receive forum suspensions. I’m not going to do this lightly. Further, it’s not a tool I will use if I simply disagree with what you say, but will resort to this if the name calling and other unnecessary comments do not cease.

While we would appreciate more data, and the more data we get, the easier we can investigate this, you are not obligated to post here, and we’re not going to sacrifice keeping this investigation civil and constructive. If you don’t want to be constructive, this is not the place for you.

I appreciate your cooperation.

1 Like

Now, on to what’s going on here.

As far as we can tell right now, this is entirely unrelated to the recent patch. From the tests I’ve seen so far on this issue, there are a mix of problems. Some players are having individual packet loss issues which will not be corrected when this ISP level issue is handled. However, there have also been some signs of packet loss and high latency at a major backbone provider which funnels your connections toward our Sydney server. For Australia/New Zealand players, a lot of you will probably be impacted by this. We’re not closing this investigation just yet, but unfortunately we haven’t been getting enough of the information we actually need to get our techs working on this problem.

Without enough tests to convincingly say to them “This is the problem” we can’t even attempt to reach out to peering because they’ll look at our email and reply “Okay where’s the data that makes me care about a gaming company’s experience?”

So, let’s refocus here and try to get that data. A lot of the tests in this thread have shown that your connections to the wrong servers have been fine, and the tests in the other thread were mostly too short to get usable data for. We need to redo some of these tests to give this another look.

For those of you who are still having this problem, please create a WinMTR test during a time when you’re having latency problems on Sydney servers, or if you are getting routed to the wrong server outside of the Practice Range. (The Practice Range will often put you on the wrong server group anyway, so it’s pointless to test here.)

For the “Host” blank, we need you to use this IP address specifically so we can see what’s going on with your test:

37.244.40.1

You want to restart the test every 10-15 minutes until you catch one of these problems. Once you’ve caught one red handed, run the test for about 5 more minutes. This will let us look for problems between your house and our servers which may cause this.

If you have problems with posting it due to a link error, go ahead and copy paste this into your next post, and replace “WinMTR goes here” with your test.

~~~~
WinMTR goes here
~~~~

Again, we appreciate your patience and cooperation throughout all this. This doesn’t mean you should stop notifying the ISPs about this if you do find problems in your WinMTR, but it does mean we’re willing to give this another check to see if there’s maybe something we can do.

1 Like

I can tell you what the problem is. Syd2 is an Equinix server (I think) aussies used to play on PSE2 which was an AWS server.
Maybe Equinix just isn’t as good at handling gaming traffic as AWS or maybe ISPs don’t have good peering with Equinix.

Why don’t you guys open up a few servers on PSE2/AWS and see if it improves?

I don’t think that’s the case. SYD = Sydney, obviously. PSE = Pacific South East. These servers are up in South East Asia somewhere, I suspect Singapore but I’m really not sure. Adding more servers in the PSE area would not help this problem I don’t think – unless I’m completely missing your point.

|------------------------------------------------------------------------------------------|
|                                      WinMTR statistics                                   |
|                       Host              -   %  | Sent | Recv | Best | Avrg | Wrst | Last |
|------------------------------------------------|------|------|------|------|------|------|
|                 192-168-1-1.tpgi.com.au -    3 |  635 |  622 |    1 |    1 |    6 |    1 |
|                 10-20-25-79.tpgi.com.au -    1 |  674 |  671 |   13 |   26 |  468 |   13 |
|       203-219-166-66.static.tpgi.com.au -    1 |  667 |  662 |   13 |   17 |  226 |   14 |
|      203-219-107-102.static.tpgi.com.au -    1 |  674 |  671 |   13 |   31 |  676 |   14 |
|      bri-pow-que-wgw1-be-10.tpgi.com.au -    1 |  678 |  676 |   14 |   26 |  537 |   14 |
|           be300.bqueebrdr11.aapt.net.au -    1 |  674 |  671 |   14 |   28 |  528 |   15 |
|Bundle-Ether16.cha-edge902.brisbane.telstra.net -    1 |  678 |  676 |   14 |   28 |  530 |   15 |
|bundle-ether7.cha-core4.brisbane.telstra.net -    1 |  674 |  671 |   14 |   33 |  465 |   15 |
|bundle-ether20.ken-core10.sydney.telstra.net -    1 |  682 |  681 |   28 |   40 |  539 |   30 |
|bundle-ether1.oxf-gw10.sydney.telstra.net -    1 |  674 |  671 |   28 |   42 |  447 |   29 |
|bundle-ether1.sydo-core04.sydney.reach.com -    1 |  674 |  671 |   28 |   43 |  488 |   31 |
|           i-91.sydo10.telstraglobal.net -    1 |  682 |  681 |   28 |   39 |  529 |   29 |
|               unknown.telstraglobal.net -    1 |  678 |  676 |   29 |   44 |  560 |   30 |
|                           137.221.85.35 -    1 |  674 |  671 |   29 |   51 |  465 |   29 |
|         et-0-0-1-pe01-eqsy4.as57976.net -    1 |  678 |  676 |   28 |   48 |  441 |   29 |
|                             37.244.40.1 -    1 |  674 |  671 |   29 |   47 |  446 |   30 |
|________________________________________________|______|______|______|______|______|______|
   WinMTR v0.92 GPL V2 by Appnor MSP - Fully Managed Hosting & Cloud Provider

Just played a game and got the brazil server. I’m in Australia. What did you guys do? WTH?!?

You have packet loss to your home router – you need to fix that.

Hey all, My ping has come down from the original 170-180ms mark, however, it seems to fluctuate from its normal 16-18ms all the way up to 90ms. This seems intermittent and doesn’t have a pattern from what I can see. I have lodged a ticket with iiNet in the hopes they can find something. Below are the WinMTR results for tonight.

|------------------------------------------------------------------------------------------|
|                                      WinMTR statistics                                   |
|                       Host              -   %  | Sent | Recv | Best | Avrg | Wrst | Last |
|------------------------------------------------|------|------|------|------|------|------|
|                         router.asus.com -    0 |  679 |  679 |    0 |    0 |    5 |    0 |
|                          150.101.42.200 -    0 |  679 |  679 |    1 |    1 |    9 |    1 |
|                 ae10.cr1.cbr2.on.ii.net -    0 |  679 |  679 |    1 |    2 |   38 |    1 |
|                 be12.cr3.syd7.on.ii.net -    0 |  679 |  679 |    5 |    5 |   13 |    6 |
|                be-50.cr2.syd7.on.ii.net -    0 |  679 |  679 |    5 |    5 |   15 |    6 |
|                  ae6.cr2.syd4.on.ii.net -    0 |  679 |  679 |    5 |    7 |   90 |    6 |
|                   57976.syd.equinix.com -    0 |  679 |  679 |    6 |    9 |  117 |    6 |
|        et-0-0-49-br01-eqsy4.as57976.net -    0 |  679 |  679 |    6 |   12 |  142 |    6 |
|         et-0-0-0-pe02-eqsy4.as57976.net -    0 |  679 |  679 |    5 |    8 |   84 |   10 |
|                          137.221.66.135 -    0 |  679 |  679 |    6 |    6 |   15 |    7 |
|                             37.244.40.1 -    0 |  679 |  679 |    6 |    6 |   17 |    7 |
|________________________________________________|______|______|______|______|______|______|
   WinMTR v0.92 GPL V2 by Appnor MSP - Fully Managed Hosting & Cloud Provider

I’m really confused why it’s implemented in this way and I feel the point I was making is somewhat confused.

A few packets can be sent to a player and if that takes .5 seconds (or whatever the threshold may be) to return because I’m connecting to some overseas server then just don’t put me in a game. It’s pointless playing the game with pings that high, no one has fun, often I can’t even communicate with teammates because of a language barrier and it seriously degrades the competitive nature of the game.

This can be achieved in a client as simple as a web browser why can blizzard not implement this simple check? rather than just running with the ping to whatever hop was last used which seems like a pretty poor implementation and doesn’t always give any true representation of the ping the player might experience. Services should be predictable and going of the last hops ping is just not predictable if you cannot be sure what the end users real ping might be.

Furthermore a simple oceanic Region option would help with this and allow further matchmaking diagnostics on Blizzards end if we’re being put in overseas servers.

1 Like