Since the patch on Aug 3, I have noticed a large amount of HTTP 429 errors being returned by the WoW API calls. I slowed down my services to roughly half of what is allowed, and yet the number of 429 errors did not seem to decrease at all. This makes me think that something else is going on, and that these 429 errors are incorrect.
Can confirm that since August 3, instead of being allowed to do 100 api calls per second (as per the developer portal documentation), I have been getting 429s after 5 api calls per second consistently.
The problem seems to have gotten much worse with today’s patch (Aug 10). I am trying to slow my services down, but I seem to get 429 errors no matter what I do. Currently I have it set to roughly 2 requests per second, which equals 7,200 requests per hour. Which should be well under the rate limit of 36,000/hour with 100/second burst. Would appreciate any help on this issue, thanks!
I got some support, posting it here for anybody else it might help:
The main problem appeared to be that I am using HTTP Keep-Alives during my API calls. This means that connections were staying open to the same server for a long time. However, Blizzard APIs try to load balance requests across many servers. Since my connections were holding on to a single server, I was receiving only a small portion of the overall rate limit, and therefore seeing lots of 429 errors.
I modified my code settings so that HTTP Keep-Alives will only last 5 minutes maximum. This hopefully gives most of the benefits of Keep-Alives while still ensuring connections cycle enough to get load balanced. Other suggestions were also to use more concurrent connections (which I was already doing) and to close out a connection after seeing a 429 error (which I could not easily accomplish).
Since making these changes, I have seen a large improvement, though I have still had a few 429 errors. I will continue to keep an eye out but so far it appears the issue can be considered resolved.
I still am not quite sure why things were working fine for so long, and then suddenly this problem started happening. I do think something changed. However, the solution 100% makes sense and am very thankful for the assistance. Thank you!
I’ve changed my HTTP client settings, it’s odd that I don’t get any 429 on US but I continue getting 429 on EU servers. US APIs sometimes don’t respond within 30 seconds which cause exception but we may look into this later.
Sample headers that I just recorded with 429 response on EU region:
x-trace-traceid: 005ddb48-2735-3b75-8aac-8208569120f3
x-trace-spanid: 1697d284-2ec7-4440-ab0a-fa165b5d6d4e
x-trace-parentspanid: 1697d284-2ec7-43f0-ab0a-fa165b5d6d4e
Thank you for your continued patience and those that sent sample traces to aid investigation. We looked into this issue more closely last week and were able to identify a configuration change that inadvertently reduced the per-node throttle in place. As called out, this would have impacted consumers that were relying on persistent connections or keep-alive more heavily.
The problem has been corrected so quota limits should be restored to the previous behavior. Please reach out if you continue to see issues.
Same here, the error rate has decreased significantly, but the problem remains. In my case(StarCraft2 API), only the EU endpoints are affected now(previously all regions were affected)
Hello! I experienced this 429 issue at approximately 11 PM Pacific today, so it appears to still be happening. The call was to /oauth/token. I tried again immediately and it worked on the 2nd attempt.
Further, when I went to log in to this site, I got a 500 Internal Server Error, so it looks like some shenanigans are still happening.
waiting for response…