I am using the “last_login_timestamp” field from calling the WoW Character Profile API for all the guild members in the guild roster to determine if each guild member has recently had data updated.
My problem is that I have to do 1000 Char Profile API calls for a guild of 1000 members to see which guild members have had changes, and that alone takes approx 3 minutes even with using a multi-threaded app with 10 threads running.
If the “last_login_timestamp” field was returned for each guild member in the Guild Roster API, then I would only have to do that one API call in less than 1 second, rather than the 3 minutes and 1000 Character API calls to determine which guild members have been updated, at present.
This would make my guild roster update scans have a massive performance improvement by a factor of 1000, would cause much less hit on the wow api servers and less web traffic etc.
Please provide the last_login_timestamp for each guild member in the WoW Guild Roster API to dramatically improve guild roster data update scans.
The “last_login_timestamp” field returned from calling the WoW Character Profile API actually contains the Last Logout Timestamp, not the Last Login Timestamp.
It is great that the value it actually contains is the Last Logout timestamp because that is the timestamp when the data changes were actually committed for a character.
If it contained the Last Login Timestamp then it would not be possible to accurately determine if a change has recently been made or not.
However, this field should be renamed to be called “last_logout_timestamp” to avoid confusion and make data change comparisons more obvious.
I think it is a good idea to include the
last_login_timestamp for each guild member. It would be a lot simpler deal with updates for sure. I have just a few considerations here.
1000 req / 180 seconds = 5.5 req/s. This is a really bad performance for an asynchronous operation.
There sure is room for some improvement. A well written asynchronous code can fetch up to 100 reqs/second (current API limitation). That would actually take about 10s - 14s to fetch all the members.
Also note you don’t need to fetch the whole data for each character profile every time, you can use the
If-Modified-Since header to check if there was any change since the last time you updated. It will return an empty body with a
304 - Not Modified response if there is no change.
While such change is not implemented, if you want some advice on how to improve your code let us know and maybe we can help you out.
Schiller has good suggestions. Also make sure you’re requesting gzip transfer encoding so you have fewer bits to download on the wire. “Accept-Encoding: gzip”
Many thanks Schiller for your suggestions and fast response!
I actually found that I was only running 4 threads (not 10 threads) which was the main reason it was taking approx 3 mins to run ~1000 char profile api calls, as you guessed.
Probably I had lots of gateway errors a couple of months ago when I converted to the new api’s and had set the thread pool size down to 4 threads then.
The gateway performance seems much more robust now and I can crank up the async threads without hitting any gateway problems now.
After some more testing I found that my home pc was fastest at 100 threads doing ~1000 char profile api calls in 9-10 secs, using Java and Spring framework!
During these test runs, I was running the WoW game at the same time, and there were some extra api calls for guildies who had logged out between test runs
With more or less than 100 threads running it ran slower due to thread swapping delays or just fewer api calls made per sec.
So very pleased with that improvement in the blizz api gateway performance now, considering i am in Australia with 200 ms ping to US servers and 300 ms ping to EU servers.
A scan of a guild at max size of 1000 members, checking for any changes, in ~10 secs is a vast improvement!
I will try Schiller’s idea for using the If-Modified-Since header to check for 304-not modified when i get some time.
Also thanks to Erorus for your idea to accept compressed data with “Accept-Encoding: gzip” which would help reduce data transfer overheads, especially for achievs, raid progress, mount collections & pet collections which are huge responses!
However of course I still want the last-login-timestamp added to each guild member in the Guild Roster Profile response, because then I would only have to do one Guild Roster api call in a fraction of a second, which would save the overheads and delay of having to do 1000 char profile api calls albeit in ~10 secs.
So please progress this change request for the Guild Roster Profile api to include last-login-timestamp for all guild members in the list, to improve guild scan performance dramatically.
Thank you for the feedback Xang and to Schiller and Erorus for weighing in with the great suggestions.
We’ve recorded this request and may consider it for future updates to the Guild Roster API. Please note that we generally do not provide status updates or ETA’s on proposed future updates.