This new "Profile API" is awful

API Discussion
1

Migrated WoWProgress to the new Profile API some time ago. The experience of consuming the API degraded a lot:

  • messed up guild activity timestamps even more
  • now need to do 6 requests instead of one
  • why adding separate /status request instead of including “status” field in the main request?

I know what people will say here: it needed to be done because it was “the right way” and now this new API will allow to add more data easily.

Guess what? I remember that when the old “Community API” was introduced, they said almost exactly the same.

This JSON structure gymnastics again… Move fields, rename fields, transform associative structure to lists with sequence keys, all that fun again.

And most important, split everything to as many requests as possible. This is good in theory. And this is fun for someone who wants to practice the new great and the new right way to make things done (and more complex).

Why do you need this separated “/status” request? The main request “/” is not heavy, you could just add “status” field there. Or even more simple, just return 404 if the character data needs been hidden.

You could even add it to the guild roster for each character. You could even add last updated timestamp to the guild roster. That what we were asking to implement for several years now. Since then servers in general got much more RAM to have guild roster in more actual state. But no, instead we’ve got next new great API with new “right things” and new bugs.

The most annoying part is trying to use guild activity to find new boss kills of guilds.
That was coded by someone who doesn’t understand timestamp concept.
Timestamp is a very simple way to have a single number to represent a moment of time not depending of time zone.
In guild activity timestamps were shifted by -2, -1, 1, 2 hours depending on realm. The logic is unknown (probably related to time zones). I had to find these shifts by logging these wrong values and building a table with shifts to work around this creative and delicate use of “timestamps”.

New API is out, now guess what, timestamps fixed? No. Guild activity tells us that character will kill a boss in 4 hours (in future). These shifts are now different. More different than before.

If we cannot use guild activity, we need to scan a whole roster.
Roster with 300 high level characters, 6 requests per character. 1800 requests just to check whether the guild killed a boss.
And I need to scan all progress guilds at least once a week or data is lost because someone decided that it is better to have timestamp of last boss kill instead of first boss kill.
The request queues got overgrown.

And about priorities.
There are about 1 million of players interested in raid progress and about 100 thousands ones interested in auction house scanning. I know people here discussing and asking things about AH more than asking about raid progress. But the thing is what it is.
By some reason you decided working and fixing progress things last is okay and that affected more people. Until recently you even wanted to throw out guild activity data…

All that static wow/creature/profession/items are not important at all. These who need it can get it from wow files or from websites that publish datamined wow files database before new patches are released.

3 Likes
2

This sounds exactly like my experience: Realm APIs Unneededly Complicated - #5 by rhox-1327 Here I was making a one-endpoint call with the old api into god knows how many now, all for checking something as silly as server population.

I thought maybe that’s just these endpoints and the rest of the API is better suited for people doing the real lifting. People like WoWProgress. But nope. A bunch of convoluted calls, exponentially increasing the amount of work the server and the client are doing just to get the same data.

Your post just more solidified my belief that these APIs are someone’s loose bindings directly to database objects with basically no regard for usability.

It felt like the folks that designed the API that was run through mashery understood the game, and data, and what clients would want from it. No it wasn’t perfect, but it was at least sensible. This API feels like an afterthought that shipped for a milestone to get a check mark in a column.

1 Like
3

In guild activity timestamps were shifted by -2, -1, 1, 2 hours depending on realm. The logic is unknown (probably related to time zones).

I guess the timestamps were relative to the realm’s time zone, which is annoying.

New API is out, now guess what, timestamps fixed? No. Guild activity tells us that character will kill a boss in 4 hours (in future). These shifts are now different. More different than before.

If they had any sense, the new timestamps should all be UTC. I don’t use the guild activity API so I can’t say for sure.

And most important, split everything to as many requests as possible. Why do you need this separated “/status” request? The main request “/” is not heavy, you could just add “status” field there.

Probably caching-related. Let’s say that Blizzard had everything related to a single character under one endpoint. Then when one thing changes (log in, check mail, log out), they invalidate everything and have to re-fetch everything before sending it to you in the response. Maybe their character privacy status endpoint references some other database and is a costly fetch, but it changes rarely. Maybe we don’t need to re-scan and generate all their profession spells when they change talents. I think they often refresh most stuff on logout anyway, but caching and bandwidth are improved with multiple endpoints.

You could even add it to the guild roster for each character. You could even add last updated timestamp to the guild roster.

If every time a guild member changed specs and logged out required an update to the roster and invalidating its cached value, that adds up to a lot of operations on their end.

Roster with 300 high level characters, 6 requests per character. 1800 requests just to check whether the guild killed a boss.

But they’re embarassingly parallel. Whether you send out 300 requests or 1800 requests to get back the same data isn’t a huge distinction. HTTP pipelining and persistent connections exist.

There are about 1 million of players interested in raid progress and about 100 thousands ones interested in auction house scanning. I know people here discussing and asking things about AH more than asking about raid progress. But the thing is what it is. By some reason you decided working and fixing progress things last is okay and that affected more people.

Squeaky wheels, man. If there’s a million players getting mostly what they need and 100k players with completely broken stuff, they’re gonna be louder. You gotta advocate for the things your users are looking for. The raid progress stuff isn’t my scene and I’m not gonna say it’s unimportant, but if I don’t hear about broken stuff from you guys, I’ll figure you’re doing pretty good.

All that static wow/creature/profession/items are not important at all. These who need it can get it from wow files or from websites that publish datamined wow files database before new patches are released.

Yeah I’m with you there. I mean, I get that they don’t want to drive people to datamine, but really, in-game dynamic data is what we’re here for, not to get an almost-complete list of all their mounts. (Or, at least, some of us aren’t.)

1 Like
4

But they’re embarassingly parallel. Whether you send out 300 requests or 1800 requests to get back the same data isn’t a huge distinction. HTTP pipelining and persistent connections exist.

But it does matter for Rate Limits.

1 Like