I think its just transition pains at this moment. In a year, it may not be as big of a deal and the active players are re-established.
The only thing that I’d like to say was already mentioned here, by multiple devs which solutions used by thousands of players on day-to-day basis.
As for me, this changes is a huge step back, and it will “kill” or heavily affects multiple sites like realmpop/TUJ/TSM/checkpvp/etc.
I doubles Aada-a-million-gold-mailbox about the market changes and I also pleased to see more API endpoints that allows us to receive more information from the auction house and do it much quicker then every hour as it is realised for now. For the other hand, Erorus is right about hashed-IDs which is could be compared between each auction house snapshots. He’s already paid for this with “realmpop”, and enabling anonymous lots-info on TUJ.
Also, I guess the whole community will agrees with this, that we need an endpoint which allows us to receive information about realm’s population and so on.
Kryo is right, that we still have guild-roster endpoint, which allows us to receive such data, but there are a lot of players which are not associated with any guild. And these players will become literally invisible, especially if the player has been inactive for a while.
For example, I have found many of my old in-game buddies, just because I could trace their characters via armory. And also we are unable to analyse data about “how much % of players have X”. So it will become a challenge to all of us.
While I agree it will be a step back for those apps lets also face the truth here: those endpoints were never meant to track realm population, as this kind of information is not publicly released by Blizzard itself since 2010 or around that year.
Personally I think it would be really great to have accurate realm population data, but I wish that information could be available for the realms endpoint or a specific endpoint.
Even if the AH endpoint still returned character names with the possibility of a character “hiding” their profile some statistics for realm population like class/race or even profession related data would be missing.
It would be really nice - and I know it is a huge thing to expect - to have a realm statistics endpoint. Something that would periodically return the amount of players, but also useful information like amount of characters that learned certain profession, amount of characters of certain race/class, amount of characters created per day over last X days, amount of characters above X item level, etc
That kind of information could really help players find the best realm to match their expectations.
And what is known about AH API for the classics, will access be introduced?
Blizzard read …
And now the delay reaches up to two hours on US realms.
I’m curious if it would be properly anonymized (and useful enough) to attach a “Session” hash to auction data
Specifically - I’m trying to track how item values change over time and the popularity of specific item values. It’s beneficial to know that two auctions were initiated by the same person
If, every day, a user was generated a new, random “Session ID” - and that Session ID was attached to all the auctions they posted that day - It would allow us to know the difference between 10 people posting 1 auction VS 1 person posting 10 auctions.
To “de-anonymize” a day, someone would already have to know a handful of auctions made by a specific user on a specific day. And then that’d only get them a day of information on that user.
Don’t know if it’s worth the technical hassle, on Blizzard’s side. But thought I’d pitch the idea!
I’m not sure a full migration is even possible - some of the endpoints have vastly different (and missing) data.
For example: Community API item lookup shows the source type of the item (crafted, world drop, etc) along with the recipe ID if crafted, but Game Data API doesn’t have any of this information at all.
Until that data exists at the new location, it doesn’t seem like a migration would be possible.
In my opinion, to achieve both user privacy and still have those kind of statistics a solution would be to remove seller names completely (even from in-game) and replace it with a seller ID. With something like that it would be even possible to allow users to share that ID with 3rd party tools to map their own auctions and even allow the user to regenerate a new ID whenever they need (similar to our API secrets).
I don’t spend much time on the AH so I might be missing something, but I never felt the need to know who is selling what except for the same reason people reported here: Are those 200 Netherweave Bags being sold by the same char or not ? At least for this purpose an ID could easily replace char names.
Anyway, I understand something like that might be even impossible to achieve, same as Melol just throwing some ideas.
The issue with using long-lived IDs for anonymization is this -
- Let’s say you tied your Auction History to a random ID
- Let’s say that I knew you sold a Netherweave Bag on 12/01/2019
- Let’s say that I knew you sold Akunda’s Bite on 12/10/2019
There are VERY few people who would have done that exact set of trades at those exact times. Each small piece of data narrows down the possible set of ID dramatically. It’s an extremely high-dimensional space
I would then be able to map every auction you’ve ever made, back to you through that ID. Also every auction you made in the future - unless you knew to hit a “Regenerate ID” button in which case I’d just repeat the process.
It’s an issue that Netflix and some other “anonymized” datasets have faced after being released to the public
But if you had a new, random generated ID for every small chunk of time then it would be much harder to de-anonymize (less data points) and you’d have to have insider information for every single random ID that was generated, to fully know someone’s history. You also theoretically couldn’t use information from the past to easily track a person through the present.
This could be done in two ways -
Every day or time the user logs in, automatically hit the “Regenerate ID” button that you proposed
Make the Session ID something like SHA(User ID + item.id + “12-01-2019”)
You could help the “Sharing of auction history” problem by literally making an OAuth endpoint that lists all the auctions the user has done
^ This information would not likely teach the system how to identify your Session ID moving forward - so wherever you authed wouldn’t have an indefinite amount of time to track you
I add the disclaimer for “probably” not allowing folks to track you in the future. Because if you’re the biggest supplier of Netherweave Bags in the game then folks can probably make some assumptions about the future when a bunch of bags appear all with the same Session ID. But that’s in no way an easy to automate nor is it relevant for 99.99% of all players
Regarding the AH API: I think most points have been made already, but let me provide some more background information on how I actually used the AH API until 2019-12-17.
I used the API for two quite different purposes:
- I play many different characters and several of them post auctions. I don’t want to spend too much time in-game on auctioning, so it was convenient to use the AH API to get a list of which of my own auctions had ended, and therefore know which alts should check their mailbox within a month. Since the owner names are no longer present I cannot see any longer which auctions are mine.
- Trying to estimate for how much gold products were actually selling, by comparing subsequent auction scans and determining which auctions disappeared. The problem is in the simple way that cannot distinguish between sold and canceled auctions (expired auctions can be estimated by checking the auction duration). My heuristic for distinguishing canceled vs. sold auctions revolved on checking if the same owner added new auctions for the same item that just disappeared. Without access to some kind of owner ID that test is no longer possible, polluting the “sold” data heavily with “canceled” data.
Solving these 2 quite different problems could be done in many different ways, but in all cases would require support from Blizzard.
The first problem is compounded by the problem that (last time I checked) there is no way to get the auction IDs that appear in the auction listings in-game. One option could be based on exposing the raw auction ID in the in-game AH API (in the list of pending auctions), so an addon could save that information for your own auctions to its LUA saved variables, where an external app can pick it up and correlate it with the auction scans.
An alternative could be a new web API that would return the list of pending auctions for “myself” (parametrized by my character name or battle.net id)
The second problem is harder to solve without leaking identifiers for all auction owners (which I guess was the core issue in the eyes of Blizzard). Several people already suggested introducing new character identifiers instead of actual user names. As far as I am concerned, what would be enough for this second problem is to have an identifier that is unique for each (owner, item ID) combination, but can not be reversed to the owner ID. To put it mathematically, something like HASH([server secret],[owner id], [item id]) could work.
A very different solution would be to realize what we are actually trying to do: get a listing of all auctions and their final fate. The list of current auctions is merely a way of approaching that. If there were an API that retrieves actual sales information (in a form suitable for statistical analysis), that would be perfect!
There is a third solution that would solve the problem of distinguishing cancellations from actual sales. I suspect that not every auction player will be happy when I suggest it but here it goes: remove the ability to cancel auctions …
My use-cases are very similar to Yoco - Love the detailed breakdown!
I was also in the process of trying to automatically identify what I was calling “Price fixers” - Folks who randomly put over-priced items into the Auction House, messing with all estimations of what the actual worth of items are
^ Not to call them out or anything but to remove their messy data from my calculations of item popularity/value
But with the removal of Owner ID I can’t identify whether or not items are actually worth X gold or if the same “Price Fixer” has flooded the Auction House 100 times with obscene prices to try to mess with folks
Yep, I also used to check for those profiles trying to overprice everything.
That is why I thought a Seller ID would be a good solution. You still know the same person posted the same items but you don’t actually know who is this person. Because unless that person shared the ID with a 3rd party app, there is no easy way to associate the ID to a character or bnet account.
Something similar is used in some e-commerce platforms where you don’t expose a partner’s personal information to end customers (except for legal issues) and the whole transaction is managed by the platform.
But again, this only works if seller names were also removed from the in-game AH, otherwise it is very easy to map the IDs.
- Love it!
Sounds like a solution that’d help us all for these use cases while still maintaining a large level of privacy for sellers is either:
Session ID = Hash(Secret + User ID + Item ID)
Session ID = Hash(Secret + User ID + Item ID + (week date?, month date?, day date?, login time?))
Then an OAuth endpoint that gives access to the auctions lists for the given user
You could also theoretically use that ID in the UI (like Schiller recommended) of the Auction House (though some effort would have to be made to make it easier for the user to understand visually). That should fully anonymize the AH data while allowing folks to still better understand trends and market manipulation!
Totally possible but it’d depend on if Blizzard finds it useful and secure enough to implement!