I’m the author of WoWdb.app, and I use the Blizzard APIs to pull data into my database for my mobile app.
Right now, I have some arbitrary number like 200k for the max ID for items. Because IDs don’t start at 1 and sometimes there are no valid items for several IDs, there’s no way to add logic to stop checking after x number of invalid responses.
Maybe there could be an endpoint that would return the highest ID in the Blizzard database so I can stop querying after that number? I understand that some types (like items and spells) have way too many records and can’t have the index endpoint.
What’s the best way to deal with this that would lower my scraping time and burden on Blizzard APIs?