Loosing character data after game build update

Good time of the day,
This is not “an issue” but rather a suggestion. Can you please keep character data after the new build of the game is pushed?

What happens right now:

  1. We are on a game build AAAAA
  2. I am updating data for all guild members, including gear, achievements, media, etc.
  3. Game build changes to AAAAB
  4. Now media and some other endpoints returning 403 as user must log on to the character after the game build updated
  5. Since i’ve made a script to automatically remove characters from the database if they are no longer in my guild, script checks and removes / adds characters every 30 minutes, so everyone who fails to log-in 30 minutes after the game build update (as character will have no guild, well, 403 will be returned) will be removed

I totally get that the game build might introduce some changes which will be deemed “outdated” (like corruption values change, etc), but straight up blocking access (returning 403) to some parts of the API if the game build is updated seems a bit too much for me.

Greetings aftersoft,

Thank you for the feedback.

We believe that the situation you described should not be a recurring issue. The 403 response for some Character Media and other character related documents is a characteristic of the way these documents were originally published and the access control applied to them at that time.

These documents were originally published with a private scope while the API was under development. In order for a new version of the document to be published with the needed public scope, the character must first log out of the game, as you’ve noted.

Apologies for any confusion, and please do let us know if this addresses your concern.

I think @aftersoft is talking about patches to the game itself (8.3.0.33775 on March 23, 8.3.0.33724 on March 19, 8.3.0.33528 on March 2) but @Maguthul is talking about patches to the API (adding endpoints, changing schema details, whatever).

@aftersoft is reporting character documents going missing after game patches. I can neither confirm nor deny whether that happens, I haven’t noticed. Perhaps @Maguthul can tell us whether it is intended for character documents to become invalidated (and inaccessible from the API) whenever the game is patched.

@Maguthul
Thank you for your response, it is pretty much what @Erorus said.

The problem is, as soon as game build updates (for example 8.3.0.12345 to 8.3.0.12346), characters who have not logged in since the build update, will have no guild attached to them in the API response (i am fine with media not being there also as i just return placeholder images based on the race/class combo), hence most of the characters who “were” my guild are removed until they login and the next “guild check” window reached by the script.

What i am trying to say is that API for some reason returning nothing or throwing 403 error after the game build change for some endpoints related to the characters.

Would you be able to provide us with exact reproduction steps for the behavior you are observing from the API? Please include the URL and namespace for the endpoints you are consuming. We would not expect character documents that were previously accessible to suddenly become unavailable as the result of a game patch.

Okay, this post might get huge as i am going to provide complete guild member update process here

try {
    UpdateSingleCharacter::complete(
        $this->region,
        $this->realm,
        $member->name()
    );
} catch (ResourceAccessForbidden $exception) {
    // Do nothing as for some reason 403 is returned
}

This is the UpdateSingleCharacter class and the complete method which is responsible of updating all character data (items, specs, guild, etc)

public static function complete(string $region, string $realm, string $name) : void {
    $client = (new Client)->setRegion($region); // Initialize Battle.Net client with Credentials flow
    $character= $client->profile()->character($realm, $name); // Set the character information we are working with
    $models = (new CharacterCompleteProfileToModel($character->complete()))->models(); // Load all character data into respective Laravel Models

    foreach ($models as $modelClass => $modelData) {
        if (Arr::has($modelData, 'lookup')) { // 'Lookup' here is only set if the character already exists in the database
            self::executeQuery($modelClass, $modelData);
        } else {
            // This piece of code will only run if it is a new character
            foreach ($modelData as $value) {
                self::executeQuery($modelClass, $value);
            }
        }
    }
}

CharacterCompleteProfileToModel class constructor

public function __construct(CharacterEntity $character) {
    $this->character = $character;
    $this
        ->generateCharacterModelData() // This is character data model (id, name, gender, faction, race, etc)
        ->generateCharacterRealmData() // This is character realm data model
        ->generateCharacterGuildData() // This is character guild data model
        ->generateCharacterSpecializationsData()
        ->generateCharacterEquipmentData()
        ->generateCharacterProgressionData();
}

CharacterCompleteProfileToModel()->generateCharacterGuildData
This is where i am checking if character has guild attached, if it does not, then we have a problem as there is no reason guild member should not have guild attached to it

private function generateCharacterGuildData() : self {
    $guild = $this->character->guild();
    if ($guild !== null) {
        $this->models[CharacterGuildModel::class] = [
            'lookup'                =>  ['character_id', 'realm_id', 'guild_id'],
            'data'                  =>  [
                'character_id'      =>  $this->character->id(),
                'realm_id'          =>  $guild->realm()->id(),
                'guild_id'          =>  $guild->id(),
                'guild_name'        =>  $guild->name(),
            ]
        ];
    }
    return $this;
}

And this is how the entire CharacterEntity is generated

public function complete() : CharacterEntity {
    $cacheKey = sprintf('services::api::blizzard::characters::%s:%s', $this->realm, $this->name); // Create Redis cache key
    return Cache::remember($cacheKey, now()->addMinutes(5), function() { // Remeber character details in the cache for 5 minutes
        $character = $this->profile(); // Get character profile information `profile/wow/character/kazzak/Astratheon`
        try {
            // Try to load character equipment details `profile/wow/character/kazzak/Astratheon/equipment`
            $character->withEquipment($this->equipment());
        } catch (\Exception $exception) {}

        try {
            // Try to load character media details `profile/wow/character/kazzak/Astratheon/character-media`
            $character->withMedia($this->media());
        } catch (\Exception $exception) {}

        try {
            // Try to load character progression details `profile/wow/character/kazzak/Astratheon/encounters/raids`
            $character->withProgression($this->progression());
        } catch (\Exception $exception) {}

        try {
            // Try to load character specializations details `profile/wow/character/kazzak/Astratheon/specializations`
            $character->withSpecializations($this->specializations());
        } catch (\Exception $exception) {}

        return $character;
    });
}

And this is an update logic in text format

  1. Query API to load data about all guild members

  2. Get list of characters from database and assign respective models to the IDs of the guild members retrieved from API

    $models = [
    12345678 => null, // New character
    57473472 => DatabaseModel, // Model of the character in local database
    ]

  3. Iterate over the $models array and fetch data for all characters

  4. In case if for some reason the guild for the member of the $models array is null, remove that character from database
    a. This could happen if character has actually left the guild
    b. Also, if it is not null, BUT does not equals to my guild name, character should be removed
    c. Funny part: API returned 403 after the game build changed and character havent logged into the game

  5. Finish the job - added new characters if there are any, removed all which do not have guild associated with them

P.S. If you need any further clarifications, i am here to help
P.P.S. Namespaces are fine as all of the code covered by the tests, and all of them do pass. But just in case, the namespace is profile-${region}
P.P.P.S. “Root of all problems” is - profile/wow/character/%realm%/%character_name%, if it is missing data after the game build update, other endpoints might return 403. For example, if profile has no guild attached to the character (and i know character has guild as i’ve requested list of guild members before requesting profile information for a particular guild member), media endpoint will throw 403

Thanks for the detailed response aftersoft.

This part here concerns us as we would not expect documents to 403 simply because the game patched. For a document to 403 that would indicate that the document was re-published with a scope marking it as private. It’s possible there is an edge case bug happening here, but we suspect that it would be more widespread if so. If you can detect this has occurred, can you provide a link to a character or the JSON document for a character exhibiting the behavior?

We are wondering if this is a race condition between the character document publishing and the guild roster publishing. For instance, the guild roster only publishes every X hours, so it is possible that between the period where the guild roster publishes you request a character document but the character has since left the guild, meaning that their document would not contain a guild property, but the roster would show them as being in the guild.

We suggest checking the last-modified header on the character and guild roster documents and if a character is missing a guild, check if the guild roster was last published before the character document was published. If the guild roster was published before the character document was published, you may need to defer processing the character until the guild document publishes, or assume the character has left the guild.

Please let us know if that makes sense.