I am simpatico! It is so ironically devastating…
I was around 300ish subs prior to all of this nonsense with the guild bank stuff, with over 250 videos. I’m definitely not a big influencer on YouTube – I made most of my impact on my blog, over a decade ago.
That said, I am absolutely astonished to see so many people subscribing and viewing my videos about it, and it only encourages me to stay at it and become more vocal. Prior to this, I’d have the odd video now and again but very little in terms of consistent effort. Still, you can go back and see some of my old raid videos, if you like. 
Server-first 25-person Heroic Hagara! 
16 Likes
Super kind of you, thank you!
I fully agree about this. Twitter is not the place I want to go to get game updates. You can’t even see what the latest stuff is because on the profile page, it’s listed by likes or some other metric that makes no sense for a customer support account. Maddening.
Thanks to you both for your support while I try to make more noise about this!
8 Likes
Nothing new here either, but I’m not particularly surprised, especially since it was a short maintenance. Alas.
6 Likes
Yep…Twitter is an outright malicious site that’s not safe to visit. The only way to make it even potentially safe is to use a secure browser on a virtual machine that has no access to any of your data, and then put that behind a VPN, using a server in a country other than the one you’re in. Obviously not worth it. I just did some router magic to completely block twitter from my network, no traffic in or out.
6 Likes
As a side-note I’ve only watched about half the video and I noticed you have a nice Youtube-voice, the video sounds very articulate/well-organized
8 Likes
I just ran a WotLK Timewalking dungeon. Got some of that expansion’s mats, and it made me sick, because I had tens of thousands of those mats in my guild bank before they were deleted.
20 Likes
I already wrote a bunch of words about this.
It wasn’t random. They know what happened. They know when it happened. They know why it happened. What they weren’t or aren’t doing is scraping logs and analyzing snapshots to figure out how to put it all back together.
If they had the data (logs and snapshots) necessary to restore everything or nearly everything, and lost them, that would be the kind of thing that happens in an undisciplined environment.
If they never had data and snapshots necessary for the restore, that’s incomprehensibly incompetent and also vanishingly unlikely.
The hypothesis that they had and perhaps still have everything needed for the restore, and have decided it is just too much work when all they’re losing is 100 or 1,000 or 10,000 customers, I find very disappointing and very plausible.
But whatever, there is a lengthy chain of errors involved here and a whole bunch of things that wouldn’t and shouldn’t fail – if any one of many people was awake and competent – failed anyway.
18 Likes
To be fair, this is actually a very good point. Despite I have tested stuff on the PTR, I never actually paid much attention to guilds or the guild bank while doing so. So, I can actually see how a bug like this could potentially slip through the cracks. Blizzard’s response is still inexcusable though; they still ought to have froze guild banks the moment it was discovered and done emergency maintenance … I just can see how it could have legitimately been overlooked.
4 Likes
Exactly. Considering they found it within a day or two of reports but sat on it for nearly 5 weeks before acknowledging? The longer they waited, the harder and more complicated recovery became. Inexcusable.
13 Likes
Yeah this seems about all we can do. Word of mouth is slow, but it works. I don’t believe they don’t have a backup they can’t use, we’re just not worth it to them. The lack of communication and transparency, they don’t care enough to even attempt to explain what happened. “here’s a forum sticky, go cry, you’ll be back…” No I don’t think so, and I’m going to make sure everyone knows what a pitiful excuse for a gaming company Blizzard has become. It’s like recruit-a-friend in reverse, starting with Blizz deleting your mount. OK Blizz, you win, I’ll get the word out.
9 Likes
Was cross-realm guild vault even enabled in any public test? I don’t know, but I don’t think it was.
A couple thousand messages ago, I already talked about how this would be pushed to production in a “normal” environment, and it’s safe to say that none of the normal best practice safeguards for this kind of change were in place, or if they were, they weren’t working or weren’t being monitored.
The original basis of the WoW architecture was that items have unique IDs (oids) per server. In order to move an item it has to be destroyed on one server and created on another, in an atomic fashion, so that if the process fails at some point, it fails in a consistent state (no duping, no item loss). This is not a trivial piece of coding, as it involves separate database and service instances.
Anyway, roll that into the future and make it so that items can be moved from one server to another en masse simply by being deposited into a vault (when an item is deposited to a vault its ownership is transferred to the guild master, and of course if necessary it is moved to the guild master’s server), and you have a situation where … you would think … turning on this feature would be very carefully monitored.
Then apparently some “maintenance” process came along, and, completely unmonitored, started annihilating items. Did some dashboard somewhere have a count of items for each server? Did anyone notice the count dropping by 1,000s-10,000s-100,000s on affected servers? Were there “test guild vaults” aka “canaries” that were monitored during the “maintenance” change?
Even without knowing anything for real about the internals, I can say confidently that this was some really, really bad operational process that led to this situation. The fact that apparently developers didn’t even detect it until reports started flowing in from what was left of CS (which seems like 1 person in the US) is mind-blowing.
6 Likes
Exactly how I felt during the last TW week (The Burning Crusade). I was sending Primal Nether, Netherweave cloth, some motes, etc. to my bank toon . . . and I’d usually be happily putting it all into the right tabs . . . but of course, those tabs were nearly empty at that point. The reminder of it all felt devastating.
I lovely stranger sent me their items a few days before they quit the game (I almost cried when I saw the stuff in the mail) so that has made it more bearable - not seeing every tab either completely or close to empty. (In my most affected bank guild, I think about 98% of the spaces became empty after the “bank heist.”) It still hurts but the kindness (from another player, NOT Blizzard) has helped and I’m slowly rebuilding - won’t get everything back in there of course since I collected them over 18 years, and quite a bit of it during the course of playing through those expansions when they were current content. And then later, through TWs or just farming to make sure I had plenty . . . even had gone out of my way to get things like Essences of Water since they are used for Tome of Illusion: Azeroth (Vanilla enchant appearances) among other things, and I wanted to be ready in case someone wanted one.
5 Likes
similarly, just because you don’t like the blue post about the problem doesn’t mean they’re lying or covering anything up.
what evidence are people submitting?
1 Like
Absolutely. This will be especially so with ACTIVE guilds. There might have been an item that everyone know was being saved for some special occasion or something and because the tabs didn’t LOOK that unusual, no one noticed - but probably will when they are ready to use the item. Hopefully, they will have heard about this or read one of the articles (but not all players regularly read WoW sites or gaming sites) so no drama ensues.
I have a few banks that I THINK were either untouched or not affected TOO badly, but even then, I’m not 100% positive about it, as I didn’t screenshot or do any inventory - I never had a reason to as no one TOOK them (at least in the guilds where there is just me or me and one other person).
4 Likes
I was following some of the Tweets (back when it was Twitter) by the “BetterABK” people, and the way they were treated sounded like it was pretty bad - underpaid, overworked and often, looked down upon by devs (maybe not ALL devs . . .)
A lot of these jobs that are ESSENTIAL are some of the low-paid ones, and because they are “support,” not considered as important. And may even be cut due to them not “directly” making the company money (even though both customers and workers - even the ones that treat the people in that dept. like trash, KNOW how essential they are). It’s possible that they may not get the support THEY need to function well (resources, time, supportive leadership, etc).
It’s not the fault of the QA personnel, but the what the company DOES with it. Are they being utilized well? Are they given the TIME to do their job effectively? Are they staffed adequately or better? Are they given the tools necessary to do their job? Do they receive adequate or better training when they are hired? Are they sticking people who have no idea how WORLD OF WARCRAFT works, into the job of testing WoW systems, with not much training (like, they worked on some Xbox games but never WoW?) Are the devs LISTENING TO THEM when they say “hey, uhh, we might have a problem here . . .”? Have the layoffs hurt the department to the point that they can’t do things like TEST BIG FEATURE CHANGES that could have catastrophic effect if not done correctly?
I DO hope they and others departments (Customer Service, for example) are being helped by being unionized. They really needed it. (Though apparently, doesn’t seem to have saved them from another HUGE layoff wave).
5 Likes
This is part of what makes it hard to forgive something like this bug, too. It’s never the QAs’ fault, the CMs, CS, etc. It’s always the higher ups that sign off, set the pace, and vibe. I want to believe Longdale in earnest that they’re listening and taking player feedback more to heart, because I’ve seen when that has been the case. But this is almost like a shadow finally catching up after all the turnover, the abysmal morale and the layoffs. So I agree with quite a few that if they can’t show compensation efforts for what can’t be returned, there’s nothing to make good on the next time or time after.
Honestly, it’s a bit ironic this is going on around the same time people are picking up a certain recently released book and finding out how much of wow’s history has been a cash cow for funding other internal projects. I almost believe that’s still the case and curse in regards to this topic.
6 Likes
They say never attribute to malice that which is adequately explained by stupidity, but this situation is not adequately explained by stupidity alone.
I make no apologies for the rumours and explanations that people (myself included) come up with to try and make sense of this situation in the absence of coherent communication from Blizzard.
Screenshots of logs, before/after screenshots of bank tabs, addon inventory data, are the main things that come to mind. Some people had the foresight to make detailed records before the patch hit just incase anything went wrong.
Whether its sufficient to justify restorations would have to be assessed case by case against whatever remaining data Blizzard does have, but we’re way beyond the point of “sorry, we lost the records that prove that we lost your data, so there’s nothing we can do”
8 Likes
No, they aren’t reading the tickets. Not with the responses I got. That much is clear.
4 Likes
It takes a few tries to break through the AI/cookie cutter responses, then a few more tries to get past the employees who are only allowed to link forum posts in response to key words in your message, and then you might start to see some genuine human interaction, people actually reading, comprehending, and appropriately responding to your tickets.
There’s still not a lot of things they are actually capable of doing, but I am currently at this stage in my ticket series, and right now they seem to be convinced that the fix that is being deployed be developers will in fact restore all missing items.
3 Likes