“Some of the boats are arriving. Not great, not terrible.”
Appreciate the insight into what happened. Thanks to everyone for your hard work resolving this and many other issues.
I really enjoyed this post, Blizzard! I understand very little of the technical talk here — I’m a Special Education teacher, not a programmer — but I was able to understand what you’re saying.
In other words? This post did a truly fantastic job explaining the issue in layman’s terms. That is a rare talent, Kaivax! I’m honestly impressed.
(Also: HUH, no wonder our CPU usage has been so high lately. Interesting.)
This so much. WotLK pre-patch was a disaster. The Achievement system was introduced, and when you logged in for the first time, the system was overloaded trying to record all your past achieves. People had a hard time logging in for 3-4 days.
MoP - Alliance players had all kinds of issues with their boat getting to Pandarea.
WoD - The Garrisions caused lag issues for weeks.
Thanks for the hard work! Love the engineering update on what has been going on in the background!
I the new expansion.
An incredibly fun and insightful read! I hope there will be more of these in the future, even with all the “lol just test better 4head” calls. This stuff is not easy, and it’s great to have a peek at what goes on behind the scenes, especially when delivered in the form of an entertaining and well written post like this one.
Thank you for this, and thank you for the wonderful expansion!
It’s great to see transparency and communication with the player base regarding technical issues like these, a lot of the time I struggle to comprehend the full scale and effort it takes to keep all of the servers running smoothly.
Utter nonsense, the quality is well above and beyond previous. With SL, it just “worked” in fact.
This makes launch day sound like a massive raid event at Blizzard HQ
You need to be better than this. The quality of launches in BFA, SL, and DF have not been to the quality that Blizzard used to be 10 years ago.
The fact you’ve said this just exposes that you never actually experienced any of the other launches.
I’m also from the software industry and work with HA systems where HA for us means four nines (less than 53 minutes downtime per year). If we don’t hit that in a year that’s considered a huge problem. When I play this game and still see weekly 1-2 hour service windows plus multi-yearly DDOS outages it does make me wonder where all of the billions of dollars is going. By my estimate WoW is sitting somewhere between two nines and one & 3/4 nines which is roughly 3-11 (full) days a year of service outage. In regular software that would go against almost any reasonable SLA and that product would be scrapped or would be shedding customers.
I’ve dipped my toes into the game industry in the past and have many contacts in that industry still. I fully understand that releasing major patches is difficult and that unexpected problems are inevitable. I can’t even begin to imagine the level of spaghetti code that is driving an 18 year old product these days. But I also don’t understand how something as simple as “A whole bunch of people tried to get on a boat and when that didn’t work a whole bunch of people tried to click a portal” can cause a problem that catches anyone by surprise. Gold farmers have been running bots that can simulate player behavior for decades. Surely there’s a way out there to spin up enough bots to simulate a launch event on a smaller scale?
But perhaps there’s a cost/benefit analysis component that we don’t see. Afterall, we’re probably all going to play the game anyways. I doubt anyone who purchased Dragonflight cancelled their subscription because the boats didn’t run on time or because the game is unavailable for 1+ most Tuesday mornings.
Very interesting read! Thanks!
Not gonna lie that was a good read and I think this is the type of thing many of us are exactly looking for. I’ve never minded the downtime or maintenance when it comes with what’s going on in the backend.
Seriously, make more posts like this in the future. I love reading about stuff like this and I think it goes a long way in helping retain player confidence.
My memories of Dragonflight would have been the same had I started at 8pm or 9pm.
But my memories of standing on the Stormwind docks with 300 of my closest friends laughing every time someone launched themselves towards the boat to Boralus and DC’d are forever.
This. This right here, this is what players love to see. Thank you for your time and patience with us and writing and explaining it so that we can understand it to an extent. It’s the radio silence that I think bothers most players. Thank you guys for working so hard on the game!
I like the transparency. People assume just because things ran smooth in the past that it’s always going to run the same way. They don’t perceive how complex these game machines really are. Any change can have unforeseen and cascading effects. Good on you guys for going into detail on the fix. Nice work!
As a software developer myself, that was amazing to see you break it down for the users. Setting an amazing precedent with this transparency. <3
Thank you for sharing this, really interesting writeup! I love it when teams share these kinds of stories!