Server queues in 2023?

Are we OK with queues in the modern era?

Back when Diablo I and II launched, and maybe even when D3 launched, a lot of server capacity was self-hosted. You had to plan weeks or months ahead to order servers and other gear, you had to physically install it, and you needed to be able to afford it - meaning you probably could afford what you need for average load but maybe not for peak load.

We live in a very different IT world now. AWS, Google, Microsoft, and others can literally spin up thousands of servers for you within seconds to minutes, bill you just for the capacity you use, and turn it off instantly after. Especially when you work with your account team in advance for a major peak like the launch of a game. Sure every bit of usage costs money but the point is that all that usage is from players who are paying to play the game, so it should all work out right?

So my point - should players still be accepting of queues in 2023? Or are we past the point where this should no longer be tolerated as an unavoidable limitation, and instead seen as a company not delivering what was paid for?

9 Likes

100% Agree, a fantastic point. The elastic capacity should be able to handle a massive traffic spike like launch day and scale down in days after to not waste money.

If Blizzard statically set their infrastructure size that’s a huge miss, and if they capped it to prevent spending too much money, that’s a miss too. They have promised rewards for completing the beta within a period of time, and included early access as part of a paid upgrade. They’ve made money already based on that customer promise.

They should have the elastic infrastructure to handle day 1 load, period. If it’s all static self-hosted servers in their own datacenters, well, that’s a dated infrastructure strategy from another era and its own problem we should call out.

5 Likes

I fully expect that the capacity knob is going to get turned up. This indeed should have been architected as an elastic-capacity cloud-based server infrastructure. It would make no sense to do otherwise.

If they start with a limited number of servers then they can investigate issues without getting overwhelmed addressing individual issues, and they likely do want to see what the limits are per server instance so they optimize the number of machines at launch.

Let’s see if things get better. If it all crashes and burns in three hours as we reach peak Friday evening prime time then I will indeed be disappointed.

No, server capacity should never be an issue anymore.

5 Likes

Also with modern containerized fleets of kubernetes clusters and pods, this REALLY should be able to scale rapidly if they’ve architected the backend correctly.

They know how many people preordered the game. Let’s say it was half a million. 500,000.

At a minimum they should have accounted for 50% of those wanting to log in on day one (250,000 connections), if not 100%. If only 25% log in on day one, fine. Scale the infra down that next morning while keeping watch on the load balancers.

Seriously man, login queues? I love Star Citizen and it’s sometimes a buggy pre-alpha but they do have their act together when it comes to server capacity on big patch launch days. You would think Blizzard would, too.

My worry here is it’s too big of a company now – they’re too bureaucratic and bean-counter money-focused to adequately prepare to offer their paying customers what they promised: open beta access on day 1 of launch, with a time limit to hit a certain level to reach a particular cosmetic reward.

Each day we can’t log in b/c of their capacity issues is one day we’re more stressed to hit that time limit they set for us, that really should be something called out more. I’d really like to see a company post about that.

  • What was the anticipated login traffic on day one?

  • How much capacity / player count was provisioned, what percentage of that total anticipated login traffic? 25%? More? Less?

  • How much traffic was queued? How many players spend how long in queues?

  • And most importantly, how many days out of the total allotted beta time before rewards threshold were lost to capacity / queue issues?

  • List item

1 Like

They’re throttling due to stability issues. I would assume if it was a sharding issue they’d spin up more shards, which means the stability issue is something platform-wide, rather than shard related. They typically have issues with authentication framework with all launches.

Do they really not do load and perf testing behind the scenes BEFORE letting the open beta floodgates open? Load generators exist for this reason, as do traffic replayers and synthetic transaction generators. This is a solved problem for the web industry for many years!

If this really is a platform-wide stability issue, then this game really wasn’t ready for open beta and they should probably have delayed the entry until that was higher confidence. Or emailed out beta keys to waves of invites until ready for more. Again, solved problem.

I’m just kind of stunned that this happens to them over and over and over – at some point this really should be systemically addressed on the backend. Or do they just know ravenous players will suck it up and deal with it anyway, so who cares?

Man, I’m spoiled working for web company that cares about its paying customers, haha. Gamers are considered fodder and fools that will pay for anything.

In 2023 you’re still complaining about server stability during a beta test?

You know one of the things they are ‘testing’ is how little they can get away with spending on server resources right?

Is there a game out there that is hyped up that doesn’t have login queues?

Yeah, I just refunded this nonsense because of this exact thought process. They know EXACTLY how many people were going to be logging on to this “Early Access” to their queue simulator. They should know exactly how much load these servers can handle, and with this ability to spin up new hardware on the fly, this should absolutely never happen in this day and age.

They under spec’d so much of this and knew they were doing it. I’m done, gaming is a dumpster fire and I hope some massive studio just utterly implodes because of this type of behavior so the rest of them figure it out.

Good luck folks

This is what gets me the most. Blizzard now has 20 years of experience with this. They know exactly how this works. They know how to set up and manage a server. Most modern game devs, especially AAA devs, don’t have these queue problems anymore. So why does Blizzard?

By all accounts, very, very few developers in the world should have as much experience and expertise at preventing these queues than Blizzard. But every single new title they produce still has these hideous queues.

Enough is enough. They must have some robust documentation or institutional wisdom about how to avoid this. Why don’t they actually use it?

The truth really does seem to be repeated reinforced by behavior like this: they just don’t seem to care. Their “customers” are ravenous loyal gamers to the lore, IP, nostalgia, and original company reputation. If people keep buying up preorders despite behavior and launches like this, then why would they change? Only hits to their pocketbook can influence that at a leadership level.

It’s gotta be frustrating for devs, producers, and artists, that pour their time and heart into making a wonderful fun experience, just to have execs and bean counter management needlessly botch the entire first impression with the world due to underprovisioning infrastructure. Come on man, this is 101 stuff at this day and age.

You think that just maybe one of the reasons for the beta is to iron out server issues before release?

Amazon had queues launching their own game lol, so maybe thats not as simple as you think