So they didn’t test it
1 Like
We’ll test it in production!
From experience, there’s actually a fourth option - the production environment is different from the staging / QA environment.
1 Like
So what you’re saying is they tested it on staging but not on production. But in the end that’s still just a roundabout way of saying C: they didn’t test it. Testing on production environment should always be the last step before release.
Despite your obvious intellect, you really come off as a Blizzard shill.
Entirely failing to integrate past MMR into the live season is not something that should have passed any sort of testing. We aren’t talking NASA precision, as nice as that sounds, we’re talking about basic competency.
If it’s as you said, and MMR is just stdev to the average player, everyone being a flat 0 would certainly have been visible. This isn’t a valid argument for it being public, but to say it wouldn’t have revealed the mistake is absurd.
Ideally (and this is what I do as a software architect at my company) staging and production environments are identical and staging uses copies of the production databases. However, even if this is exactly what they do at Blizzard, there’s no practical way to put the staging environment under a production-level load.
Even if they do literally test in production (and I’m certain they have a suite of smoke tests they run before making the servers publicly-accessible again after an upgrade), it wouldn’t be representative of live production servers under load.
I appreciate where you’re coming from, but it’s not an accurate picture.
2 Likes
When you change the title and content in your post, the replies don’t make sense.
3 Likes
I think that’s his new “tactic” for when he gets thoroughly shut down and doesn’t have enough echo chamber participation.
3 Likes
You say that, but the truth is this isn’t an issue of load. Past season MMR wasn’t integrated, this is something that should have showed up in staging even if the test was only using a subset of the production data.
Realistically, this is a single small mistake by a single person. The issue isn’t that a person made a mistake, it’s that there was no oversight to catch and correct it before reaching live. Failing to link an entire dataset that’s crucial to the game’s performance is not acceptable at the level Blizzard operates at.
- I didn’t say that it is an issue of load.
- What’s your source for what you’re positively claiming it actually is?
1 Like
well if I could delete the topic I would
I guess I will quote, quote, quote from now on.
3 Likes
Combined with the numerous reports of placements being ±600 from the accepted median of 2350, it seems to be pretty clear that is what happened.
Extrapolation and inference are not the same as proof for a claim.
1 Like
I’m not sure why you’re so inclined to defend them, but as a fellow programmer, can you honestly tell me it isn’t the most likely scenario? Everything appeared to be fully functional aside from MMR being treated as the default value, their developer outright said that the system was failing to integrate past matchmaking values.
There are limited explanations for this, and I can’t think of any that shouldn’t have showed up in a staging environment.
I’m not defending anybody, I’m just pointing out that you’re arguing based on assumptions. We don’t have the information needed to determine cause. We likely never will.
1 Like
Since I don’t go get a pitch fork any time a mob is forming, I must be a shill, right.
I’m pretty sure the issue is more subtle than that. It wasn’t just a MMR reset, as top players were only getting 2 SR for a win after placements. So it’s like they were getting matched as golds but gaining SR as GMs (who just stomped a bunch of golds). I’m guessing there is a factor in the matchmaker that was very wrong, but MMRs were correct. But that’s just a guess.
1 Like
I’d imagine it’d only decrease the work they need to do as they dont need to worry about people gaming the system as much.
Not like we all work for Blizzard, so they dont exactly stand to gain by us knowing their exact formulas.
1 Like
While it may decrease the work they need to do to keep it private, an ideal system would be very difficult to game. If you can climb while not actively helping your team to an adequate level, that’s something that should be addressed instead of hidden. (DPS moira comes to mind, with her amazing performance based SR gains at low ranks yet frustration provided to teammates.)
Tough to say the exact cause, but it seems to boil down to MMR not being integrated given where everyone placed and the comment by Bill Warnecke. Whether that’s due to it being improperly linked to the dataset, omitted in the calculations, or some other oddity on the back end… it’s something that should have easily occured during testing. It didn’t affect a small portion of games under specific and hard to recreate circumstances, it was everyone.
There’s a flaw in your logic.
Yet another option:
D: They tested it, and their QA team is good, and the problem didn’t show up in test environments.
E: The beta season was the test, and this was an automatic rollover. The problem didn’t show up in the beta season, and something was different with Season 18. (Note there was not a client release at the rollover time)
All of your 3 options assume Blizzard is incompetent. Defects making it into complex client / server applications is not a sign of incompetence. It happens all the time to the best software companies, they’ve just gotten very good at making it so fewer people notice problems.
It is cost prohibitive (I said impossible before edit) to fully test software. It’s always released in a state of “tested enough to have a high degree of confidence”, not fully tested.
1 Like