The controversy is on the forums, in these very threads. Thousands of players have voiced their support for my argument and upvoted over the years, compared with only a few hundred detractors in the vocal minority. Of course you don’t hear about Match Making Rating and algorithmic handicapping anywhere else. It is only possible to discuss these topics with a shared foundation of factual knowledge, which I have put into my main thread on the subject. Everywhere Activision/Blizzard has blacked out the discussion by omitting and distorting crucial information.
Cuth, you know what I think. I think you need to stop this nonsense.
I try to do your argument justice, but in the end, I really just don’t see the problem with the system as is, even if I accept that it’s the way you describe. Which I think I do with some relatively minor changes, i.e. I think MMR tracks closer to SR than you do, and I think MMR is far dumber than you do, but neither of those would break your argument.
So, either I’m misunderstanding it, or you need to do a much better job at the “why is this a problem” part of your argument and work past the “this is happening” argument, which I already think is non-controversial, even if you gin up controversy through emotionally charged language.
Or you can just stop, man. I’m here because I’m genuinely curious about your argument and what you think, but part of that curiosity is caring, and I’m not sure if I can both care about you and not tell you to pack up and go home.
I mean…you made a video for a game you don’t even play? C’mon, bro.
I would REALLY APPRECIATE knowing if my post that you quoted now was a fair and accurate portrayal of what is meant by “rigging” and “algorithmic handicapping”.
I honestly think that you are getting at the larger issue here:
Using terms like “algorithmic handicapping” and “rigging” hinders meaningful communication. They are designed to hinder meaningful communication. Moreover, insisting that others use them and attempting to propagate them is, functionally, an attempt to frame the conversation in a way that is both loaded and non-specific.
It is loaded because both terms are framed in a negative way. It is non-specific because it does not actually discuss the systems in place. These terms exist in order to make it less likely that the actual specific tools can be discussed and to make it more likely that anyone participating in or reading the conversation will be given a negative impression of those systems.
And those are valued outcomes for some people in this discussion. They do not want to discuss real specifics, they do not want to acknowledge the reasons for these systems, they do not want to have that discussion in a value neutral way.
In short, they do not want to investigate. Investigation requires a neutral disposition. Investigation requires looking at specifics without having a particular outcome in mind. Instead, they want to confirm a supposition.
The supposition is that the system is evaluating them (and others) incorrectly. The supposition carries with it a pre-existing negative view of the systems. And it wants to have that supposition confirmed rather than investigating to discover how something works. Therefore, loaded terms that are non-specific are propagated in order to cut off any legitimate investigation/discourse.
We want to head off the possibility that something other than “this is terrible! burn it to the ground!” will be discovered, so loaded non-specific terms are useful. We would only discard such terms if we wanted to meaningfully explore the topic under consideration. And that is clearly not the goal here.
(Non-specific terms also allow us to do this dance where sometimes we acknowledge that it isn’t actually a forced 50% win rate, but most of the time we heavily imply that it is- even though that idea makes no sense whatsoever.)
You’ll note that the title of the thread doesn’t say “rate” or “chance”, just “forced 50%”.
I’ve always wondered if Cuthbert knows that he’s doing this. On one hand, it seems pretty next level to apply to a video game one doesn’t play. On the other, he’s very consistent and effective about it. He really missed his calling being a propagandist. He’s a natural at psy-ops. I’m less upset that he’s pushing this narrative than I am that he’s NOT using his obvious skills for his country’s national defense. I don’t know his politics, but I want him on my side.
Yes, I checked WyomingMyst’s archive. It is very nicely done but it is not comprehensive. It only has a small amount of the Battlenet forum’s contents, and it is missing the post from Scott Mercer that I am looking for and which I excerpted five years ago.
No need to ponder it. It’s not. And I believe you really should get A.H. down to a single paragraph/or short statement you can paste over and over without having to refer people to your lengthy thread post. I explained this issue to my dad who’s a lawyer and we came up with this succinct catch-all definition of AH/rigging:
“the systematic manipulation of matchmaking which undermines the competitive process, and can result in competitive outcomes that are different from chance encounters, and/or a process which helps decide an outcome in advance.”
I do think this definition is much more useful (because it is more specific- it invites examination). But I also think it is based on a pretty severe misunderstanding of how the matchmaker works.
As I’ve explained before, the matchmaker accomplishes several things by seeking out 50/50 matches:
It improves the competitive process (in the same way that having the best athletes compete against each other does).
It makes being ranked appropriately less grindy.
It makes rankings more accurate than more random matchmaking.
And it does not determine an outcome in advance. It does not even help to determine the outcome in advance. It merely ensures that we are best able to differentiate among the players based on their relative skills. More random matchmaking would make this harder, would undermine the competitive process, and would be more grindy.
Let me be clear in saying, I see the logic of what you’re saying. It’s like seeing a movie that you dislike and someone else likes, and you’re like “I can see how people would like this, but it’s not for me.” I don’t think anything you’re saying is unreasonable UNTIL you start making proclamations about what is and what is not in the matchmaker, because you literally cannot know.
My conclusions about the matchmaker are based on my firsthand experience, and what the likely experience would be if a company WERE trying to institute engagement (aka grind). From that, I INFER (not claim as true, but INFER) that there are shenanigans going on with the matchmaker. I never go so far as to claim that I KNOW such and such is the case. And that’s the problem I’m having with some of your arguments. You’re making statements that can – only – be derived through first hand knowledge of the algorithm itself: both what it includes, and what it doesn’t, both what it is, and what it is not.
Let’s say it’s X%, honest, ethical skill based matchmaking and Y% is engagement based shenanigans, how could you possibly claim to know values X and Y?
How could you possibly know how well the matchmaker IS accomplishing what it’s trying to accomplish without knowing WHAT it’s trying to accomplish? My resistance to your argument stems from you essentially claiming that what you don’t know about the matchmaker is fundamentally non-existent. You’re not coming at this from a more humble position of “I don’t know what I don’t know” (which to me seems an unavoidable circumstance of the matchmaker being a proprietary secret of which you know precious little), but rather a position of “I fully understand how this thing works, and I know what is and what isn’t in it.” And that is demonstrably – false. You DO have presuppositions, there ARE unknowns, there ARE variables, and there ARE unknown intentions, all of which are unknowable to you or anyone else making similar claims. This is a fundamental, inescapable problem as far as your claims go.
#3 I do agree with you on because we hashed this out already, and I said I changed my views on it accordingly, although I do think after X number of matches, the matchmaker DOES have enough information to relax or scale back its 50% rule sets, so that it’s essentially saying "ok if these players have all made it to X SR, avoiding a situation where bronzes are playing Plats, and we now have 2500 - 2800 SR players playing each other so that we don’t have to hold their hands and manage the experience (which you absolutely must do if the SR spread is 1000 SR as it is now). The reason why those against the current matchmaker want a lottery system is not necessarily because it’s the most accurate system – it’s because it’s a promise that engagement based matchmaking CANNOT be a reality in such a system. So yes, the current matchmaker might be some % better at doing things X, Y, Z but it – also – allows for the possibility of engagement based matchmaking to creep into the equation. And please, this last point is KEY. A pure lottery system allows for NO engagement based matchmaking, and that’s a major part of the argument made from this side of this debate, so please don’t dismiss this as an ancillary point.
This seems to defy logic, since if it serves no predictive purpose, why would it exist at all? Devs have said that they realize one team, according to their algorithms, may be at a disadvantage, based on their computations, and so they alter the points doled out to the winners and losers accordingly. This practice seems odds-based and would seem contrary to your claim.
In terms of grindiness, you’re saying that there’s more variability between 12 random people all within say 2500-2800 SR, than there is within the current iteration of the matchmaker where players can be 1000SR apart and whose skill levels can and do vary dramatically between different roles? I would ABSOLUTELY rather have Plat tanks and silver healers than vice versa, but the current matchmakers seems to think that these distinctions don’t matter so long as the SR spread balances out.
Futhermore in terms of addressing the problem of throwing or “casualness” when it comes to approaching competitive and large (1000 SR) player spreads, isn’t a lower ranked player far more likely to do things that contribute to a loss than a higher ranked player (by virtue of a higher ranked player’s ranking itself, which IS a demonstrable statement about their tendency to NOT throw or NOT play the game more casually)?
I’m basing my understanding on the same dev statements that Cuthbert likes to use (as well as my experience with similar systems and my experiences teaching others about those systems*). Those dev statements describe a competitive matchmaker that operates as other well-designed competitive matchmakers do. It’s true that they could be lying. I’m just pointing out that the statements Cuthbert is using do not mean what various people think they mean. They simply describe a matchmaker that operates as other highly praised competitive systems operate.
Things like pushing toward 50% accuracy, separate ratings for publicly visible ladder ranking and hidden skill ratings, accelerated SR gain/loss when there is increased uncertainty that the hidden MMR matches the SR, etc. are all best practices. They are used in a variety of fields because they have been demonstrated to work better than the alternatives.
We could all decide that the dev statements are lying and assume the worst, but if we are going to use the dev statements as evidence, we should try to understand what they are actually saying and the broader context of competitive ranking systems.
The way that Cuthbert and others interpret those statements suggests that everything that we have learned about similar systems over the past 60 years has been in vain. And that is an extraordinary claim to make on the basis of one’s personal impressions of the matchmaker (particularly when one does not acknowledge the wealth of other similar systems and the current best practices).
In particular, I expect these sort of anecdotal experiences to crop up with a well designed competitive ranking algorithm- I see them every day when discussing the GMAT with highly intelligent, educated students, who nevertheless make similar assumptions about that test.
I’m going to address the grindiness aspect again (although I’ve already written at least one lengthy explanation of this- I believe in this thread). And I’m going to address the way that predictions are used in these systems again (although I’ve written probably about 10 lengthy explanations of that across these various threads.) First why do well designed competitive ranking systems predict the outcome of the match before the match begins (and why do we learn more if we select for 50/50 matches)?
The ladder ranking is a prediction of the relative skill levels of the players on the ladder. It says that if we rank this player higher than that one and lower than this other player, they should contribute more to their team’s success than the players beneath them and less than the players above them. But we could be wrong! We play the matches to test this. The prediction is valid to the extent that our current rankings are valid. The more accurately we rank players on the ladder, the closer we will see the outcomes of the matches correspond with our predictions.
But again, if we are wrong our predictions will be wrong! The predictions are not determinative- the skills of the players determine the outcome of the matches.
That’s why we adjust the ladder rankings more when a team that has a lower average ranking beats a team with a higher average ranking. And that’s why most people will level out to roughly 50% win rate once they’ve played enough matches- we have enough information to make more accurate predictions. And that’s how we know where they should be ranked on the ladder.
The goal, then, with the matches is to gain as much information as possible about the relative skill levels of the players involved. And that happens when we intelligently (not randomly) make matches. We want to pit people who are close in skill against one another so we can determine who is better (and thereby improve the accuracy of our ladder rankings.)
Here’s another detour to a related field (because part of the assertion of these arguments is always that these systems are somehow unique to OW and other similar games because they want something other than what they purport to want- but why would something like the GMAT be designed that way as well, and why would top colleges want some weird online gaming system designed for nefarious purposes- they would not; they want the best understanding of the relative skill levels of all the people taking the test):
The GMAT tries to find questions that it predicts the test taker will get right roughly 50% of the time because it knows that it gains more information that way. If it gave easy questions to someone who would get any easy question correct 100% of the time, it gains no new information. That question was worthless for the purpose of accurately determining the skill of that test taker. The more random the selection of questions, the less useful they are. Some of those questions have no utility whatsoever for some test takers.
Similarly, in Overwatch, allowing some matches to be made that would be made randomly would have no value in a competitive ranking mode. They would provide no utility in testing our current understanding of the relative skill levels of the players in that match. And that is what these systems purport to accomplish. So random matchmaking is less useful for a well designed competitive ranking system.
As for grindiness- the very fact that a random matchmaker would make matches that serve no competitive ranking purpose, means that the random matchmaker is grindier than a matchmaker that seeks to make 50/50 matches. That 50% win rate prediction exists to ensure that we learn as much as possible from each and every match we make. That’s what prevents grindiness in the matchmaker.
And, as I pointed out in a different post, we’ve actually tested this in game- when people thwarted the matchmaker’s ability to find 50/50 matches what they got for their trouble was a less accurate competitive ladder and a much, much grindier experience.
