Algorithmic Handicapping (MMR) is Wrong for Overwatch

That quote is from Kotaku, not Scott Mercer. Overwatch Ranks You Lower Than It Should So You Feel Better About Yourself

And even if it was correct, it is very out of date. The main topic of the article, people’s SR being busted down below their MMR at the beginning of each season, hasn’t been true since Season 5.

If Skill Rating actually used win/lose results to rank players then you would be correct, sure.

You can’t balance matches then use the win/loss results, that’s so obviously stupid I’m actually shocked that anyone thinks that’s what they do. Didn’t you stop to think that maybe you’re missing something before you conclude that the system works in the dumbest way possible?

I described a system that uses win/loss records only. MMR is not it. This is why I don’t think you understand how Skill Rating works. I get that it is hard to understand how it works, so I tried to describe that in the linked post as well.

It’s telling that you keep saying that win/loss is how we rank players. I’m trying to tell you, as a simply factual matter, that’s not the only way to rank players. OW ranks based on estimation and validation, not really win or lose. Individual wins or loses are nearly meaningless to the system. Basically, a loss “counts” just as much as a win.

MMR being hidden is something we could debate, I guess, but even I don’t think having it hidden adds anything to an active account’s ranking. That’s why I say it doesn’t matter, not because it never matters to anything but because it’s not really relevant to your desire to get rid of MMR (after all, if it’s gone whether it’s hidden is a moot point).

MMR would work in any set of relative skills on the teams. I mean, any chance of winning between two teams would provide information for a skill based rating system. 50% estimation is a good starting point, but it’s not necessary at all. That’s what I mean when I say it doesn’t matter, it’s a goal for QoL but it’s not really important for actual ranking of players.

The thing is, if a skill based ranking system works, a 50% win rate is what a person will experience, because in ANY endeavor, two people of equal skill in that endeavor will triumph half the time.

2 Likes

Skill Rating doesn’t go up without a win, or down without a loss. Performance-based SR adjustment is Blizzard’s feeble attempt to restore the meritocracy of Competitive Play; to offset the profound SR-skewing effects of handicapping/MMR. It is a tacit admission that the SR system fails its supposed function of ranking players according to their skill.

If matches were made by an impartial system based on SR rather than MMR, wins and losses would naturally drive players into appropriate ranks. Blizzard chose to pervert that ideal, in order to make their game more addictive. They manufacture 50% odds for either team of every match, at the expense of the best players. At the benefit of the worst players.

Blizzard doesn’t want players to know about Match Making Rating, much less understand it. Competitive Overwatch lacks any user-interface mention of Match Making Rating. It lacks an in-game scoreboard. Blizzard doesn’t want players to perceive the fundamental dysfunction of the Skill Rating system.

1 Like

“They manufacture 50% odds for either team of every match, at the expense of the best players. At the benefit of the worst players.”

So basically what you’re saying is you’ve completely ignored for the millionth time everything we’ve explained to you about this. How does getting a fair match hurt any of the players on either team? How in the world can you be so utterly convinced that the system rewards bad players and hurts good ones, when clearly the better players get ranked higher, and there is a progression in skill across each rank.

1 Like

You must realize that PBSR is not necessary as part of a skill based rating system since it doesn’t exist above Diamond, so I’m not sure why you’re tugging at this thread at all. Are you claiming the system works fine for Diamond and above?

Also, words like “profound” make zero sense in this context. It’s not simply a synonym for “large”, it implies emotional importance. Using words wrong like this makes you sound like a charlatan. It would be easier to take you seriously if you stayed away from the thesaurus.

sigh The primary driver of the rating isn’t the win or loss, it’s WHO the win or loss is against. A win against me means something entirely different than a win against Jjonak. To think that it simply “goes up on a win and down on a loss” is exactly the mistake that you are making. That’s not, strictly speaking, true. Though in practice it appears that way, which is good.

You’re missing the forest for the trees. Read the post I linked to above (I’ll post again if you need it) and for Pete’s sake at least TRY to understand how people can be ranked without using a win/loss RATIO.

I’m curious what makes you special, that you can perceive these things that are so well hidden by the multitude of blue posts, tweets, and interviews that attempt to explain that secret MMR that they don’t want us to know about. If you could explain it without using language of morality but rather technical, referenced accuracy, I could be convinced. It’s hard to go from “balanced” matches to “unfair” matches. Bridge this gap.

Attacking the person, rather than the idea he brings to the table, isn’t refuting the idea. It just is a less obvious way to say you have nothing better to say, but don’t plan on backing down either.

1 Like

Just look at the amount of people who dislike it and think that Blizzard’s matchmaking does a very poor job. 83% of posts during the start of the seasons are complaining about placements. How in the world can you be so utterly convinced that it’s a good thing to have a majority of your users dislike the way you assign them to a team?

1 Like

So you complain he’s attacking the person, but then imply that popularity has some impact on truth. And with a statistic pulled from your rear, too!

The motivation for so many people to be wrong is pretty obvious…

Blizzard forums is an echo chamber filled with people who don’t understand why they are stuck at a particular rank. Post this same thread on the competitive overwatch section on reddit and watch how it gets taken down

1 Like

He did post it on the general Overwatch subreddit.

The result is available to witness.

1 Like

I tend to agree.

I’m not sure which part you find to be attacking the person, but I suspect it’s either my request for clarity regarding language or my request for details regarding a claim of special knowledge.

As for the first, there are entire careers that can be had by ensuring that writing is clear and it’s meaning is correctly conveyed. It’s hardly an insult to suggest edits and I’m not the first to point out that the tone of the post gives a poor impression.

As for the second, refuting a claim of special knowledge is discussing an idea, he appears to claim to perceive that which is hidden. Maybe there is some background of which I am unaware. If there is something he knows that the rest of us don’t know, he should tell us and give references so it can be verified.

Overall, I’ve given him much more intellectual respect than most in that I try to understand and discuss his actual idea, which often gets confused with the more common complaints.

There are many valid complaints about the system. You’ll find discussions on the hidden aspect of MMR, PBSR, throwers, leavers/disconnects, how groups are handled, and even aspects of matchmaking. Heck, I wrote a long (for an internet post) paper about how to handle team building better. No one is saying that nothing should be changed.

This particular discussion sometimes sounds like it’s about matchmaking, but it’s about ranking. As best I can understand, the OP wants ranking to be how matches are made, rather than teams to be built then your results used to rank you. He’s fine with the handicapping as long as the results aren’t used to rank you. As the result of playing ranked is that you are, well, ranked, he takes umbrage with the notion that teams are balanced prior to the game beginning.

I’m trying to get across that what he claims he wants is what he actually already has. I mean, if he understood how a skill based rating worked, he would realize that teams aren’t balanced then ranked, but that the ranking and the team building are both results of the same process. To be “matched only on SR”, that is, matched on your current rank only, it’s how this all works already.

The debate isn’t really about the conclusion, but rather the premise, of the post.

That is my beef with detractors of this thread. They assume that if anyone is dissatisfied with the system, there is no logical reason for it. But there is a huge part of the Competitive Play system (Match Making Rating) that is hidden from players. And we have very little idea how it’s designed. The people defending Blizzard are ignorant, failing to recognize how the company’s secrets pertain to their lives as consumers.

1 Like

If this is your chief complaint, you can make it in far fewer words.

Most of us would agree with you, too.

The rest of the post indicates that hidden MMR is only a secondary concern.

I just read all 260 replies, and @cuthbert, I disagree, but you’re an excellent orator, and I look forward to anything else that comes of this post

1 Like

The way I see it if my official SR (the only thing players have) says Im a plat player or a Diamond player there should be no case where I get matched against Jjonak, let a lone win against him. And if we accept a system where this happens then the game needs to be willing to propel me into GM instantly after 1 win and then we get to deal with all the deranking people on all levels of play.

If im mid Plat then why would I ever get matched against someone in Diamond? According to the game we are at a different skill level, at a minimum of 200sr appart and considering that the average SR gain is at about 20-30 SR per win it doesnt make sense to get placed against people who are, according to the official view of the game at a higher skill tier than me.

The way I see it you are either supposed to get matched with people of your official skill tier (with exceptions to rank up games) or we just accept that we dont have a Competitive mode, scrap the ranking system and officially declare that everything is made up and the points dont matter. Cause yeah I personally dont see how you can take points away from mid gold players for losing a match against plat players and call that system ‘‘working as intended’’.

1 Like

Do you think that this happens? In other words, do you think the system is so bad at balancing that gold players will oppose plat players? I’m actually asking, this thread is about how matches shouldn’t be balanced so the general assumption is that they are balanced. Maybe you’re talking theoretically or with groups of a wide range though.

Also, in a theoretical skill based rating system, a much lower ranked player WOULDN’T lose any, or many, points against a higher one, so your intuition about how it should work is correct.

The “against Jjonak” thing was an exaggeration. I didn’t mean to imply that that should happen. You’re in the right track, though, in that if it did happen, you’d get a large increase on a win.

Practically speaking, in the Overwatch system, such large differentials aren’t intended to be made. Bugs and bad people do happen though.

The general assumption is that the matches are balanced according to your internal MMR and not you SR, as you yourself said ‘‘its not about win or lose its about who the win or loss is against’’. In a game where SR is the point system that determines your skill rating theres 0 reason to bring the ‘‘who the win or loss is against’’ argument when you are supposed to be matched within your SR skill rating. Using the hidden MMR stat the way I see it should adjust your SR gains and losses based on your performace compared to other players in your SR rank instead of debating who you won or lost against.

This way its bad either way, if you get matched in a game where your team is favoured to win based on MMR and you win you will gain an abysmal amount of SR and end up feeling like you just wasted time. If you lose a match where you were the underdog sure you wont lose ‘‘much’’ but for some reason the game decided to give you a much harder match up, let you sit there in frustration playing against much better players and still lose points for it, even though you wont lose ‘‘a lot’’ of points you still lose points cause the game matched you into a harder match up than your SR suggests.

Yes, ive played with plats since I crossed mid gold and Ive been matched with Diamonds from about 2800 SR plat. And sure I mean you get like 30 SR for a win, but I still need to win 6 more games minimum matched with Diamonds as a Plat in order to get to Diamond ranking. And I mean sure, you lose less than you gain but youre still playing against players that are, according to the official ranking system, above your skill level. Which the way I see it should never happen except in wonky group situations, or youre about to rank up.

I dont understand the justification of being matched based on your MMR but be competitively classified by a different point system.
You either use one or the other or scrap the whole thing.

I wasn’t complaining; I was pointing out a fallacious arguement. I don’t see how attacking people has anything to do with a popularity contest either, nor why you would think I’m associating anything with “truth”.

If the question is “Do people like fish sticks?” and Kanye West loves fish sticks, and I point out that Kanye West likes fishsticks, I’m not saying “The answer is true.”

And sorry if you didn’t understand that I was obviously making up a statistic; I was lowballing the number just to be safe.

I guess it wasn’t that harsh of an attack. Also, if this entire argument is about rank in the sense that people don’t think they belong where they ought to be ranked and not how well the matchmaker puts together teams I am way lost and need to find my wrangler.

It can be hard to pin Cuthbert down on what his actual complaint is since he’s constantly trying to dance around reality, but in the distant past he slipped up and explicitly gave the ol’ “my teammates are all idiots and dragging me down and that’s the problem” argument once or twice. In fact he’s even edited his mammoth original post to remove such things.

edit to elaborate: I’m pretty sure his argument could still be boiled down to him being convinced that there exist players who are soooo much better than their rank indicates and that the game is punishing these secret stars for being good and thus keeping them low ranked.