Algorithmic Handicapping (MMR) is Wrong for Overwatch

It’s only “fair” for the relatively good players to go up and the relatively bad players to go down if they each deserve to be ranked lower or higher than they are at prior to the match starting. So, in a sense, stacking the best against a worst could potentially end up producing a desired result.

The issue in play is that the result produced by making a match like that is one that is entirely fixed. The 6 worst (if the MMR is accurate) couldn’t reasonably beat the 6 best in a lobby. They’d be going through the motions in a match that serves no purpose other than to fulfill the arbitrary requirement of a game being played to produce the result the system wants (the best going higher and the worst going lower). So, while the result/rewards for the match would seem “fair”, the match itself is the most unfair possibility available for those 12 people.

If a system like this was used, how would it go about determining who deserved to climb and fall? Your hypothetical relies on putting people that “should” win on one side, but without matches being played that are fair to start with, how could it reliably say which side should win each game? I would offer the contention that each match needs to be as fair as possible (close to 50/50) to actually determine who is deserving of climbing and falling. It isn’t until you gauge performance in even matches that we could even lay claim to the idea that Player A deserves to be higher or lower. Repeatedly stacking matches with the best on one side and worst on the other is the epitome of rigged matchmaking - as the result will nearly always be what the system wants it to be, as opposed to a system that lets the players decide who wins and loses by balancing the matches.


For randomness, it is “fair” in the sense that the system isn’t trying to produce something balanced. After all… it’s random. That said, the match such a system produces isn’t necessarily fair. You may have the best all end up on one side and the worst on the other. You may have the best player stuck with the worst 5… and they might lose while being unable to carry the weight, so to speak. It will occasionally put the top 6 in an even split between the teams to get something that actually resembles a balanced match, but more often than not, the match quality will suffer.

The fairest matches are surely those where the teams are close to being even. I realize that in the short term that can result in unfair rankings at times, but in the longer term, I honestly believe that gives a more accurate picture than something truly random or a stacked system.

I get the feeling that your biggest complaint about the system isn’t actually aimed at what the system is attempting to do (make something balanced on a match by match basis that produces accurate rankings over time), but rather, it’s frequent inability to accomplish the task.

1 Like

I’m not saying that matches should be fixed this way, I’m just saying that it should be possible, and it should happen by random chance. Match Making Rating ensures that it never happens. This is not only unnatural but also counterproductive, because it ensures that players cannot be ranked accurately in the long term.

I understand your feelings about this, but they are based on an inverted understanding of what MMR is and what it does. I don’t blame you; Blizzard has encouraged your misconception.

You may be confusing me with other contributors to the thread. Some players have taken my side for the wrong reasons, saying that competitive matches are balanced badly or not enough. I’m saying that competitive matches should not be ‘balanced’ at all. Blizzard does this as a best practice for selling their game. It has nothing to do with fairness or objectivity, which is what I think players really want. And a ranked competitive game demands fairness and objectivity.

1 Like

I had a super long-winded reply prepared, but it seemed overly antagonistic, so I’m going to try and keep this simple. You’ve been nice to me throughout the many iterations of this thread, and I see no reason to be combative. So, I’ll make one point and ask one question:

You can’t reasonably pull the “I know you think you know, but you don’t. I’m the one that knows how it works.” card without sufficient evidence… which is something you’ve never managed to do in this or the other handicapping threads, imo. (In all fairness, we can’t get the evidence we would need without more transparency, but to suggest you’ve somehow found a truth while I’m an ignorant fool is simply insulting.)

So, here’s my question. It’s realistically the only important part of my reply:

Are the many other competitive games that exist that use similar matchmaking and MMR systems through various iterations of ELO/TrueSkill/Glicko to produce 50/50 matches also inherently unfair, unobjective, and unreliable when it comes to ranking players, or are your complaints exclusive to Blizzard’s implementation of the MMR system used in Overwatch?

2 Likes

Holy wall of text man! I do agree with you that mmr sucks but if you spent that time on your game play, you’d probably be top 500 lol

1 Like

Dont the other games release their mmr formula or whatever it is to math players? (i dont know so if you know please do tell ^_^)

Unlike Blizzard?

Also it would not be fair to compare OW comp to other game comps especially other FPS. As the composition of the team is already handled in the competitive mode for straight forward fps games. Whereas in Overwatch presumably we are matched in 50/50 matches.

But I would question are the teammates were getting even compatible.

For example i could be the highest rank player in the team with Lucio being my best Hero to play. If the rest of my team has a ana, moira, soldier, cree, and Hammond main. Then I would have to switch to rein or hopefully one of the healers.

That alone already puts my team at a unfair advantage and me. Since their “best” player/ highest sr player will not play their best hero. Or if I do get to play Lucio that means one of our healers are subject to play a hero that they dont play best. And again the team is at a disadvantage.

Though the argument can be made that the other team maybe subject to the same problem. It is the lack of consistency in the team matchmaker in which im calling attention to. Because at that point the Matchmaker is not calculating your skill per se but rather how many heroes you can competently play.

This can again be countered with: Well thats the point of overwatch! play different heroes. While I agreee I would atleast prefer if the matchmaker made the teams “balanced”. Where there is no overlap among roles between team mates. And let the players themselves decide when to switch to different roles rather than being forced to.

Also there is a difference between being competent at multiple heroes and being great at a select few.

“I fear not the man who has practiced 10,000 kicks once, but I fear the man who has practiced one kick 10,000 times.”

  • Bruce Lee
2 Likes

There are varying levels of transparency for different games, but there is a ton of information out there that covers the general ideas and math behind the more prominent systems. The most important thing to remember is that not being given the specifics on any game doesn’t automatically mean the system is designed to screw you over… which is one of the main contentions many supporters of the handicapping movement seem to repeatedly raise.

There is a tendency to believe the system has a sense of agency that is somehow singling them and a few others out, and then holding them back from getting the rank they believe they should be at, despite many other people being able to consistently hit the same rank without any issues. Notice that we rarely, if ever, see the inverse proposed - where someone is incapable of falling into the rank they deserve to be. Those complaining consistently seem to be held back with no fault to be placed on themselves. This, to me, is a problem.

It is unfair to compare OW to other games, but Cuthbert’s statement was made to point out his contention wasn’t that the balancing wasn’t good enough, but that the act of balancing the match to 50/50 on its own was a problem with the OW matchmaker. I’m specifically zoning in on the 50/50 vs. random aspect due to Cuthbert’s last post that heavily implies the act of attempting to balance a match at all is at the core of his handicapping stance.

I agree that the OW matchmaker has problems that can cause unfun experiences - the way groups can be handled, a lack of consistency with teammate quality and roles played, etc. That just isn’t the point of my most recent post.

This is one of those threads that is really difficult to get feel for the way each individual participating in the conversation feels about the matchmaker, 'cause there are tons of people that think it’s fine, some that think it needs improving, some that think it’s broken, and some that think it’s rigged… and we all seem to have slightly different rationale for why we feel the way we do.

2 Likes

But your own understanding of MMR is extremely wrong. Your understanding of MMR, if I recall, is that MMR is a differentiator between different players of the same SR, which is to say, the matchmaking system knows between “good” and “bad” players at the same SR, so let’s say, SR 2500, and intentionally balances the teams evenly based on “good” and “bad” players of the same SR. And you came up with that idea based on a quote by Scott Mercer that talks about 50/50 matches.

Said quote by Scott Mercer appears in post #3 of the below topic:

It’s an extremely large stretch to somehow interpret what Mercer said in that topic to somehow come up with the idea that the matchmaking system can differentiate between “good” and “bad” players at the same SR, especially because that wasn’t the point of that topic at all.

However, there is a more concrete statement by Jeff Kaplan that flat out states that MMR acts just like your SR, and unless you are a severely decayed player due to inactivity, your MMR is very linked to your SR.

(The following two links previews to the same topic, but I’m linking two different posts in the same topic)

https://us.battle.net/forums/en/overwatch/topic/20758686566#post-6

https://us.battle.net/forums/en/overwatch/topic/20758686566#post-16

And also posted on the Overwatch Twitter is that SR isn’t used at all in matchmaker, only the MMR.

Basically saying that the idea that matchmaking is done by taking people of the same SR and evenly distributing the “good” and “bad” players between the two teams is a very fallacious idea. There’s no good evidence supporting its existence, and there’s a lot of more concrete evidence refuting that idea entirely.

3 Likes

Does Overwatch have some kind of role queue now?

It has an in-game tool where you can find groups. This includes the ability to queue for specific roles.

So… kinda/sorta, but it isn’t widely utilized. I don’t think what was implemented was quite what players had in mind when they asked for a role queue.

1 Like

Here’s a quote from the OW dev team:

The simple and primary goal of our matchmaker is creating fair matches. To do that, it evaluates potential matches by synthesizing an expected win %. The matchmaker is normally really good about creating matches with a win % that is close to 50%

Why would the matchmaker synthesize a winrate for “potential” matches unless it’s waiting until it finds a match where each side has a 50% chance (or close) of winning? The matchmaker should grab 12 people of relatively similar SR and place them into a match at random and do no other arranging of teams whatsoever.

The way it actually works is If it grabs 12 people at random of similar SR and the synthesized winrate isn’t close to 50% it won’t launch the match. This keeps people at ranks they don’t belong.

3 Likes

Nothing in the quote you shared says that the matchmaker does anything more than match based on MMR.

It grabs people based on MMR, and matches from that. MMR similarity is the number one priority for the matchmaker.

And no one is being kept at a rank they don’t belong by the matchmaker; they’re being kept at a rank they belong in and have a warped mindset because of nonsense like this.

I read all your post and I totally agree, as a one trick Bastion in GM I sometime stuck myself in diamond on a alt account.

Now, I totally see what’s happening to me and why I always need to carry all my games. I currently have a 80% win rate in diamond but the games feel more and more difficult and I’m only earning 10 SR per win.

2 Likes

Good to know I’m as good as a one-trick Bastion in GM, I guess.

I can’t tell if this is a troll or not, because it’s too obvious…

I’m not trolling dude, I just say the truth about what is happening to me on a smurf in diamond and why I agree with this post.

1 Like

There’s no “stuck” if you do everything to win a match. If you one trick a hero, well, of course it’s gonna be harder. (counter comps, tilted players, …)

You one trick Bastion & Diamond is elo, where people start to coordinate & communicate more (Bastion is not that hard to kill if you have coordinated team).

Umm, you’re a GM, that’s okay that you get the feeling that you have to carry them. It’s because you don’t belong to their elo.

2 Likes

I keep having to tell you that the entire point of the devs using the 50% chance statement is that Overwatch allows groups. A game where everyone has the same MMR but it’s a team of 6 solo-queuers vs a full 6-stack is NOT 50/50 to the matchmaker. Even 6 solo vs 2 solo plus a 4-stack is NOT 50/50. Are you suggesting that the matchmaker shouldn’t factor in groups as well?

Yeah, but I only win like 19-23 SR per win which is strange.

1 Like

No, it is used in all of Match Making, just because one of the quotes is from a response to groups, no where do they segregate that this information is ONLY for groups.

1 Like

You said you earn 10 SR per win, and that you’re finding it difficult to climb that account with an 80% win-rate.

Now you’re saying you get 19-23 SR per win.

Like I said, I can’t tell, because it’s just too obvious.

What’s your problem dude, you don’t believe me so you say I am trolling ?