“Handicapping” or “MMR” - please settle this!

Competitive Discussion
1

Hello everyone,

I’ve followed the discussion about “Handicapping” and “MMR” for quite some time now. Everytime it’s basically Cuthbert vs. Kaawumba. One point of view vs. the other.

Cuthbert claims that MMR is basically a handicapping system, that places a handicap on the more skilled players in every team to create a match with a 50% win chance.

See his post “Why Handicapping (MMR) is Wrong for Competitive Play”

Kaawumba claims that the MMR is nothing more than a number that goes up if you win and down if you loose. The “Matchmaker” then tries to form similar teams based on MMR, with a 50% win chance for every team.

See his post “How Competitive Skill Rating Works (Season 8)”

But essentially, we as a community only bother with these systems because of one simple question:
Why can’t I climb out of my rank?

Again, there are two contradicting answers to this question, both of which represent one point of view in the “Handicapping vs. MMR” debate:

  1. I can’t climb because my teammates are holding me back! (Handicapping)
  2. I can’t climb because I’m not good enough. (MMR)

While I don’t have the answer to who is right, we should notice that the whole argument, boils down to one little detail:

How does the matchmaker distribute MMR across teams?

To make this as digestible as possible, let’s think for one second that MMR = SR. In fact it isn’t (it’s probably some very long and odd number like 389723,2903), but let’s just assume it. Now let’s look at how each faction thinks the matchmaker is building teams in a Solo Queue match.

The “Handicapping” factions thinks it’s like this:
Player (You) (2500) ----- Enemy 1 (2225)
Teammate 1 (2225) ----- Enemy 2 (2225)
Teammate 2 (2225) ----- Enemy 3 (2225)
Teammate 3 (2000) ----- Enemy 4 (2225)
Teammate 4 (2000) ----- Enemy 5 (2225)
Teammate 5 (2000) ----- Enemy 6 (1825)
Both teams have a rounded MMR of 2158,3 so the match is a 50% win chance for each team.

The “MMR” faction thinks it’s like this:
Player (You) (2500) ----- Enemy 1 (2480)
Teammate 1 (2510) ----- Enemy 2 (2490)
Teammate 2 (2470) ----- Enemy 3 (2520)
Teammate 3 (2480) ----- Enemy 4 (2510)
Teammate 4 (2460) ----- Enemy 5 (2450)
Teammate 5 (2530) ----- Enemy 6 (2500)
Both teams have a rounded MMR of 2491,6 so the match is a 50% win chance for each team.

As you can see, the whole argument is about the question:
Does the matchmaker place me into teams with considerably higher/lower MMR or does it place me into teams with about the same MMR?

In case of the former, it’s handicapping.
In case of the latter, it’s not.
Plain and simple.

So while I can understand that Blizzard would not make the details on how MMR is calculated publicly available, I think it is fair to have this simple question answered.

Does the matchmaker place me into teams with considerably higher/lower MMR or does it place me into teams with about the same MMR?

I wouldn’t be mad either way.
If it’s handicapping, then it’s probably because there simply aren’t enough people available within your skill range that are looking for a match right now. Matches would be handicapped, because otherwise, the queue times would be too long. That may be disappointing, but I would understand.
If it’s not handicapping, this means that you simply belong at your rank. That may also be disappointing, but it would probably help many players to accept it.

Please Jeff (or Geoff), or whoever reads this … just answer this simple question:
Does the matchmaker place me into teams with considerably higher/lower MMR or does it place me into teams with about the same MMR?

Let’s settle this once and for all!
Thank you!

[ Forum Moderator Note: Calling out the development team in your title is against our forum guidelines, and as such I have edited it to remove the callout. Please focus on gameplay discussion in your thread. ]

13 Likes
2

I hope he responds, but I’m sure he will mention there are multiple factors. Such as prime time and depending on your rank, that those two options change. If you’re in the gold rank range and playing on prime time you will get Option 2. If you’re GM playing off peak, you’re more likely to get option 1.

Every GM player knows this, because the range for queuing at off peak gives a huge range of players. This been a problem for a long time for GM players, but also was an exploit for team queuing at off peak. They said they would fix this for the longest time ever, and this season was suppose to be the season they finally fix it. I haven’t played this season to be able to confirm this.

2 Likes
3

I am pretty confident that it has been said/confirmed many times that you are matched with players of similar skill / MMR but still felt like I should do a quick search. The first relevant post I found was one from Jeff on 21st June 2016 (can’t post links) where he sort of finishes the MMR section by saying “All the system does when it comes to matching on skill is attempt to match you with people of a similar number.”.

Also if the system indeed would do odd things like your first example it would be super obvious. I mean it’s no news that for pretty much everyone their SR is a very precise proxy of their MMR rating so we would typically see a lot of matches where one player with much higher SR has been placed with a team of significantly lower SR players. But instead players have very similar SR and the average SR of each team is usually quite close to your individual SR (unless ofc your MMR says you are on a significantly different skill level than your SR says).

4

Saying that it’s just two sides of the argument is like saying the flat earth theory and reality are both on an equal playing field.

4 Likes
5

Yup. That about sums it up. After reading Cuthbert’s thread long enough I started to get flashbacks to reading arguments with Andrew Schlaffley.

To the OP: the first example you give only occurs when you are GM+ and the game literally can’t find a tight MMR spread for all the players. I guess it could also happen if you’re the wood-tier equivalent at the bottom of the system, but that probably happens at sub-500. It’s not 100% certain the matchmaker even allocates players to make sure that it’s as close to 50% as possible; the game may just shuffle to avoid some maximum level of imbalance and rely on SR reward/loss adjustments for the rest. The primary fallacy of thinking being “higher skilled” is punished is the mistaken belief that you’re more likely to be at the top of the MMR range for the match than you are any particular spot; the whole point of the matchmaker is to get everyone as close as possible, and unless you’re so good there’s just not enough competition at your rank the odds are against you being the top or bottom more than somewhere in the middle.

3 Likes
6

If handicapping is the thing keeping someone back in a certain rank i can surely say it’s false, with me as a counter-example. Season 4 i finished in Silver close to Gold. After placements in Season 5 i got around 1600-1700.
In season 6 i hit Diamond(I could have hit Diamond in Season 5 but i was just 4 SR away).
Highest i’ve been was 3492 (soooo close to Master 3500). Now i’m around mid-Diamond.
In just one season i managed to climb 1200+ SR with another 500 SR in the next one. This was NOT a smooth climb. I had ups and downs of over 400 SR along the way.

I lived the counter-example.

7

I don’t recall seeing the argument or question before, but how does the MMR chasing your SR factor into things? The game tries, as we’re told, to match as closely as possible (excluding some factors like human behavior that just can’t be tracked) to match teams with similar MMR, and thus SR.

But if you’re on a win streak, your MMR is now chasing your SR if the system notices you’re improving, and lowers if it notices a drastic fall. I’m always curious how that is factored into matching with teams. We always reference MMR/SR, at the moment, a static number, but it’s in constant flux.

8

In the Season where you climbed 1200 SR, how many games did you play?

I played 90 games this season, briefly hit Platinum, then went on a loosing streak and ended up pretty much exactly where I was placed.

9

Nice post! An issue that I’m having with this however is the assumption on the players MMR. The players MMR in most games is a completely separate system to their skill rating.

Your MMR is not a number that goes up or down.
Your SR is your position on the ladder.

In a majority of matchmaking systems your MMR is a string of characters representing you as a player. Your stats in each game are compiled and are given certain weight values depending on how long ago they were recorded.
This string of text is your MMR, it represents your playstyle, every statistic that it can get it’s hands on is accounted for.

How they compare multiple players MMR in matchmaking, that’s a different kettle of fish. It varies wildly from game to game but I can assure you that you are of a similar skill level to your teammates and your opponents in a match.

That’s just in my experience though, Overwatch could work completely differently!
Don’t expect a response from the devs on this, revealing your matchmaking system would be an expensive mistake.

Thanks for reading!

10

over 800…not really proud of that.

11

This is very wrong. Of course I can’t comment on every game, but modern MMR systems are generally extensions to Elo, which can be considered the founding idea. This is an entire are of machine learning research. Winning/losing is the most important, and often only, used factor. When other factors are included, they impact how much you move, but whether you go up/down is still based on win/loss.

Consider that in diamond and above personal performance has no impact on your rating and how it moves. It’s clear win/loss is the dominant factor in Overwatch.

Now, I like this summary, but I’d like to bring up another point against Cuthberts argument. Even if his absurd idea of always being teamed with “worse” players was true, it wouldn’t “handicap” your rank if the games were 50%. They might not be fun games, but they wouldn’t impede you climbing.

The game is balanced on the assumption your rank is accurate. So it’s only 50% if you are at your real rank. If you believe you can/should be higher, that means you play better than your current rank, which would unbalance the game in your teams favor. Whether you are the highest or lowest MMR player doesn’t matter for this.

Even in Cuth’s fantasy world, your rank wouldn’t be held down. That said, the idea that he is usually the best player on his team is an obvious delusion. You will be the best player on the team just as often as you are the worst player on your team.

4 Likes
12

The matchmaker will try to place you in a team with similar MMR.

Quote

In Overwatch, whether your MMR goes up or down is contingent on winning or losing. But there are a number of factors that determine how much that rating goes up or down. For example, what map you’re playing on and whether you were attacking or defending is factored in. We know the win rates on attack/defend on all of the maps and we normalize accordingly. Not all wins and losses are equal. We also look at your individual performance on each of the heroes you played during the match. Everyone has better and worse heroes and we have tons of data showing us what performance levels should be like on those heroes. We also look at your opponents and whether or not their matchmaking rating is higher or lower than yours. These are just a few of the things that are considered when determining how your skill should go up or down. At no point in MMR calculations do we look at your win/loss ratio and win/loss ratio is never used to determine who to match you with or against. We are not trying to drive your win/loss percentage toward a certain number (although the fact that so many people are at 50% win rates makes us extremely happy). All the system does when it comes to matching on skill is attempt to match you with people of a similar number.

Quoted from:
/forums/en/overwatch/topic/20745504371#post-3
(I’ve bolded and italicized the important piece)

1 Like
13

You want the truth? You don’t climb because you don’t play like people in the rank above yours. I’m not GM because I don’t play(yet) like a GM player does. Stop looking for failures in the system and try to improve. When you play in the level of the people in a rank, you will get there.

3 Likes
14

IMO the only 2 reasons Blizz can have for not releasing details is because

A: The system is faulty to an extent and they know people would be pissed
B: There’d be a way to exploit it if details were known.

4 Likes
15

People in GM get there for 3 reasons, They’re good, they’re carried, or they’re abusing a broken kit.

2 Likes
18

Blizzard owes Competitive Players complete information about the game they are playing. They have not explicitly stated the purpose of Match Making Rating (MMR), which is to handicap matches. They also have not explained its workings, and they have hidden it from view in Overwatch’s user interface. It is unethical to mislead competitive players this way, because everyone reasonably assumes that matchmaking is impartial. It’s what we expect from “Competitive Play.”

Many players would not engage in Competitive Play if they understood that it was being handicapped via performance-tracking algorithms. We have a right to choose how we spend our time, based on complete information about the games we play.

5 Likes
19

MMR places you with people around similar MMR. The longer the queue, the wider that gap gets. So fast queues, everyone is at the same MMR. Long queues means the MMR gap will slowly get bigger.

End of discussion.

1 Like
20

It is a Handicapping system. For season 8 I ran into a bunch of games where I had the most SR on my team and the team SR as a whole was a great deal below my actual SR - usually at least 200 points below it.

The way the handicapping works is rewards and penalties applied after the game ends. If a loss occurs and your SR is higher than the team SR you lose more points, and if yours is lower then you lose less points, conversely if a win occurs and your SR higher you gain less and if lower you gain more. Next if the two teams have about equal SR then no bonuses or penalties apply. However if one team has a significantly higher / lower average than the other then it “expects” the higher team to win and puts a higher bonus and smaller penalty to the lower team or a higher penalty and smaller bonus to the higher team. This means that even if you are winning 50% or less of your games you can actually move up in SR, and even if you’re at 50% or more you can actually move down in SR. Which is what happened to me in season 8. Overall I had a bad season, but at one point I had a stretch of 10 games during off hours play… I won 6 and lost 4 but ended up losing over 150 SR more than I gained thanks to only gaining 12~16 from wins and losing more than 60 per loss during that stretch of time.

3 Likes
21

I am prety sure its a MMR system. The only big difference with other simular systems is the performance based adjustment witch, In my oppinion, they should remove.

22

I want to mention that the forum moderator is right to disallow us from issuing ‘callouts’ to Blizzard. And I have to object to parts of this thread, like the subject line, because this isn’t really about settling an argument. This is about Blizzard providing competitive players with complete information about the game they are playing, which they have not done.

By hiding players’ Match Making Rating from them, and failing to disclose the constituent data of MMR, Blizzard is violating players’ right to consent in how they spend their time. They are distorting players very notion of what fair competition is, and what they are doing in Competitive Play.

So just to be clear, that is where my argument comes from. I’m happy to discuss this issue in other people’s threads. And thank you for taking notice of this debate. Just please be careful how you construe things.

3 Likes