Algorithmic Handicapping (MMR) is Wrong for Overwatch

People like transparency. It’s why we demand it in every sector of business and government. Does it change how they play? No. But it opens up room for discussion and improvement. Right now, people are just speculating and yelling at each other based on nothing but made up pseudofacts about chess. I don’t see how that is any more constructive when the fact of the matter is: people don’t like the matchmaking process. Yes, it’s an opinion to not like something. But if a majority of people don’t like something, that’s different territory and brings us to the questions ‘Are they making this game for their users? Or are they making this game for themselves?’

2 Likes

Part of my point was that people don’t even believe, choose to misunderstand, or refuse to accept what they have given us. One side of this “debate” is purely speculation. The other side accepts what has been said at face value but asks questions about implementation details.

My point is that the side that thinks everything is speculation tends to also believe what information they have given is, at best, intentionally vague. They seem to want something concrete, but it’s really not at all clear how or why they think what we do have is not enough. I’ve seen mincing, parsing, and rejection of language on a whole 'nother level in these threads. If you won’t be happy until you have the math behind it, fine, but I somehow doubt the same people who confuse “estimated 50% chance of winning” with “forced 50%” win rate are going to be doing anything useful with that math.

I think it’s acceptably clear but could be clearer as an academic issue. MMR systems aren’t really that difficult to understand. It’s not exactly “speculation” to think that it works very similarly to all the other ones when it is described very similarly to all the other ones.

Unfortunately, I also think that the “rigged”, “handicap”, and “forced” reactions that come out of every single explanation of the system actually do more to convince them to keep it hidden than anything else.

I mean…if people take general concepts like attempting to have equally skill matched people on every team and turn them into multi-thousand post diatribes about how teams shouldn’t be balanced at all because it’s against some “competitive spirit”, why on Earth would they give us the detailed math behind each and every player and why they are in that particular match? HOW would they even do that and keep the game…a game…which is all it really is.

They’re not hiding it from people that are curious…they’re hiding it from people that twist words and intentions into something villainous.

Also, people tend to get things confused. MMR and matchmaking are related, but complaints about one isn’t necessarily a complaint about the other. For instance, it’s frustrating to get 5 Mercy mains on a team, but that has nothing at all to do with MMR. Related to that, people seem to want unrealistic perfection from the system. If you can figure out how to perfectly match 12 humans together every time you’ll be very, very rich in no time. There’s a discussion to be had there, sure, but people don’t even try to have a basic understanding before they start the discussion.

1 Like

There is a middle ground between being ultra-apathetic and raging about someone bringing up chess rating systems in a discussion of Overwatch’s rating system (which is not off topic, by the way). Stay in that middle ground. Take a break whenever you get angry. Feel free to come back and debate when you are calmer.

When I asked you a question, above, you just gave me a snippy response. That isn’t how debates work.

1 Like

So there is a blue post saying that SR and MMR are very close. They have also said that SR can go up or down while MMR doesn’t move much. These two points lead to an issue on what “very close” means.

In this game, it is very easy to go on a 10 game loss streak, even when you are playing the best Overwatch you ever have. Please don’t argue about the ability for this streak to happen, that would be a moot point as there are many ways it can happen (and I’ve experienced it along with many other people) - ie maybe it’s a top 500 GM smurf account trying to win but with a streak of leavers that just stopped them from being able to 1v6 for 10 games in a row. Focus instead on disputing the outcome and how it is reached. So, in one of those instances then SR will move down while MMR would barely move:

Start:
Game 1: SR: 2500 ESR: 2500
Game 2: SR: 2480 ESR: 2495
Game 3: SR: 2460 ESR: 2490
Game 4: SR: 2440 ESR: 2485
Game 5: SR: 2420 ESR: 2480
Game 6: SR: 2400 ESR: 2475
Game 7: SR: 2380 ESR: 2470
Game 8: SR: 2360 ESR: 2465
Game 9: SR: 2340 ESR: 2460
Game 10: SR: 2320 ESR: 2455

So after 10 games, this player now has a difference of over 130 between his SR and his ESR (ESR being the Effective SR if MMR was made into the “same” number for comparison purposes). So Blizzard must mean a greater difference is still “close”, and that “chase” could mean upwards of 40 games (based on how long they had people with abnormal SR games straight after placement before Season 5) for MMR and SR to realign.

But to realign then no further streak would need to happen. This game is very streaky, so every time a person has a streak like this, their MMR and SR would split further apart.

The only argument against this being possible is that MMR and SR are the same value always. Which contradicts the quote that SR always moves yet MMR doesn’t. And thus is speculation on what “very close” and “chases” mean.

As they now need to give that account a 50% chance to win, while having a mismatched SR/MMR, how MM works to reach that chance to win is still unsure. The recent post helped understand that SR definitely creates a limit on that predicted chance, but if they have 2 players that have that same mismatch then it still holds that they will always be put against each other, not on the same team. Otherwise that 50% chance to win would be skewed past 60%.

See, this is the reason I want to see MMR as well as SR (and I don’t care how MMR is made) is that it would within 1 day show whether the above is happening, that handicapping is happening.

If there is an obvious MMR vs SR issue leading to handicapping, then we could enter the “why” and “how to fix it” issues. And like Kaawumba said, if revealing MMR allows MMR to be easily manipulated by stat chasing then it was never good enough to begin with.

A few pre-counter arguments:

  1. But but I can climb still!
  • The above doesn’t mean you can’t climb. I have never said that.
  1. It contradicts blue post xyz
  • It doesn’t.
  1. SR matches MMR
  • There is a blue post describing how they move differently. If they move differently then they do not match. “The more certain the matchmaker is about your MMR, the less your MMR will change in either direction based on a win or loss.”
  1. MMR is described as a single number. Not composed of something and SR.
  • MMR is a number resulting from a formula. SR can be part of that formula. Or, think of it a different way - at the end of each match the adjustment to MMR is based off a formula that includes SR.
  1. If MMR depends on SR and SR depends on MMR, that would be circular.
  • Why does SR depend on MMR?

I’m not going to argue with you. It’s pointless. You make up a fictional “loss streak” and ESR numbers and expect someone to argue with you. No.

I don’t think you want to understand how it works. You want to think the system is designed in such a way that you can have an MMR that is permanently higher than your SR. You want to think that your 50% win rate at X SR is keeping you from climbing because you actually have a much higher ESR and of course you can’t climb with a 50% win rate.

In short. You want to think you are better than you are and the system is holding you back.

I posted something awhile back that explains this phenomenon of climbing with a 50% win rate. It’s not a silly question. I wondered the same thing. But once you think about it for a minute it’s easy to grasp.

https://us.forums.blizzard.com/en/overwatch/t/competitive-matchmakings-mmr-system-is-very-good/43184/28?u=ozoneooo-1681

There. It’s long, but it’s about the best explanation I have regarding your question, if it’s an honest question.

I hope I’m wrong about you.

1 Like

Because you can’t. Not because the numbers are made up, but because you can’t fault the logic. You’re the one who asked for maths.

Completely incorrect. I have even posted before how I prefer games in Plat. I’m a casual older player who is at a rank that works for me - I have fun games in this area because it seems to have the least amount of MMR/SR BS. In other words, I’m where my SR and MMR match. I’m too old to really improve anymore and play the game for fun.

I’m off to read this now, but can you clarify the question you think I’m asking/you are answering?

1 Like

The question is made quite clear in the post.

There is nothing in that link that has relevance to anything I posted. Are you sure you posted the right link?

What is my question? I didn’t ask a question, thus why I was confused.

Edit: did you mean this question?

That’s about a formula that determines SR based off your MMR, that would somehow affect your MMR, that would then affect your SR etc as a cyclical pattern.

1 Like

Sorry. I wasn’t trying to be coy. I was on my phone.

You only have one question in your post and it doesn’t appear to be more than a rhetorical question, I had to infer your question from context.

The question answered is in the second paragraph:

It’s the most common question I see. It’s perfectly legitimate too, but it’s not generally phrased in that manner. Often it’s couched in this “forced 50%” or “handicapping” language.

1 Like

You want to put money on that? You’d lose. Here’s an example of why.

Let’s take a Rein player who’s main goal is helping the team win. This Rein mainly shields, makes space and otherwise helps his dps and supports do their jobs safely. It works, his team has a 60% win rate. He gets 15 SR for a win and -25 for a loss.

This results in a net -10 SR per 10 games on average.
6 wins at 15 SR each = 90 SR
4 losses at -25 SR each = -100 SR

Out of frustration he tries something different. He sees how off tanks, who mainly focus on elims, get more SR per win. He becomes a dps Rein who only shields long enough to get to within hammer distance. Then it’s clobberin’ time.

His team drops from a 60% win rate to 50%, but a weird thing happens. Rein is getting 28 SR for a win and -23 SR for a loss. At 50% win rate he gets a net gain.
5 wins at 28 SR each = 140 SR
5 losses at -23 SR each = 115 SR
That’s a net SR gain of 25 per 10 games.

Now which would you rather do?
Before you get all crazy saying that I just made up these numbers, let me tell you that I have played Rein both ways. These numbers are accurate in my experience. The main reason I can’t regularly play Rein like this is people start tilting due to the losses.

The pain of a loss is immediate, and it takes many games to realize the SR increase from a lower win rate with higher SR gains per win.

So yes, if people knew how the SR calculation and matchmaking worked, they would change how they played in order to maximize their gains.

1 Like

Every piece of information is based on blue quotes. Silly forum doesn’t let me do links.

You seem to have failed to understand the post, which is that MMR does not match SR. As shown by your need to somehow make it about:

And I clearly said:

Which again means that your entire direction in this thread is off the rails, and has to be intentionally as I’ve seen you do it multiple times. Thus it’s trolling, and being marked as such.

You mentioned several times that you’re wondering how a mismatched MMR/SR combo can still have a 50% chance to win, right? You’re concerned that other players are put on your team to make that 50% chance, either worse or better, hence “handicapping”?

The post I shared is about how MMR is used to make a 50% win based on an individual’s MMR and that the MMR/SR of the remaining team actually doesn’t matter at all.

Ok, it’s not JUST about that. But it’s in there. It takes the whole post to develop that concept fully.

You comments would only make sense if you ignore everything in my post about how a mismatch in MMR and SR could be leading to handicapping.

So, if I am to read your linked post in reference to the above assumption being true, then it isn’t valid as it ignores how the handicapping is occurring. But if I am to read your linked post in reference to the above assumption being false, then it is simply irrelevant.

Before your post can be given any relevance, you need to first establish:

  1. That MMR and SR match
  • which is what my original post establishes as highly unlikely when logic is applied (from the blue quotes)
  1. That MM does not include SR in any way (as a part of MMR, or as a secondary restriction)
  • which again SR has been mentioned so many times of late in reference to MM that you’d be hard pressed to make a mild case
1 Like

Um…if you’re wondering how mismatched MMR/SR combos go back to matching you should look at decayed accounts which have mismatches far in excess of what you’re making up. It’s done all the time. They’ve explained it on at least one occasion, probably more. This should be a settled thing? It’s one of the more transparent aspects of the whole thing. It’s obvious, it’s also obvious when they did it in a small way when they suppressed initial SR to make people feel good several seasons ago.

So

  1. No one has ever (seriously) said they always match. In fact it makes more sense to think they rarely match, as “closely” and “chase” would indicate. But just because they rarely match it doesn’t follow that they are widely disparate, either. You’re refusing to notice the middle ground.

  2. Look, if you want to believe they do MMR in the dumbest way possible and then choose to twist their words of references 3, 21, 25, and 40 (quoted below) from Kaawumba’s guide and continue to claim that SR, somehow, for some reason, is one of the factors used to estimate skill, go right ahead. There’s nothing anyone can do to stop you. I don’t really care what you believe.

“This decay does not affect the internal matchmaking rating we use, so we can still place you in fair matches.”
“Skill Rating decays but your internal Matchmaking rating (the thing that determines who is matched against who – not SR) does not decay.”
“We use MMR for matchmaking, not SR.”
“MMR doesn’t decay. SR isn’t used to make a match.”

But I now see why you responded to my post:

I’m sorry I gave you the benefit of the doubt and started talking to you about implementation details. It seems was correct in my first estimation.

I’m pretty sure I’ve seen you say a few times that you can ignore MMR and go by SR as the true indication of what MMR is. I could be wrong, but there are quite a few people saying exactly that. My example showed that this is impossible.

You want to believe differently, based on zero evidence, and against the different blue posts that say that MMR doesn’t move much but we can see SR fluctuate by hundreds of points over the course of a single days.

This is pretty much a QED.

Ok, so that one dealt with let’s move onto 2.

During one of my “let’s actually test the system” periods, when I was climbing my silver account, at about 2200SR I was waiting 5 minutes for a “fair match” during peak time because their numbers are too skewed to allow a fair match unless there was another outlier on the other side. This is silly, and while climbing it is causing more painful games that are unfun than needed by doing this. If I was being matched purely on SR, then why did my Gold account (sitting on the machine next to the climbing Silver account) get games within a minute queueing at the same time? If it’s matched purely on MMR, then why did it still have games at the same SR level?

So yes it is stupid. That scenario should have been impossible. It was two machines sitting next to each other, both at the same SR, with no difference in ping. Something to do with MMR was screwing with the climbing account.

As for the quotes you gave, none of them actually contradict what I said though. I’ve gone into it before, I’m sure you’ve seen the arguments which you discredit because “reasons.” But even ignoring those you’ve failed to include the last six months worth of comments that talk about SR as part of matchmaking. So pretty much you put forward an invalid argument.

For the past year, pretty much every quote gives a little bit more info to show that handicapping is inherent in the system. When the next blue post comes out talking about how they’ve adjusted the SR range within MMR to make the LFG system work better, will you finally take notice?

1 Like

Citation needed.

20 characters.

I somewhat remembered that I first saw it in one of your posts.

Scott Mercer:
“There’s a couple of other things at work. One is the matchmaker’s confidence in what your MMR should be. Play a lot of games, it gets more certain. Don’t play Overwatch for a while, it gets less certain. … The more certain the matchmaker is about your MMR, the less your MMR will change in either direction based on a win or loss.”

Functionally equivalent is by definition not “always match”. What differences are there are either rare, obvious, or small. It makes no sense for the dev’s to differentiate between the two in posts unless necessary, but that doesn’t mean that it’s always the same number.

Even in your example, which ends up like 125 something different, it’s near equivalent. They give 5000 SR to play with, but it’s hard to notice a difference within +/- 100. It’s accurate, but not precise. This is one of my complaints about the system, that the numbers give an impression of precision that isn’t actually there. It’s like if you looked at your gps and it gave you a 10 digit latitude/longitude. That would give your “location” in terms of millimeters (ok, microns, technically). Assuming you’re bigger than a micron (you are), this level of precision becomes meaningless. There are many different 10 digit gps coordinates that are just as accurate and give your location just as correctly. There are many different SR values that describe your skill sufficiently.

You don’t know why. Don’t make things up.

You don’t have enough information to make this determination.

I’ve actually never disputed that handicapping exists in a VERY technical sense. I dispute it works the way described in that a person is matched with two numbers so that the end result is that people can’t climb due to their MMR being too high for their SR and getting “handicapped” by getting an enemy of a higher skill or a teammate of a lower skill…but with the same general SR range, leading to a 50% w/l ratio but a lower SR than their skill deserves.

Teams built on MMR alone are balanced so that one side doesn’t have all the higher MMR players (I assume here that it’s extremely unlikely that 12 people will have the exact same MMR). Handicapping really isn’t an accurate term for it but some people call it that anyways. It doesn’t matter who is on the team to generate a meaningful match for ranking purposes. That’s all covered in the post I linked. It’s also pretty common knowledge when talking about Elo systems in general. If you tried to learn how MMR systems work in general you would know this.

The alternative to matching based on estimated skill is to match based on nothing. Those who are good will have high win rates and those who are bad will have low win rates. Win rate rating vs. skill rating…we’ll call it.

People around 2350 will have around a 50% win rate in this system, but winrates will drop exponentially as you look further down the ranks. If T500 players are allowed on the same teams as <500 players, GM with Bronze, Masters with Silvers…etc. It would just be random.

You’ll get an accurate ranking that way, sure, but at the cost of a miserable play experience for 60-80 percent of the population. Your wins/losses will actually be by pure chance (if you got the GM or the Bronze teammates) if you are average and will be increasingly your job to carry/fail miserably the further up/down the ranks you go.

There are all sorts of subsequent problems with this especially considering no one is being forced to play the game and is unlikely to do so if they get stomped as a matter of course, but I’ll leave it here.

The other way to do it is to rank based on estimated skill. The details of which, again, are in my post.

So basically, “handicapping” (balanced matches) isn’t occurring in the way you think of it and it’s good for competitive play, not bad. Everything the anti-handicappers are afraid of, handicapping actually prevents.

You have to stretch the meaning of words pretty far to think that “We use MMR for matchmaking, not SR.” leaves open the possibility that SR is INSIDE MMR and used in that way. Not only is that crystal clear there’s no reason to think otherwise. The whole system wouldn’t make sense if they matched on both values. We all have this in common, really, that we recognize that such a system would be broken. The difference is that I take the idea that such a system would be broken, observe it’s not broken in the way it would be broken (no one would ever climb, it would literally be impossible, but no other effect would result), hear this phrase, and come to the conclusion that they “use MMR for matchmaking, not SR.”

You, somehow, take the idea that such a system would be broken, observe that the system is broken in an unrelated way (games being apparently random and streaky), and come to the conclusion that “use MMR for matchmaking, not SR” means “use MMR for matchmaking, with SR.”

I just don’t get it. I really don’t.

They don’t, they won’t, and adjusting SR range within MMR makes zero sense. The system isn’t the mysterious evil black box you think it is. It’s really not that hard.

You think this quote says somewhere in it that “MMR doesn’t [ever/usually/generally/often/once well established] move much?”

Like, which of those possibilities do you believe and what on earth do you mean by “much”?

And where again does it say that?

People are going to change how they play to maximize their gains no matter what. Right now it’s just like throwing everyone into a pit blindfolded. It’s like asking them to be toxic to each other.

1 Like