Algorithmic Handicapping (MMR) is Wrong for Overwatch

Competitive Discussion
1081

Few more thoughts. People, that like to analyze and speculate things in regards to Blizzard’s matchmaking system are in a bad position to begin with.
Blizzard can tell you whatever they want about their system, that might have nothing to do with their real implementation, and there’s pretty much zero falsifiability, because they can simply say “We have all the data, you don’t. What we say is true, what you say is false” and what means do you have to prove them wrong?
Become a cybercriminal, hack their servers and steal, or reverse engineer their formula?!

But that doesn’t mean, that we need to take everything they say as gospel.

1 Like
1082

  1. The point isn’t that unfair teachers don’t exist, but that some people will blame their failure on it, even when the teacher WAS fair. Which obviously happens.

  2. Let’s be clear, the matchmaker attempting to make a 50% match doesn’t mean it always will suceed. Maybe you go up against once of those bronze to GM smurfs with an incorrect MMR, for example. Not every game will be winnable. But these things happen to everyone at the same rate. You aren’t specifically targeted by the system, so it doens’t hold down your rank in anyway.

I’m the person who made the follow up reddit post (which got way more upvotes) about why MHz theory is wrong. Just like Cuthbert’s, it relies on an incorrect understanding of how the matchmaker works. We don’t know all the details about the matchmaker, but what we have been told is enough to disprove it. So either you have to believe Blizzard has gone full conspiracy and lied to players, or Cuthbert+MHz are wrong.

I do appreciate the MHz theory though, because it’s the exact opposite of Cuthberts. In his idea, playing well means you’ll be “marked to rank up” and given easier matches, rather than “handicapped” with bad teammates. Both are CONVINCED that their theory is true, based on their experiences. Clearly, both can’t be true. And if one was true, it means the others is completely wrong, the opposite of what is happening!

You could not ask for a better example of how anecdotal data is flawed, how people find patterns in randomness, and confirmation bias.

I guess on to Jorlan…

Well, as long as they stopped doing things to SR that make it different from MMR, since those would obviously break the matchmaker. (Imagine if decayed players actually became smurfs!) But yes if you changed SR to act just like MMR, it could be used instead of MMR. Or we could just make MMR visible. Same thing either way.

By punishment I meant have your SR lowered. Both leaving and decaying lowers your SR (leaving lowers more than a normal loss). That is the punishment, though leaving also entails other punishments.

But PBSR has nothing to do with rank distribution? Basically Blizzard decided to adjust the percentage of players in different brackets. I.e. what MMR matches up to what SR/tier.

It’s just another example of the type of stuff they can use SR for, that they couldn’t if they just had visible MMR. Not saying it was a good idea, but that’s the sort of flexibility that SR provides them.

Did you forget what you were responding too? It was a list of how minor ways SR is used and different than MMR. Hence, answering why they don’t just set SR=MMR or equivalently use visible MMR. Also why SR has nothing to do with matchmaking.

Fair enough, but I think that has to do with a matter of perspective. Go skim through the “Trueskill” paper I posted, for example. If you’re considering what the optimal update to a player’s MMR and confidence is after a game, and doing so via Bayesian inference, yeah that’s complicated.

But we aren’t discussing details like that. If we’re talking about how the game tries to match players with equal MMR, that’s not very complicated. Make groups of 12 with as close of MMRs as possible based on the people queued up. This is clearly the best thing to do, and very simple.

1083

@FriendlyFire

By all means, do link it, I would love to read it.

Nah. Let’s be honest here, no post is like that of Cuthbert’s :stuck_out_tongue:
MHz at least understood the basics about the system. I’ve read Cuthbert’s post throughout the years. To call his thinking “outside of the box” would be a charitable interpretation.
Don’t make me go back and read the original thread (actually, I think there were multiple) on the old forums, but basically many that speculated, that there might be an issue with the system didn’t agree on the specifics with Cuthbert, because he didn’t even understand basic things.

To make an analogy, if MHz post was about the shape of the Earth, it would be “I’s round, but I disagree on some specifics”, while Cuthbert’s would be “Nah, it’s flat!.. and we live in the Matrix!” No offense, Cuthbert.

While I like MHz’s hypothesis better, let’s not judge them based on which one sounds like a better motivational speech.

Slight offtopic, so that you better understand what I mean:
I recently watched a show called Big Brother, or at least my country’s version of it. By no means quality entertainment, but I do like one of the people that participated, so I watched it.
There was this situation, that I found rather interesting:

Big Brother did split the participants into two teams. Then, Big Brother rigged the game, so that one of the teams would win multiple missions one after another and would totally dominate the other team, by giving them the correct answers in advance and assisted them in any way possible.
When the “losers” were asked by Big Brother “How come you failed so many missions?”, they doubted their own coordination and capabilities and it took them a long time and very ridiculous situations to figure out, that they were actually being trolled, and that there was nothing they could have done.

So, while I do agree, that it’s good for people to try and take responsibility and to look for faults first in themselves, you shouldn’t take this as a religion and let that blind you, when there might be other things at play.

Back on topic.

Both are CONVINCED that their theory is true, based on their experiences. Clearly, both can’t be true. And if one was true, it means the others is completely wrong, the opposite of what is happening!

In MHz’s hypothesis, there was a scenario, where you could get marked for derank and matched with legitimate derankers, because you don’t satisfy one or several of the metrics the personal performance factor uses to assess if you’re good… and it might decide you’re not good, despite you playing well and contribution for a win.

So I would argue, that this thing from MHz’s hypothesis does accomodate the “handicapping” part of Cuthbert’s theory… but there were other parts, that didn’t match.

We don’t know all those factors, and I think MHz might very well be correct… unless you’ve totally disproven him, which I’m sorry to say, but at this point in time I kinda doubt it, but I would still love to read it.

Thanks in advance.

2 Likes
1084

That is no actual punishment because mmr is unaffected, meaning within a few games you are back where you were.

Being able to adjust gains and losses can not be used for adjusting distribution? Ok, you are fairly clueless on this part still.

So you believe hidden mmr is for flexibility. I am trying to inform others (logical others) that this could be reached other ways without the need for hidden mmr.

So, from this bizarre comment, you believe PBSR allows mmr to diverge from SR - which by the way is the crux of one type of handicapping.

I get what you trying to say here. The update to SR/mmr could be super complicated, while matchmaker part is simple.

This makes sense until you hear that the matchmaker takes 12 people of “similar mmr” and then synthesizes as close to 50% win chance as possible. This synthesis would not be necessary if all players were considered equal as you argue.

1085

https://www.reddit.com/r/Overwatch/comments/7he0b7/the_frontpage_post_about_the_match_maker_is_100/

While you could say the example you gave, with MHz’s theory, “handicaps” a player, based on Cuthbert’s theory the player would be given easier games. Rather than marked to derank, the game would assume they are worse (lower MMR) and then pair them with higher MMR players to compensate. The ideas are perfect opposites.

You can read the post but to summarize, in an MMR system, your MMR is the games best guess as to what your skill is. There’s no concept of “you should be a at a higher rank”, or not, and thus having the system mark players to go up/down. There’s just your MMR. If the system thinks your MMR should be higher after a game, it just increases it. That’s exactly what happens when you win! (and reverse when you lose).

Once you understand only MMR is used for matchmaking (which is an indisputable fact) it should be very clear how the ideas of both Cuthbert/MHz fall apart.

Most people consider having their visible rank lowered a punishment, the entire point of decay is to get high rank players to keep playing to avoid it. You are right you will catch back up. I also think the whole decay thing is terrible, but that is a separate issue entirely.

No, it can’t be used for adjusting distribution. PBSR is inherently neutral. What one player gains from it, another must lose. Even if it wasn’t, we know (given that you will get bonus SR based on your MMR) that there is some matching of this SR is equivalent to this MMR. That matching determines the distribution. Set the level for 3000 SR higher, and less people will be in diamond, or vice versa. This completely controls the distribution of ranks.

Hey, I agree with you! I mean, they can’t do some of the things they did without it, but in principle I don’t think they need hidden MMR. I played Dota2, which has visible MMR, though it’s pretty rare among games these days.

Again, complaining about how ranks are displayed and SR is separate from matchmaking and whether players are handicapped with worse teammates.

I think so, at least temporarily (bonuses will always fix it). Though we are now getting into details of the matchmaker beyond what I think any of us players can determine.

Still, possible small deviations in SR and MMR due to PBSR doesn’t impact matchmaking and whether players are handicapped!

Sadly there aren’t always 12 equal players, especially when you get into the rank extremes like with top 500 players. Sometimes, the matchmaker has to compromise and do the best it can with the players in queue.

1086

The same reason to which you deny that the matchmaker creates matches to meet as close to 50% chance and doesn’t randomly give you 11 other people within your MMR.

Jeff Kaplan “At a most basic level, the matchmaker is trying to put you with 11 other people. But it doesn’t just randomly select 11 people. It takes into account a number of factors (more than I am going to list and not necessarily prioritized).”
-Overwatch Forums

Scott Mercer " The simple and primary goal of our matchmaker is creating fair matches. To do that, it evaluates potential matches by synthesizing an expected win %. The matchmaker is normally really good about creating matches with a win % that is close to 50%"
-Overwatch Forums

2 Likes
1087

this: exactly what I’ve been saying for almost 2 years. their system blocks smurfs but makes it so that your SR doesn’t correlate with your MMR at all. then they just balance from a wider MMR range to create balanced matches. However, the play level of the players in the game end up being lower the better you do to balance out your MMR gains… it may create 50-50 matches but at the expense of creating a system that punishes players for playing too well within their rank. In reality players with low MMR for a given SR should not be placed in 50-50 matches. They should be placed in matches they have a strong likelihood of losing so that they drop back to where they belong. However, people are likely to quit the game if they have loss streaks, so the current system was created

1 Like
1088

I’ll ignore everything else since we’ve argued those points for 3 years. I’ve done my best to prove you argue lemons while everyone else is comparing apples and oranges.

This point however again shows to me such a lack of imagination.

For instance, if they remove the hidden mmr and make SR the new mmr, you can redistribute based on stats. Too many people in plat and you want people averaging lower? Add a higher weighting to aim, a lot of players will go down more than up. Chose the stats and weightings you want. There is no way a zero sum game with PBSR as you claim, as it’s entire purpose is to value the individual against all players of similar SR/mmr not just those in that match.

QED? Yeah, sorry that’s QED.

1089

It’s not zero-sum per match, it’s zero-sum for the system. While you could come up with a different type of PBSR that isn’t zero-sum, that’s not what OW has. (Unless the devs are lying).

PBSR is based on average performance, right? The difference between your own play, and the average (on that hero/rank/etc). Now, to establish whether it’s zero-sum or not, we need to look at the average impact of PBSR, which is how much SR it gives (or takes) on average.

Well, the difference between the average performance of players, and the average performance of players, is zero. They are the same thing, by definition. PBSR based on the average performance of players is inherently zero sum.

QED.

1090

on a side note: as long as the current system exists I will be abusing it to widow practice. so whenever I lose a couple games, you guys better expect one heavy widow for you to carry

1 Like
1091

@FriendlyFire

All you need to disprove the claims is take a look at what happens to any decayed player.

Some people are asking “Couldn’t decayed players be treated different?” .

Theoretically, yes, but consider this. The way decayed players have their SR converge back to their MMR is simple, fast, efficient, and let’s everyone involved play balanced matches. No rigged games required. It works even when someone has decayed over 1000 SR.

Why would Blizzard make a second system where players are put into rigged matches, just to correct small deviations between SR and MMR, when they all ready have a better system in place? Why would they choose stacked games, instead of using the simple SR bonuses that decayed players get?

Here’s the thing. After reading what you said, I don’t believe, that you’ve invalidated MHz’s hypothesis.

I don’t know when this was written chronologically in relation to Overwatch patches, but in regards to your question “why they need a second system?”, I would have to say, that they already do have a have two systems in place!
One for Diamond and above without the personal performance factor, and one for Plat and below with it.

So while this might have seemed very intelligent way to counter him, doesn’t seem as much nowadays.

why would they secretly implement this conspiracy theory system?

First, I don’t believe Blizzard necessarily have to have done this on purpose and be malicious about it. Might be a bug.
But Second, if they were to do something, they do have the incentives to do so.

However, here is the thing, that really bothers me when it comes to Overwatch:

  1. In other games, the matchmaking is defined in a way, that the other players (opponents and/or allies) must be of similar or equal skill. In Overwatch I don’t see these words being mentioned.
    Instead, what you get here in Overwatch is “matches that we think you have 50% chance to win”.
    Overwatch Forums
  2. The MMR is invisible for some reason, unlike in other games, including Blizzard’s World of WarCraft.
  3. Activision Blizzard was granted a patent for what essentially is a rigged matchmaking:

https://archive.is/N8wdV#selection-1175.0-1181.46

https://www.youtube.com/watch?v=6wZQZQDNk40

What it basically does is, that it matches noobs with better payers. The idea is to expose the noob to an awesome players in an attempt to trick the noob to try and emulate the awesome players. And by emulate they don’t mean for the noob to learn how to play from the better player. By emulate what they mean is for the noob to purchase the same in-game item (weapon, skin or whatever) that the awesome player is using.

Now, while such a system does not necessarily have application when it comes to Overwatch, and is more about a Warframe or an MMO type of a game, one would have to wonder if they have something in place in Overwatch, that would incentivise people to spend more money on the game.

Players complain about unwinnable matches. Some might be perceived, but I would argue, that some are very much real. People complain about getting 4 support mains on a team. How is that winnable match?
And yes, you could argue, that players should master multiple roles, but the reality is, that when pro players are not subjected to that requirement, the regular players shouldn’t be either.

Blizzard haven’t done anything to alleviate the situation. They haven’t provided a role queue, which is one of the most demanded features. They’ve given LFG, that pretty much nobody uses. They don’t have the incentive to alleviate the situation. They are literally making money from it.

At some point, if you assume Blizzard will maliciously make a more complicated system just to screw with players, there’s no evidence possible to collect to disprove your theory.

Unfortunately, after the company did pattent a rigged matchmakign system, it would be very hard to convince people, that they are not willing to do something like that.

1092

Do you read those posts to see why they mention 50% winchance?

You need to read the part about grouping. You can have 12 people in a single match have the same MMR, but 6 solo-queue players vs a premade 6-stack with all 12 players in the game having the same MMR isn’t going to be seen as a 50/50, unless you’re ok with that.

From: Overwatch Forums

Groups are a big challenge in our matchmaking system. You can group with people of wildly varying skill and ping and we allow you to. It’s pretty unlikely that there is another group in the queue that exactly mirrors the unique circumstances that you have set up (pings, skills etc.). We want you to group. We feel that it’s the best way to play the game. So we try to avoid things that discourage grouping and we want to continually improve the social systems so that you’ll find it easier and easier to group with people you have chosen to play with. Playing with people you choose to play with is going to be more reliably fun than playing with people we choose for you. I once used the analogy of hanging out with people on a Saturday night. If you were to go out with five of your friends it would probably be a better time than if we tried to find 5 random people for you to go out with, no matter how smart we were in our selection process…

They are NOT systematically looking to pair “good/hot/winning players” with “bad/cold/losing players” like you, Cuthbert, S23, etc. keep trying to push.

1093

When the pros end up playing on the same ladder the rest of us are subject to, they too subject themselves to sometimes having to play an off-role. When the pros are playing proper Overwatch, it’s premade team vs premade team only, thus the equivalent for the rest of the general population would be clan-vs-clan only. As Seagull mentioned when he made a couple videos a few months ago, Overwatch wasn’t even supposed to have solo-queue, it was supposed to be premades-vs-premades only, but that’s logistically impossible for many people.

1094

I did see those videos, they were quite interesting.

This, appart from the ping part, is nonsense.
They could have a dedicated Team League mode, where the rating and the MMR belongs to the team, not the individuals, and where the system is handling two units – the two teams, instead the twelve individuals (6 on each team).
There was no BS like that back in WoW. You could form a team and play with no BS limitations. I don’t know why they are even trying to reinvent the wheel.

1095

That’s why I mentioned clan vs clan. No solo-queue. But for people that can’t dedicate themselves to being in a clan, then what? They can’t play the game at all?

1096

Wow it is so painful watching you cling to words without understanding them.

Zero sum would mean no SR gained or lost across all players for PBSR. Simply put if player A gains 4, thus player B must must lose 4.

This is proven false by character patches, such as Mercy. Players were gaining way more PBSR each game. Overall, they were adding SR to the system, while all other changes stayed the same. This is the exact opposite of a zero sum.

So, expand on this intentionally (who am I kidding, we know you can’t grasp this fully) and you could redistribute people through the SR field as you like through PBSR manipulation.

If your sole claim is “they don’t do it now” then you both don’t understand how it works and how it could be adjusted to do even more.

1097

Yes, and your 6 stack vs 6 solo ques is in their equation and off balance.

The quotes are given to ascertain that yes: The match maker does not grab 12 random people in your SR/MMR and throw them in a game where ever they may land in the match sides and the match maker does its algorithm to make a 50/50 game. Therefore, the conclusion is that it manufactures a game as close to 50% as it can, with players within the range of ranks it has determined to use.

We can also add that the math behind SR and MMR gains are not the same.

Jeff Kaplan - "MMR works very similarly to SR. There are some minor differences that make it feel worse though, when you just watch that number. For example, it’s possible to win a match and not gain any MMR. We make it so that if you win a match, you always gain SR – even if it’s just a little bit – to feel psychologically rewarding. "

Thus can conclude that within a given SR, there will be people either above or below the SR in the MMR.

1098

And we’ve seen where those SR gains happen. If you ever watched GM/Pro streams during Seasons 1-4, back when they complained about getting Diamond players in their games. You’d see them win a game and go up only 2 SR (Seasons 2-4 when the scales changed from SR1-5000), yet lose 30+ upon a loss because they’re getting so close to 5000.

The only other time you see screwed up SR gains of 2 is when you leave a game after the safe period begins after one of your teammates leave first. That is a bug.

The middle ranks DON’T get screwed by this unless, say, an SR 2000 player voluntarily groups up with an SR 1000 player and loses the game. The SR 1000 player loses way less than the SR 2000 player.

Rethink about the posts where you actually picked up the quotes. In what context are they trying to tell you that they’re trying to intentionally pair good players with bad players? Think about the “evidence” that Cuthbert intentionally uses out of context, exactly what you are also doing right now. Now ask yourself, why would they piss off the majority of the playerbase by doing what you claim they’re doing?

Also consider this, do you think people on this forum in general are good at evaluating themselves and evaluating their peers?

1099

I consider it not to “piss off” people but to create a wider gap from your rank to have 50% win chance matches. Though there are people that are not happy with how things are at this time.

You see, I am not here to defend “Oh I am better than what I am at.” where as I am here stating they have said these things and conclude, thus. I clearly state within this thread, I am not a great player. I am, according to the spread I hover under the median of the players

You discount the quotes and trying to state that they are being used incorrectly, We are stating hey are not just “intentionally pairing good player with bad” what is being stated is that if you are an outlier, you may either be the potato or the hard carry in a match, based on where you land on the spectrum.

It is not to say that ALL matches are swung with large gaps in skill variance or that ALL matches after a streak (up or down) fall into this, it is that, with enough iterations, it could occur. The intent is to make a balanced match at 50/50 after all.

Neither side knows all of the algorithms or statistical data the MMR truly used, nor do anyone outside of Blizz knows what the Matchmaker looks at, overall, to calculate a fair match.

Some of us are just looking at the information and concluding that these things can happen, and, with a large enough pool of games, does happen.

1100

If anything, you’re the only one here I misunderstood when it comes to how I thought you interpreted those quotes, and that’s my fault and I apologize. However, S23 has straight-up, and not in very nice terms, exclaimed that he thinks it’s intentional. Cuthbert has as well, albeit not in such an angry manner as S23. And personally, I get kinda heated about it because I IRL hate the idea of people thinking of themselves as superior to other people without proving it, and not simply fall short and blame everything else but themselves. Hell, I hate the idea of people not simply just letting their play do the talking.

Variance is the result of the human condition. However, it’s really up to you to believe or not believe when Jeff says the following:

SR closely chases your MMR up and down and is a more “digestible” number. With the exception of top players who have decayed, MMR and SR are closely linked.

which I interpret to mean that someone’s SR is an accurate representation of their MMR with any real difference being negligible. Pretty much every single Blizzard statement ever made on matchmaking is compiled in Kaawumba’s topic, and it’s a very extensive list.