Algorithmic Handicapping (MMR) is Wrong for Overwatch

Competitive Discussion
1017

We’ve been told the matchmaker creates matches in a way that each team has roughly a 50% chance of winning. I think we’ve had proof this isn’t true right in front of us the whole time.

Assume:

  1. MMR is 100% accurate measure of player skill
  2. Matchmaker creates games where each team has a 50/50 chance of winning - this is where handicapping can be involved
  3. MMR may or may not change after each match
  4. SR does change after each match (assume no draws)
  5. MMR and SR may or may not be equal
  6. One of the matchmaker’s goals is to make MMR and SR equal.

If #1 and #2 are true, then all matches will have the teams evenly matched (50/50 chance of winning). Therefore, in any given sample, each person will have a 50% win rate. This means no one would climb or fall. They would stay about the same SR.

This also means that if they play mostly comp, then the game will adjust their MMR as their skills improve. Thus, no matter how much they gradually improve, they will always be in a 50/50 match. They will not climb or fall.

The only way to circumvent this would be to improve play outside of comp (QP, custom games, arcade). So, when you return to comp, your skills will be higher than your MMR would suggest.

We know this is not true. People climb and fall while playing mainly comp.
Therefore, either the MMR and/or the matchmaker don’t work like we’ve been told.

MMR/SR Mismatch
The matchmaker actually generates streaks.
As people play matches, it becomes more likely their MMR will not equal their SR. This can come from wins where the person played poorly, or losses where they played well. Once this mismatch reaches a certain threshold, then the matchmaker creates games to push SR towards MMR.

If MMR higher than SR, then the player is placed in games where they’re likely to win.
If MMR is lower than SR, then the player is placed in games they’re likely to lose.

This seems to agree with what we observe instead of every match is a 50/50 as stated by the developers. This is all theory and conjecture. However, without further information, we can only create theories which match our observations.

2 Likes
1018

I want to point out that @Kaawumba has inadvertently admitted to handicapping in his reddit post here:

specifically this part:

Before each prospective match, estimated match quality is calculated. Matches have higher quality if the mus of the two teams are close, and the sigmas are small compared to beta. Essentially it is statement of how confident the system is that the match will be fair. It will not be fair if the mus are very different. It may not be fair if the sigmas are high (because we don’t really know the mus well).

Clearly, by this persons own claims, the matchmaker does not just grab 12 people of similar MMR and place them into a game. It additionally looks at things like “inconsistency”, here’s the quote:

Higher inconsistency means that a player is less reliable in their results. This parameter helps calculate the winner of each match.

How could the matchmaker possibly measure the consistency of a person? Really think about it…there are two ways:

  1. By tracking performance through individual metrics, things like elims, damage output, ults used effectively etc.
  2. By seeing if the person has been winning consistently, losing consistently, or winning and losing at an even rate.

Monitoring one or both of these metrics could create handicapping.

To put it all together: The match will only launch if the two teams appear to be even and a persons consistency is used to calculate the winner of each match.

So let’s say you are consistently winning matches, you are on a win streak and now the matchmaker is going to create your next match. It sees that you are very consistent recently and it will use this information to determine if the teams are fair (as per the quote above). In order for the expected win rate of each team to be at around 50%, it can either place you with 11 other people who are consistently winning matches, or it can place you with a mix of people who are inconsistently and consistently winning matches in order to balance it out**.

A realistic scenario would be this: Players 1,2,3 on your team are consistently winning (probably good), while players 4,5,6 are inconsistently winning (probably not as good).

The teams would be fair because there would be people who are inconsistently and consistently winning matches on both teams, but here’s the problem: the people who are inconsistently winning matches on both teams should probably be ranking down compared to the people who are consistently winning matches who should probably be ranking up, however in this matchmaking because they are split evenly on two teams, there will be at least one group of people who are inconsistently winning matches that will win and rank up, while at least one group of people who are consistently winning matches will lose and rank down. That’s not right, or fair.

This would easily explain why after winning several matches in a row, people are then placed with potatoes. The inconsistent will naturally get placed with the consistent in order to create balanced games. In reality, it’s very likely that the inconsistent players should actually be consistently losing, but because of team balancing they are placed into games where they have a chance to win and will stay at ranks they don’t belong!

If this is the way the matchmaker works, what the programmers don’t realize is that the inconsistent and consistent metrics for the players is actually created as a bi-product of the matchmaker algorithm itself and therefore cannot be used as a variable to calculate new matches, because it is like a circular reference!!

4 Likes
1019

Sorry dude… Those are parameters for his simulation - not for the matchmaker.

1 Like
1020

Yes of course, I’m saying even by his own calculations there would be handicapping.

2 Likes
1021

No. The simulation takes people of similar mu (analogous to MMR) and places them in a match. More specifically, see the “Description of the Simulation”, steps 4, 5, and 6.

The simulation does not measure the inconsistency of players. Inconsistency (and skill) are parameters that are fixed for each player at the beginning of the simulation. They do not change for the duration of the simulation.

You misread the quote. Skill and inconsistency of the players (and a random dice roll) are used to determine who actually won the match. It’s a simulation. They are not used to predict who will win the match.

These misunderstandings render the rest of your statement incorrect as well as your other statements depend on them.

1 Like
1022

This is exact, i made an experiment with a couple new accounts.
I played well and at the best of my chances, in one account intial placement went extremely well, 9 win-1 lose, placed around 3300. Won other 3-4 matches and went 3400, at that point my winrates were so good that mm probably thought I was kinda smurfing (and I wasn’t cuz i play between 3300 and 3700), I started losing almost every match till 3150 with embarassing average teammates, some of which i found impossible to be considered even low diamonds. At that point my winrates were around 50% and i started climbing normally.

In the other account things went the exact oppisite, placemente went worse. Won 6 lose 4, but my stats were probably considered cause I’ve been placed 3100.
I won almost every match and went near my skill gap (3600) again until winrates were again around 50%. Again the situation normalized.

I noticed this so clear because new accounts have less data , while probably you have to win/lose a lot more in a row before “triggering streaks” when the accounts is filled with more data.

I don’t think this stop good players to reach their deserved level, i just think how sad is the quality of the competitive and how can a potato inquinate the ladder by being helped by this system while usingin heroes like mercy moira and brigitte.

It’s kinda insulting.

2 Likes
1023

Your simulation does additional things rather than just get people of similar MMR and place them into a match, it looks at the consistency of the players. A good matchmaker should not look at this whatsoever because it can’t quantify that data. Why would you even include such a variable? It creates inaccuracies.

Of course the simulation can’t measure inconsistency because there are no actual people playing each match, but the actual real matchmaker would measure consistency because real people would be playing, and I went on to describe how it would do that.

I believe you suffer from dunning-kreuger effect where you believe you are better at statistical analysis than you actually are. To the admins who are going to look at this because no doubt kaawumba will report me for abusive chat (another avoidance tactic), kaawumba has accused multiple people on this board of suffering from dunning-kreuger effect as well.

1024

Dude. I understand my simulation better than you do.

You haven’t said anything bannable in these two posts. Being wrong isn’t bannable.

2 Likes
1025

Yes!! But the spelling remains to be determined. “Arti, Artie, Arty?” We’ll figure it out. Thank you very much :slight_smile:

What is this simulation we’re talking about? Are the details in another thread?

1026

S23 linked it above (but did not understand it). Here it is again: https://www.reddit.com/r/OverwatchUniversity/comments/aatezy/why_match_quality_is_frequently_poor/.

1027

Okay right off the bat:

“Trying to measure individual players’ skill through solo-queue 6v6 matches is a really hard problem.”

Is it?

Competitive Overwatch is not solo queue. Competitive Overwatch allows groups to queue for matches, and it creates games by mixing groups of different sizes together with solo-queue players. This is an important point, because it’s the very reason why Blizzard created Match Making Rating in the first place.

If competitive play was solo queue only, there would be no justification for handicapping/match balancing. Players could rise and fall in rank depending on their ability to win matches, with no interference from Match Making Rating. And that’s how I think it should be.

1 Like
1028

I just don’t understand why Blizzard made it this complicated. I personally hate the idea of W/L determining a players skill. But if they are so dead set on that style of comp, then it literally should be +25 SR if you win, -25 SR if you lose. Don’t try to find even matches, Find 12 people with in 100 SR of each other and throw them in a game. Don’t let win/loss streaks determine who plays with who. If you are in gold you play with 11 other gold players and see what happens. It literally is this simple, and does not need to be made way more complicated by MMR like it is. Now someone is going to tell me how this wouldn’t work and needs MMR blah blah blah.

4 Likes
1029

You’re completely right! Don’t listen to the player haters and the nonsense that Blizzard gives us on this subject.

1 Like
1030

But that’s what the system does right now, I don’t think there would be any difference, as “SR closely follows MMR”. That means that if you’d be matched by SR, not MMR, nothing would change I guess.

1031

It tries to find a 50% match. It takes players who are winning a lot and gives harder opponents, players losing a lot, better teammates. None of this should be the case. None of this should be taken into consideration. If everyone is truly in the same rank, then any combination of 12 Plat, or gold players should be a 50-50 chance to win. Players shouldn’t get helped because they are losing, or hurt because they are winning. All this algorithm, and MMR, and PBSR and all this hidden factors they use to determine our “skill” and who plays with who is totally unnecessary.

3 Likes
1032

Oh. So you read the first sentence. Great. That’s totally enough to make an informed opinion of the post. (sarcasm warning)

1033

It only arranges 12 player who are close to each other. When you win a lot, you get harder opponents, that’s simple. How are players hurt when they win or helped when they lose? PBSR can help you lose less when you performed perfectly in a lost match. And that’s not the system helping, it’s how you played over the whole match. Also PBSR isn’t a big factor, biggest factor is if you win or lose, how you managed to end the match.

I either missed point of this thread or you are just whinning into nowhere because of nonexisting problem.

1034

IT SHOULD BE TOTALLY RANDOM WITHIN YOUR RANK. Gold is 2000-2499. So any game i play it should pick 11 other players within that rank and create the game.

MMR helps or hurts players

“If a player’s MMR is wrong and too low, then the odds to win will be greater than 50%, and the player will win more games than he loses, which will cause his MMR to rise over many games played. He will then be placed with stronger and stronger opponents (and stronger and stronger allies) until his MMR is correct, and his win percentage approaches 50% (with some random oscillation around 50%)”

None of this should be a thing. If you win, you gain SR if you lose, you lose Sr, rinse and repeat.

1036

When you lose too much, you get matches with weaker opponents, so it’s expected that you are going to more.

That’s what it does right now.

1037

Matchmaker does find a 50/50 match SOLELY by putting people with the same number together (in the case of a game of all 12 solo-queue players), and RANDOMLY arranging them into two teams. It does not systematically try to put good/hot/winstreak players with bad/cold/lossstreak players or any of that handicapping nonsense. Obviously, the existence of premade groups complicates the matchmaking, but that has nothing to do with what people perceive as intentional matchmaking trickery.

1 Like