Competitive matchmaking's MMR system is VERY good

First, if your win-rate hovers around 50% you shouldn’t expect to change ranks, right? Even if you just put 2 random people together, in any game, if they both won about half the time you would say that they were evenly matched. One is not better than the other. If you are not better than your competition, you should not expect to increase in rank.

That’s not what you’re asking though. What you’re really asking is “If the MM can produce a 50/50 match, then how can it expect anyone to advance even if their skill is higher than their current rating would indicate.”

This is actually a really simple problem to solve.

The MM, as good as it is, assumes in ALL instances that it is wrong. Even if you were perfectly placed, if you win you gain SR. Why is that? Why would you gain if you are perfectly placed even 1 SR?

It’s because the win result shows that it is, in fact, wrong.

When the MM makes the match all it has as information is the MMR value. We tend to think that that MMR value is what the MM thinks we belong, but that’s not the right way to look at it.

That MMR value is a guess. Educated guess, but a guess nonetheless.

That guess needs to be validated. How do you validate this guess? You pit players against each other to find an actual result. So the MM makes a guess. Puts you in a match (it really doesn’t matter against who, I’ll get to that) and takes the result to see if the guess is valid…not CORRECT, VALID.

So, if you (or me) were to be put in a game against NYXL that game would indicate that we were not as good as NYXL. If we kept repeating that match we would be even more confident that we were not as good as NYXL. Each and every drumming by NYXL would validate that, yes, we are not as good as NYXL.

Of course, in that example it would be pitting a team where you estimated had a skill of around 1600 with a team where you estimated had a skill around 4500. The 4500 winning against the 1600 does validate your guess, that 4500 is better than 1600, but that’s not really saying much.

What you want to do is pit two players that you think are the same skill against each other to show who is better. That way, you find out where you are wrong and in which direction.

If you “think” two teams have an equal chance of winning but the final result is that Team A is better than Team B, then your next guess will be that Team A will win again. You no longer think that the result is 50%. You think that it’s 60%, or 51%, or 80%, depending on how you set up the system.

If that same set plays again you can gain information, but that’s not how OW is set up.

OW is set up so that when you VALIDATE that result, when you WIN that estimated 50% match, it updates what it thinks of you. Now, it thinks you’re better than 1600. Maybe it now thinks you’re 1650, maybe 1700, but in our system for most people it appears that it guesses next around 1625.

So instead of getting put against a 1600 team, you are now put against a 1625…STILL A 50% CHANCE TO WIN, JUST AGAINST A BETTER OPPONENT THIS TIME. If you lose, you go back to 1600. If you win you go up to 1650. The cycle continues, giving you 50% matches, until you can’t win anymore. Of course, your win-rate will be above 50% while you’re climbing, but if it weren’t why would you expect to climb. Each validation shows more and more that it’s about right.

Now, say you lost a game that you were correctly ranked at. It happens, in fact it would happen about half the time. You lose again, against a (slightly) worse opponent…you lose again, worse opponent. It happens, because now maybe you have a 60% chance to win. Lose, lose, lose, lose, lose…ASSUMING NO BAD ACTORS this is perfectly plausible…down to the point where you basically have 100% chance to win. Like, 1 v 6 for your entire team. IF YOU TRULY BELONG AT THE RANK YOU WERE…your chances of winning will be increased and you will see a higher win rate up to the point that you are correctly ranked and you are at 50% again. THE MM THINKS IT’S MAKING 50% MATCHES, but you know better, these games are easy and these poor souls don’t stand a chance. Remember, that’s fine, the MMR always assumes that it is wrong.

Now, you’re probably thinking 2 things. 1) This is not what I experience. According to this, I should win all the way up to my rank, then WLWLWLWLWL around my true rank. 2) This is not what they are saying, they are saying that they are happy that people have a 50% win percentage, not simply that they put equal skill sets together.

1_ If you truly are only 25 SR below your skill, you won’t win that game 100% of the time, of course. Your EXPECTED result is 50%, but if we simulated that game over and over indefinitely your ACTUAL result would be, say, 51% BECAUSE THE GUESS IS INCORRECT. You can’t win 51% of a game, but you sure can lose 49 out of every 100 games played. The actual order of wins and losses in that scenario won’t necessarily be WLWLWLWLWL, either. You will get some streaks in there because streaks happen, just flip a coin 100 times to prove it to yourself.

2_ Say you wanted to study some natural phenomenon to see how it worked. One way you could test your idea of how it worked is to create a model. We (humans) do this all the time. In fact, any “theory” in the scientific sense is a model. E=mc^2 is a model. The algorithm that determines who will win a game of Overwatch is a model. You can study the phenomenon, think about how it probably works, build a model that expresses what you believe is true.

How would you validate that model? How would you know it’s correct? You put data into the model and see if the model matches the actual result.

So when we initiate nuclear fission, we take the known mass and take the released energy and see if E does in fact equal mass times the speed of light squared.

It’s the same with the OW MM algorithm. They take the players, guess their skill, and need SOME WAY to validate if their guess was correct.

If they guessed two people were the same skill and they tested it to see if their guess was correct what do you think the result would be?

50%. If their model of how good players are were correct, they would expect a 50% win-rate. They find a 50% win rate, indicating that their model is correct.

That’s part of the reason this “forced 50%” stuff is annoying.

They want a 50% win-rate to validate if their system works, not for some nefarious purpose. Testing mathematical models isn’t something the average person does everyday, so I get the confusion, but this is a process of guess, collect data, adjust, guess, collect data, adjust…not “Lhun is getting a bit above his rate, lets put a leaver on his team.”

This is how the MM can be good (overall 50% win-rate) and also allow people to climb (games are individual validations of SR).

If you want to discuss PBSR, that’s fine. I won’t participate because I really have no strong feelings either way on the issue. I can understand both sides, I don’t believe it’s a significant effect, but I’m not very certain of that belief at all.

Yep. The way they handle groups sucks. It’s really an impossible problem to solve. First, the above system works best when everyone is of the exact same skill. That’s not really realistic but as long as it’s close it’s fine. But you get 1000 SR differences in there? No. It’s that scenario where we were playing against the NYXL, you’re just proving what you already know.

Second, there is a bonus (apparently) for grouping to account for the “synergistic” effects of being in a group. Synergistic effects that don’t apply in the same quantities over all groups.

In short, they know groups are screwey. It’s one of the hotter topics out of their own mouths. Unless you have a good idea on how to stop groups from screwing with the MM there isn’t much else to be said.

Maybe I don’t understand what you’re saying, because the only people that will have “nobody to beat” would be the top 1…not 1%, not 100…top 1. Even they have to constantly be beating their competitors.

6 Likes