Why Handicapping (MMR) is Wrong for Competitive Play

This back and forth talk is all nonsense. They said explicitly that there is a complex system that tries to balance games.

It’s a nice idea for QP, but moving on…

This isn’t really doable in a game where you have tons of completely different heroes and different roles. Some guy may be good with a Tracer and suck with a Mercy.

You have people queuing who only know how to play some characters and then are forced into characters they don’t know how to play or are stubborn and end up with comps that won’t work. Then the balanced game is all of a sudden not balanced anymore.

Trying to balance games is pointless, unless you queue with a hero beforehand and then the MM can go and produce teams based off of that hero’s stats.

So it leads to frustration a lot of times. The solution may be some sort of hero or role queue or just plain win lose and random matchmaking. At least you know for certain you have to ride the variance.

1 Like

I left this out, for simplicity. But since you brought it up: I agree that a predicted win percentage of 50% does not mean an actual win percentage of 50% even if all the MMRs are accurate. Boiling thousands of variables down to one MMR is horribly reductive. Just because teams are in theory equal does not mean it turns out that way in practice. Aside from the fairly obvious issue of wanting a balanced team, hero wise, and well suited to the map, sometimes teams just don’t work well together. You could imagine an example where a team has no leaders, or two leaders who refuse to work together.

It’s reasons like these that make a statement like predicted 50% win percentage for a single match not particularly valid. It is only really valid over very many games where the players are constantly being mixed (or you only match teams, not individuals). And by that time, people’s skills have changed, so it is always a moving target.

2 Likes

So, there it is, I believe they should scrap the whole nonsense of saying they track your progress since beta and stating they have MMR and PBSR and just go ahead with a simple and transparent system of wins and losses, fixed points and decay.

1 Like

What they really should do is to re-evaluate all players like very 10 seasons or so. Because I am sure that most people should have some kind of improvement if not a big leap of improvement. Like, When I started at level 25, I screwed up my SR by playing Genji and now, I play Zarya, Dva, Moira, Ana, basically I flex and do way better than only playing Genji.

I feel like I deserve a re-evaluate on my SR placement than always just staying in Gold when a Junkrat main of level 25 lands on Platinum because he just aimed and holded m1.

so what your saying is that everyone the game grabs for a match generally is close enough in mmr that it doesn’t matter who they are paired with?

In a nutshell, it’s reasonable to believe a system that attempts to make balanced matches will produce them more often than one that doesn’t care about the balance within a match.

That said, we have been given some insight on this by the Blue’s, and I think Kaawumba did a good job summarizing/citing the issue. So, I’ll just quote that.

https://us.forums.blizzard.com/en/overwatch/t/how-competitive-skill-rating-works-season-9/14009

1 Like

I feel as though everyone whose posted for both sides of the argument have had good points, but very few seem to grasp what the real issue is in comp. The issue is very simple. MMR should NOT be a factor at all in competitive mode. SR should be the sole factor in deciding teams. Balancing comp mode off of MMR is bad for everyone.

Imagine a platinum player getting unlucky in a string of games, let’s say they get game after game of leavers, throwers, feeders, etc on their team that results in a long losing streak, which is easy to imagine, since it happens to nearly everyone at some point. Now, imagine that plat player drops down to 1400 SR after two days of terrible luck. They’re legitimately a plat player, so their MMR still reflects that. What does this mean for how the matchmaker bakances games? Well, it’s easy. The game will take all those 1400 players, look at their MMR, and see this plat player who is better than the others. His MMR is high. To balance the overall team MMR, it places that plat level player with 5 below average MMR players to go against 6 above average MMR players. This equals out to a balanced overall MMR for each side. In theory, this makes the game balanced, but in practice, it turns into a stomp. That stomp will be against the plat level player’s team, unless he hard carries his 5 teammates, all of whom will be performing badly compared to the enemy team. If the high MMR player does not hard carry, he loses even more, rather than easily gaining rank back to plat, since he isn’t facing a normal bronze level team with normal bronze level teammates at his side. The game makes him hard carry in order to get a win. Thus repeats itself game after game after game, and it usually creates the huge grind and the 50-50 win/loss ratio everyone complains about.

Why is this bad? Well, first of all, a player who should easily climb in normal circumstances is now given a far more difficult path to get back to his/her baseline rank. They are forced to hard carry 5 potatoes over and over and over again to get back to plat, because they’re hindered by MMR being the mitigating factor in the comp matchmaking process. It’s unfair to them, all in the name of ‘balance.’ It’s unfair to his enemies, because the game forces them to play against a plat level player in a bronze level game. It’s unfair also because his teammates will be carried by him if he does manage to pull out a win. They essentially have an easier time to climb, even though they have lower skill, while the higher skilled player has a more difficult task.

If MMR was not used in competitive matchmaking, the plat level player would quickly win all his/her games in bronze/silver/gold, because they would be facing off against average bronze/silver/gold players, with the same calibur teammates. They would easily be able to tip the scales, and in effect, they’d bump back up to platinum very quickly. After that, they’d have a more difficult time ranking up, because they’d only be able to do so by improving to a diamond level player at that juncture. Luck or bad luck streaks would be quickly mitigated this way.

The issue is that the game doesn’t do this. Instead, it sees the high MMR of that player and faces them off against tougher competition, with lower skilled players on their side to balance the difference. In effect, it gives that player have a very tough grind to get back to the level they belong in. It’s simply unfair and against what a competitive mode should actually stand for. Why have SR at all if MMR defines how the game matchmakes?

The reason people still eventually rank up over a very long period of time is due to the fact that at bronze, there may not be as many players with the same level of skill as a plat player who deranked due to bad luck. Even though the game tips the scales against them, the matchmaker may not be able to find enough other plat level players who also deranked to place on the enemy team. Even though that high MMR player will be playing every single game against other above average MMR players for that bracket, they still may only be above average for their rank. It’s a large enough factor over time to tip the scales ever so slightly to where many games will be hard carried for wins, which otherwise were created to be more balanced with MMR averages.

What it does in every case is force a far tougher grind on the better players in each bracket to rank up to where they belong. It also gives lessor players an easier grind to rank up. It’s an unfair system that isn’t representative of what a pure competitive ladder should look like. Players stuck in lower brackets than they belong should naturally have an easier time to rank up, but blizzard decided against that in favor of balanced matches over short term volatility. While games would be stomps early on in tge season without mmr balancing, they would naturally level out as each player got to the bracket they truly deserved. Blizzard decided that the short term volatility would be a bigger negative on player perception than the unfairness that MMR balancing creates in the long term. The big issue is that the majority of players have seen how the system punishes good play, so the overall backlash is just as great, if not more so, than what would have been with a system that balanced games solely on SR.

3 Likes

Generally yes.

However, things can get weird with groups, or high/low ranks,or unpopular hours and platforms.

2 Likes

No. Their MMR falls as well, down to bronze.

“SR closely chases your MMR up and down and is a more “digestible” number. With the exception of top players who have decayed, MMR and SR are closely linked.” – Jeff Kaplan
https://us.battle.net/forums/en/overwatch/topic/20758686566#post-6

2 Likes

I think we need go no further than people don’t always get to play the same hero, so their individual skill varies a lot.

That alone, negates any attempt of a matchmaker, no matter how complex or simple, to be able to produce balanced matches unless you queue heroes before looking for a match.

1 Like

I can’t decide whether I agree or disagree with the above statement but I’m leaning towards disagree.

If the game has placed you in a scenario where you have a 50/50 chance of winning then you would not be able to tip scales in your favor since you can’t play beyond your current maximum skill. The match outcome would come down to things out of your control e.g. someone throwing, someone disconnecting, someone making a poor decision that they would not make in 95% of their games etc., basically someone deviating from their normal skillset/behavior.

1 Like

Except your MMR isn’t based upon your max. It’s based upon your average (in theory). So anyone that performs above average would tip it one direction, and below average would tip it the other.

Now, if MMR is materially inaccurate… well then everything s an unbalanced pile of bad. :stuck_out_tongue_closed_eyes:

3 Likes

MMR does not move at the same rate as SR. It’s far less volatile. This can easy be proven as well. Don’t disagree just to be contrarian.

Show me this proof. Keep in mind that it has to be stronger than an explicit statement of Jeff’s.

1 Like

Yes, the part that SR is not used for match making and the team average SR is usually at the same level as your SR, hits a bit this theory. But for me there is still something which doesn’t add up. I’m observing my own situation, where I got from mid/low diamond (been there for 3 seasons) to low gold and I’m struggling to climb properly. Subjectively viewed I have the same difficulty to get a win out of it in both tiers (3200 and 2200).

I watched a lot of stats of all the players in one match, looked them up on overbuff and compared them to each other. I tried to group the players in skill categories. The average result: For 3 good players the team had 3 statistically bad players. Same for the enemy team. It looked like the team were balanced. But also the gap between the good and bad players were huge. In one example both dps were playing mainly dps: One of them had an elim / death ratio of 3.3 the other one of 0.7 . All other stats were the same showing a big gap between those players.

I also used a deep learning platform which tries to predict your skill. I tried to determine the skills of the players in a match. It showed the same result:
predicted skill of team 1 (average of all skills in the team): 2500
predicted skill of team 2 (average of all skills in the team): 2560
best player predicted skill: 2880
worst player predicted skill: 2002
both players were on the same team

I have to add that it is not easy to find out the exact skill level of a player only with the given data but it shows a bit the direction. I also hope somebody else tries this to see if this effect appears also to other players.

But this observation does not disprove the theory of this thread. More the opposite…

3 Likes

Well that seems perfectly logical for a matchmaker on a 50/50 quest.

Not for a balanced ranking system though. The players should be more approximated in theory, all 6 of them.

One game isn’t really enough to draw much in the way of conclusions. Twenty would be the minimum, for an obvious effect. For more subtle effects, more games are needed. Everything you are looking for should be written down.

Do you mean c0derwatch? Be warned that it is not super accurate. Last season it said that I belong in plat. This season it says that I belong in silver. Now, my statistics are worse this season (I messed with my crosshair and took some trips and my aim is still recovering), but I’m solidly in gold both seasons. c0derwatch should be seen more as a statistics summarizer than an accurate ranker.

What I expect is that if you take data for many games, you will find that it is random where the overplaced or underplaced players are. Sometimes they will all be on one team. Sometimes they will be on the other team. Sometimes they will be split. There won’t be any correlation between how you do in one game and who you are placed with in the next.

This is consistent the system as Blizzard has described it.

1 Like

Yes, I did this for more than one game. (Sorry I wrote it a bit misleading)
I think it was about 10 games. The process involves a bit of work to find every account out.

Yes it was c0dewatch and I used it only as a second source once. I know that c0dewatch is not the best predictable source. It’s the same for the data we get from overbuff. It’s too less information to clearly predict a skill of player. That is why I wrote the lines after that. But it was a bit funny that it showed the same result.

I did that. And for me it looked like, that the player between the teams were more or less balanced. I tried to focus on the overall stats (most played heroes) and not on the stats of the hero they played that game.

It is not enough data to use this as a prove. I would be happy if more people would try this. Or better, if we could somehow gather a database of a big number of games with the additional information: who played with and against who.

I’m not a “natural” English speaker but I think you misunderstood me. I’m not talking about using the algorithm but creating/implementing it. Like not the part when you feed the player informations to the algo to determine their rank (and here I agree, once it’s done, it’s done and you can use it the same way 1 or 1,000,000 times the algo itself won’t change) but the part about how you will determine their rank and what player informations you actually will need.

We’re thinking about it a bit like trying to determine a physic’s law. We want to know the relation between a “player skill” and a set of variables, the “performances in game”. But the problem here is that where an object gravity is related to its mass weather we like it or not, in the case of skill rating it’s up to “us” (or at least Blizzard) to pick if a certain information is relevant (or not) for it.

It’s this process of choosing which variables are relevant and how (much) that I call complex, the “making-off” of MMR if you permit the phrasing :confused: So maybe I did not express myself correctly. :confounded:

The core principe of Elo is that your score goes up when you win statistically more than you should and down when you win less than you statistically less than you should.

Note that the 50/50 chances are for the team and not the player. And the a real problem is just that : we are looking for individual rating when performing as a team. We want an accurate depiction of our own skill when it’s actually not possible to strictly cut where your performance begins and stops. It’s not like a relay race where even if you loose, we can still say you were the best runner based on your individual time.

About the 50/50 ? Well, I think I understand were it comes from ? Not sure about it but :

MMR is probably a range more than one number (with all these MM’s confidence in your MMR). The more certain the system is about your merit the smaller the range is. It’s more accurate the more you play, the goal being to place you at your skill rank. Aka where you have 50% chance of winning against same skilled players.

The game already suppose it calculated the right MMR for you and makes you play games against supposedly same skilled players (or at least your MMR range overlaps). Then the game think you should have won 45/55 games on a total of 100.

But your teams actually won 70 games. The one common factor that is certain about all these teams is you. It is supposed that these better performance were the result of you being better than calculated. You brought these number up. So the system declares that your skill is actually better than it thoughts and your MMR goes up too.

It’s probably not that simple but I think it’s a possible logic behind this.

About my personal opinion about the rest:

  • I don’t like SR. It’s a lolly given to placate players with instant gratification about their win. I’m a bit skeptical about its pertinence but I do understand that nobody want to play 100 placement games before receiving a more accurate number.
  • I find the “mixed” ratings a bit weird ? OW is my first FPS so I’m not sure about that but my SR doesn’t really means the same thing if I compare it to a player with the same heroes/function (main heal) and someone who is a DPS for exemple. I have one alt-account where I play off-tank (Zarya and Dva mainly) and it’s actually a bit lower ranked than this one.
  • I like playing with a wide range of ppl and skill and it’s the reason why I play more QP than comp. If ppl would take QP a bit more seriously, and by that I mean just doing the effort to play to win, I think both QP and comp could be more pleasant.
  • Winning or loosing is the easiest way to know if you are better (or worse) than someone and I have difficulties to come out with another main way to compare ppl in a game. I like the fact that individual hero performance impact on your ranking too. I’d also like players who do good hero changes to be recompensed for it.

I hope I could explain my thoughts properly, have a nice day :smiley:

2 Likes

If you feel that winning a 3200 match is as hard as winning a 2200 match then you 100% do not belong in 3200. If you actually deserved to be 3200 you would totally eradicate 2200 players, trust me. It means you just got lucky and ended up in diamond, then proceeded to get to the rank where you actually belong, nothing wrong with that, just you deserving to be at 2200

2 Likes