Algorithmic Handicapping (MMR) is Wrong for Overwatch

Are you going to call people pointing out how wrong you are disinfo agents next? I’m not posting here because I care about defending Blizzard; I doubt anyone is posting here because they think Blizzard needs advocates against uninformed forum threads. You’re getting these long posts explaining things because people are annoyed at how misinformed, biased, and plain wrong you are. You’re complaining about features of a typical competitive ladder, and therefore basically complaining about EVERY competitive system.

It’s really no surprise that your complaints and theories in this thread could be found scattered across forums in every competitive game in threads posted by people who are very obviously suffering from Dunning–Kruger syndrome; whether you gripe about teammates that are too bad, enemies that are too strong, a system that is pointlessly malicious, or desiring basically unbalanced matches entirely in the mistaken belief it will benefit you, it always boils down to thinking that you’re really more skilled than your current rank indicates.

Just to give a food for thought, you do realize these statements are the reason we are having this debate at all.

Precisely, you should have a large effect on the game.

1 Like

Those stats say nothing because its my smurf and I’m not even trying to rank up anymore but if you wanna know why I am mad its because most oft those 38 games were lost at ~1500 SR and boy were those games unwinable. Its not even the elo its this stupid MMR system that gives you those impossible games when ever you are playing too good or give 100%.

1 Like

I understand that you want to discredit the arguments, but ad homonym does not help the message or the argument that this thread is trying to convey.

It doesn’t matter if they are paid. Arguments are made, points are disputed. Attacking the people does nothing for either side.

I will not dispute that Dunning-Kruger does indeed effect some arguments. But, to blanket it across everything, like you seem to want to do, again, like above, does not help your argument.

1 Like

I feel safe estimating it is the root of MOST of the arguments.

Orrot, I’m not sure if you saw my deleted post earlier, but I didn’t mean to send it. I basically butt-posted it…story for another time.

Perhaps the phrase “outsized effect” would have been more accurate. You shouldn’t expect to have more effect on your games than your teammates. If you think otherwise go play a game that’s not team based, please, as you’re probably a selfish teammate anyway.

Having a outsized effect on the game and having teammates and opponents that are just as skilled as you are mutually exclusive concepts. You can’t carry a game where you are well matched, only where you are mismatched.

I understand quite well that it’s often the case that what people say they want, what they actually want, and what is actually possible can be totally different things. The way people tend to describe how they want the system to work is basically how it actually works.

Most of the complaints are about an imaginary system, one that would never work and can only exist in your imagination, one that can only be believed by those who claim they can’t climb but pass off those who have climbed as a lying, boosted shills.

This is why so many of these posts devolve into the ad hominem mess that you see. Inevitably, at some point, those angry at the imaginary system realize that system can only exist if large amounts of people are deceiving them. Once people move into conspiracy theory territory, though, I don’t waste my time trying to educate them. There’s no coming back from that toxic thought loop.

Really, you just need to look at people who have climbed to know that this whole thread is based on an idea that doesn’t match reality. If the system is really how Cuth describes it, that you are ranked based on wins and losses but accurately held to a 50% win ratio, no one would ever climb (or fall, actually.)

Not that there aren’t problems, but it’s really hard to accurately rank people. Some natural problems and variations are going to arise. Some games will be stomps. Some games your teammates really will be idiots. You will have bad days. People sometimes will be bad actors, etc.

People need to realize that this MMR/SR system works very well in the aggregate but it’s not a promise to make every game perfect every time. When we get data, it shows what we expect, that your win rate will increase the lower in SR you go and decrease the higher you go. Not become easy, but easier. 50% to 50.1% if you are slightly underranked. In short, people’s expectation is completely unrealistic. Look at that Meliodas above, complaining about a 65% winrate that 35% of the games were unwinnable. Yep, and? I think more realistic expectations are in order here.

It’s also not real clear what 1SR means, especially in gold. It’s quite possible that people from, say, 2100-2400 are essentially indistinguishable in terms of skill. I discussed this in more detail here: How Competitive Skill Rating Works (Season 11) - #56 by OzoneOOO-1681

I do think Blizzard has kinda focused on the wrong parts of the explanation. Using engineer and scientist language to describe this stuff but then not being consistent with it. But that brings me to my next point:

I think maybe you underestimate the prevalence of illusory superiority. It is accurate to say that we ALL suffer from it. In fact, if you have answered anything other than “I have no idea” or “My test score was X” to any question about how much you know about anything, you probably did the same. No worries, we ALL do it, even those that always know better, like me :rofl:

Seriously though, when confronted with an objective determination of skill most people either accept it or try to improve. A small minority reject that the test is truly objective. It may not be Dunning-Kruger, necessarily, there are a couple forms of illusory superiority bias. It really is true that around 90% of people say that they are above average in various studies on the matter.

So far no one has debated me on whether a skill based vs win-rate based system is better. I don’t think it would be much of a debate, as a win-rate based system isn’t really possible, but I guess someone could try to convince me otherwise.

This is an education on how what people say they want is what they already actually have (or sometimes how what they want is unrealistic).

I’ll explain how a skill based rating system works all day. I truly understand how the concept is foreign to people. I’ll explain how a win-rate based system cannot work in this instance as well, as many times as need (actually already have, post 241, again…big post).

People want to be matched with people of their skill level and for their skill level to be reflected in their SR. Well, that’s how it is my friends.

Back to your earlier post that I missed, sorry again:

First, I’m going to define some words. You may not agree with my definition but it’s what I’m using for discussion.
TrueMMR= Your ACTUAL ability at the moment you are playing a game. This can change from game to game and even within a game, though some people erroneously believe it to be stable over periods as long as a month. Over longer periods of time it’s generally expected to increase unless neglected.

MMR=The games guess as to what your TrueSR actually is. In theory, two persons whose MMR=TrueMMR will have equal chances of winning any given 2 person game if matched based on their MMR.

TeamMMR= The games guess as to how well a team will perform. Though any set of MMR can comprise a team, it is expected that teams having the same TeamMMR will have equal chances of winning any given match.

misranked (Over/under)=A situation where your MMR =/= your TrueMMR. Over if TrueMMR<MMR, under if TrueMMR>MMR.

Sort of. Going from a 50% chance to win to a 51% chance to win is “easier” but again that’s in the aggregate. Each individual game may not actually be easier. I think if more people understood this they may be less likely to tilt, which essentially lowers your TrueMMR.

This is a common misunderstanding. The 50% condition only applies if MMR=TrueMMR and TeamMMR is accurate (I’ll get to this in a second). That is, if you are misranked, the 50% condition isn’t the case…which you acknowledge in the sentence above, actually. If your TrueMMR stays the same but you drop in the ladder the games should be “easier”, meaning that you don’t actually have a 50% condition, right?

Now, some will say that the system is SUPER accurate and that it somehow really, truly KNOWS your TrueMMR and matches not on MMR, but TrueMMR because it’s so gosh dern good (in fact, if you take the OP and swap MMR for TrueMMR and SR for MMR, it actually makes a lot more sense). This is where my bit about the imaginary system not matching reality. If the system somehow knows your TrueMMR (first, how?) why wouldn’t it just keep putting you in your TrueMMR matches? If it did that, you would win exactly 50% of the time. If you did that, you wouldn’t rise or fall at all.

You rise and fall and notice games get easier and harder (see: Smurfs), therefore it doesn’t match on TrueMMR. Besides, that’s silly. It can’t predict the future.

Alternatively, some people seem to believe that all that is true but Blizzard is evil and manipulating your results, that is, it does match on TrueMMR but manipulates it so you play more games. That doesn’t explain why games get easier and harder when you go up and down in SR, though, which is why they then resort to people who claim otherwise to be part of the conspiracy.

I think you have it backwards. Despite peoples’ vehement claims to the contrary I think it’s far more likely that tilting causes a decrease in TrueMMR which causes you to be overranked. No doubt it’s a self repeating cycle. Now, I’m not saying that’s the ONLY reason people are misranked, in fact everyone is always misranked for a large variety of reasons. It’s just that people think that they should win 100% of the games where they have a 51% chance to win and that unreasonable position can cause the tilt which leads to even more misranking.

Welcome to TeamMMR. Look, does it really matter if you have in internal 4999 SR difference in one of the teams as long as TeamMMR is equal between the two teams?

No.

This is what drives me nuts about the whole handicapping argument. Cuth says it’s all super accurate and all, but when it comes to putting a team against another one suddenly you’re “carrying” the lower player. How do you know that the other team isn’t in the same position as you? What does it matter, actually, as long as the TeamMMR is the same, that you have a 50% chance to win?

It doesn’t.

People come up with crazy unrealistic sets of SR on a team as examples that it’s not fair, but then go on to say that each team has a 50% chance to win.

Look, if the TeamMMR matches the composition of the teams is completely irrelevant, yet people still act like a 50% chance to win is somehow different if you’re the highest MMR player in the match. Each person STILL has to play in a way that their TrueMMR meets or exceeds their MMR.

Who cares? You already have, as part of your premise, indicated that the match is even. The TeamMMRs are the same. You’re in the exact same position as if it were two evenly matched people.

See? You did it here. If the match is even then offsetting the lower end person is baked into that cake. It’s not a problem, by definition, because the match is even. You can say that it’s unlikely that the match is actually even, but then you no longer think that the match is handicapped. You’re not “carrying” that person anymore than if you were on a team where all the MMR were the same.

There are two responses to this.
First that having large MMR ranges in a team may make for an even match, but not very fun. This is a fair argument and why I think the 1000SR range for groups is extreme, but that argument can be made without resorting to “handicapping”, having unfun but fair matches doesn’t affect your ranking, it’s a QoL issue.

Second, as you said, some think that it’s more about whether people know that they are on teams that are evenly matched and of different MMRs, but if that were the case the proper response would simply be education, not calls for an abolishment of the entire system.

Now, if someone is misranked, the team with the misranked person is more likely to win/lose than is estimated, but that’s how it all is supposed to work, keeping in mind that it’s all in the aggregate. Each individual match isn’t all that important. It may FEEL bad to lose, but if you let that affect you you will end up actually misranked.

In fact, smurfs thrive on creating a condition where their TrueMMR is far in excess of their MMR. Throw games, lower MMR but not TrueMMR. Play games where you are overranked and games are easy. Miserable for the opposing team, but easy for the smurf.

If you want easy games, though, don’t smurf, just play a different game. This game is hard and it should be, that’s what makes it fun.

If the system can determine your skill sufficiently to put ENTIRE TEAMS into 50/50 matches, it’s doing a GREAT job of determining individual skill. Once your skill is determined your skill can be ranked from highest to lowest. If your skill is accurately ranked from highest to lowest, it’s perfectly acceptable for competitive play. That’s why it’s called Skill Rating.

If you want to claim that it doesn’t do a good job of making 50/50 matches and therefore your ranking isn’t accurate, that’s fine, but that’s the OPPOSITE of Cuthbert’s argument.

The truth is that’s it’s good at determining skill under conditions that aren’t always the case but are impossible for the system to account for (playing drunk, for instance). We’re all a little misranked and that’s perfectly fine.

1 Like

Which part of MMR just being SR without decay do you not understand?

I did, even had a couple of side bits quoted, but then saw you removed it so I decided to wait. :slight_smile:

There are forces behind the scenes that, in all fairness, does feel like this is true. But, it is not true that the system gives you a 50% win ratio. feeling and observing bias, does come into it.

My, personal, stance is that its not impossible to climb. It allows you to do so, but the adherence to the 50% match ups, do add that one to 5 wins to climb up when you start becoming the outlier.

I am also a part of that. :imp: It happens, and as you said its ingrained. But my point is that there are some good points, and not all of it is hot garbage.

The only time you should win 100% in OW ladder is if you have a 6 stack of smurfs in bronze…

Seriously though, that took a bit of time and energy to write. Its good to see the logic behind things, and you have given me a bit to chew on.

The Human Condition of being total sinkers is, shockingly, outside of a lot of formulae in skill rating systems /s

This makes smurfs, throwers, and leavers, all the more problematic to an otherwise stable environment.

The outliers will impact the median. If the matches are even, and one side has an outlier and the other does not, the non outlier team has members that are closer to the mean of the two groups.

Unless the MMR groups people in matches by a set range, then scrambles them up and then determines the outcome, giving the % deviation from 50%… throwing out anything with a >60% match up in the “Lobby” to re establish from the pool…hmm… That’s a thought pickle.

I play OW because it is fun, because I like to challenge and push. I also have a tendency to obsess. OW is it at the moment lol.

“Its not a tumor!” -Kindergarten Cop.

2 Likes

To help with your pickle, don’t think of it as outliers and median. Don’t even think about ranges of MMR/SR.

Imagine a data set of 6 SR values. The individual numbers can range from 1-5000*. There are 5000^6 possible combinations. Add to that an additional set of numbers of the same properties giving you (5000^6)^(5000^6) possible sets of that 6x2 array. That’s 1.5625e+22^1.5625e+22, btw. Google won’t do that math. It’s a lot of possibilities.

*I’m not a programmer or that advanced in math, btw, the numbers may not be right, notation is almost certainly wrong, and I realize that the scale may not actually go from 1-5000.

There exists some members of that 6x2 array of SR values where a game of the first set against the second set will have a completely uncertain outcome.

We know by definition that arrays consisting of all the same number will meet this criteria. [1,1,1,1,1,1 x 1,1,1,1,1,1], etc. Edit: We also know matching pairs will meet this criteria, e.g. [1, 1000, 2000, 3000, 4000, 5000 x 1, 1000, 2000, 3000, 4000, 5000].

What we don’t know prima facie is what other sets also will also result in a 50% chance of winning. People talk of averages or medians or whatever…no.

The only way to know this would experimentally.

If only there were some way to run this experiment… :sunglasses:

Maybe build a multiplayer game, build the teams, and look at the data? Before you freak out that Blizzard has been experimenting on us remember that these systems have been in existence for a long time. One MS Trueskill patent was in 2005. https://patents.google.com/patent/US7050868B1/en

I don’t know what 6*2 arrays end up having an exactly 50% win rate. But whatever it is, the experimental precision of that likely far exceeds the precision of our own ability to play consistently as long as a certain range of values isn’t exceeded. Edit: I mean as long as it’s not something stupid like a GM and 5 low bronzes against 6 silvers…not that that mix would necessarily be unbalanced but that such extremes would be rare occurrences that would be extreme extrapolations of the data to estimate.

Btw, a related patent is called “Determining relative skills of players”.
https://patents.google.com/patent/WO2007094909A1/en
Sound familiar? I’m not just making this all up. The idea that the formula behind MMR is “hidden” or “unknowable” or “secret” is flat out incorrect.

Just play the game… All these speculation nonsense.
If you’re stuck in bronze then you are a bronze player.
Same goes for the rest of the ladder.

Don’t blame the system. Blame yourself.

2 Likes

You guys can argue all day, but it’s simple.

Dominate a couple of games, and a switch flips and gives you the bottom players for teammates.

Blizzard must think this is creating a fair match. Obviously, stick the TOP recent player with the WORST recent player. If you’re a duo and you BOTH dominate a few games… then you earn the TWO worst players.

In fact. I remember Jeff Kaplan saying this in a video, but it must have been removed or deleted or something. Others have said the same. He literally said they team winning players with losing players in order to try and make a good match. I wish someone could dig it up.

So, you do really well and feel punished for it when the system gives you bad teammates on purpose.

I feel that’s wrong. It should be truly random. If the 4 best players randomly get placed on the same team… Good. It’s random. They stomp. They rank up. You probably won’t see them again, because they game will actually let them rank up instead of artificially force them to stay the same rank.

Instead, they’re trying to make every match a 50% chance to win. How? By punishing the best players, and rewarding the worst players.

This makes it harder for people to find their actual ranks.
The real way to create fair matches is to let the random, non-handicapped matches just happen… and people will find their ranks. Resulting in naturally fair matches.

All that said, i personally don’t think they should even allow random players to affect your SR. It should be 100% personal. You should play. Get what YOU deserve. Save the Team SR for pre-made teams, tournaments and stuff.

1 Like

Yes exactly like this.

And I also feel like this is happening on full 6 stacks as well. You are in a good win streak and suddenly get absolutely owned by the other team. Maybe 1-2 smurfs there and voilá your win streak is done. After that the lose streak just continues.

Please Blizzard stop making these 50/50 win chances and let it be truly random based on your SR (and maybe the hidden MMR) only.

1 Like

This post definitely makes the top 10 list when it comes to being misleading, misinformed and/or stupid in this thread - and I’ve read most of them.

Not going to comment much other than if someone actually reads the above post and think that there might be some good points - at least don’t believe that random matchmaking would be a good idea.

I am going to put this as plainly as I can - random match making is objectively not only sub optimal but even a disastrous way of creating games!

The notion that the best people would climb in a random system is false on every level and I even went to the lengths of proving how idiotic this would be by running a statistical simulation and posting the results earlier in this thread. So please if anyone want to argue this they need to actually prove it mathematically or at least via a simulation. Alternatively point out any technical errors I made in my simulation and then we can talk. However even common sense would be enough to realize that the best people will not be the ones climbing in a random system.

No, Jeff Kaplan has never said that. Only you, and people with an attitude like yours, have said this. Your entire posting history between the new forums and the old forums displays it.

It already does that. There’s a reason there’s so many complaints of without smurfs being involved.

1 Like

The thought pickle comes in the form of, what if we are approaching the 50/50 rule set as a defining measure. We, those that think of the handicapping, have approached the idea with a “The system finds 12 players, then sets them into equal number pairs to create a 50% (or as close) win condition on both sides.”

Now, think of the multitudes of MMR/SR in your math equation. Place the win condition to the “Start” condition and not the “Group Match” condition.

The new approach is that it is a random set variable. The System looks for groups, then arbitrarily sets them into two groups. From that point the system would “evaluate” their set MMR number and determine a win %.

At this point the system sees that it is within tolerances (40-60%). Then starts the lobby. When the lobby is set, the system can mix and arrange the 12 people in multiple games. To the system, you are all relative in skill and as long as the tolerance is maintained, you get a new match.

This parse way of the division might be a reason for:

Long Que times: you are the outlier in the set of SR it has tried put you in. Thus whenever it has a game for you it is beyond the Tolerance and thus continues to look.

Getting Potatoes/Being the Potato: its random, but by random, you get wildly differing skill MMR within a match, as long as it is, again in tolerance.

The feeling of Not being able to Climb: Since the random jumbling of players, you are being put with and against people that fall within a tolerance. That variable may stretch, depending on time/available players, to the point that you are now saddled in multiple lower spectrum matches ~40-45%. Because it is random. and As random does, strings of duplication can and will occur.

This is only speculative, and a pickle. But this is why I like discussions, new ideas can form.

… Sorry for the Tangent.

1 Like

There is a good reason you’re bronze and the matchmaking or the game has nothing to do with it.

1 Like

The first 3 paragraphs follow from the remaining, but ultimately are the most important to what I believe your true concern is so I moved them to the top:

The only way to make sure your TrueMMR>MMR is to make a focused attempt to improve at the game.

I suspect your objection is that you want a fighting chance and you feel like this “50%” thing is unfair. That the result of your game shouldn’t be a “coin flip”. That But anything more or less than 50% is what’s unfair, man. Every loss is someone’s gain, every head that comes up someone’s tail is down. It’s not even that you don’t deserve a more than 50% chance to win it’s that it’s not mathematically possible in the aggregate.

Look, the 50% thing is an estimate. Not a promise. Using the terms I defined earlier everyone’s TrueMMR must >= MMR in order for TeamMMR to be accurate. EVEN THEN…as our ancestors used to say, you need to be “Favored by the Gods”. Assuming no smurfs, everyone at your MMR belongs there. They may be misranked, but we’re all misranked. Getting the potato is random because there is no way for the system to know the TrueMMR. It can only work from the MMR.

On to your ostensible post:

So you’re saying to build pools of people where any combination would have a win condition of between 40-60% and then randomly build teams from that pool, right?

What you describe isn’t all that different from what we have. When a person goes into the lobby there IS a range of MMR values that are within 40-60% (or whatev) of THEM, where MMR:MMR(40,60). Teams get built from the closest MMR up (cuz that’s the easiest way, remember that the only prima facie even matchups are where all have the same MMR).

You can’t really find two teams within a certain win condition of each other until you have one team, i.e. to find TeamMMR:TeamMMR(40,60) you need the first team built already. But I think you would just build both teams randomly from the MMR(40,60) set.

All you’re saying is to eliminate the extra step that makes it go from a random 40% match (or worse, if it’s random you could get all the 40% on one side and 60% on the other) to a fun 50% match.

If you’re going to build teams, why build ones on the edge of acceptable win conditions ON PURPOSE? I get that sometimes it has to be done simply to get a match made, but the default should be 50%. You don’t seem to believe this, but I can’t understand why?

Cuthbert says it corrupts the ranking system, but I’ve shown that it doesn’t, he just doesn’t understand how the ranking works. That’s the crux of his argument, that you can’t rank people using even matches. That’s factually incorrect, it can be done, you’ve read the patent of how it works, I’ve explained it thoroughly (I hope), so what’s the problem now?

Once you realize even matches don’t inherently misrank you, you’re left with QoL of games. Do you really think being in a match where you only have a 40% estimated chance to win is better than a 50% estimated chance?

ANY set of teams could be ranked using the skill rating based system, but the further away from 50% matches you get the less accurate it is AND the worse the games are. You learn very little, but you do learn something, from a 1%-99% matchup. The further you get from 50% the less you learn, that is, the worse the ranking system works. The only alternative is a win-rate based system which I go over in post 241. (Have you read post 241? You should read 241, please confirm that you have.) TL;DR is that it’s simply impossible.

I agree but be careful. I’ve been silenced for far less. It’s…annoying…considering how many ToS violations are already in this thread. You can defame people as liars and shills but dog forbid you hurt someones feewings.

I didn’t see this but would really like to. I’ll look, but you could probably find it in the 538 posts faster than me so I would appreciate the guidance…

1 Like

I re-read your post and think I may have misunderstood it.

The part I quoted, what you say is how you think of handicapping, is pretty accurate. Not the “equal number pairs” part, but even if that were the case it would just be an extra unnecessary step.

The rest, if it’s meant as description and not prescription (which is how I took it) is nonsense.

Sorry bud.

Edit: Typo

I understand what you’re saying and I usually keep very low profile when it comes to commenting on the quality of other posts but sometimes it’s just so blatant how certain posters are just getting to stir things up and put fuel on the fire so to say.

My post was actually in the previous thread I noticed and I’ll dig it up for you tomorrow when I’m not on my phone.

Edit: Why Handicapping (MMR) is Wrong for Competitive Play - #127 by Patman-2341

2 Likes