Algorithmic Handicapping (MMR) is Wrong for Overwatch

Glad to see this is still going after all these months. I quit a while ago. Recently, i thought i’d come back to see if anything has changed.

I quickly saw that you’re teamed with the worst players around if you dare get all golds.

Terrible system.

Almost doesn’t matter because they’ve completely ruined the game with all these no skill heroes anyway.

It used to be fun, because they fights were intense. Now, you’re batted around or stunned half the time, or being killed by a random orb flying around with no rhyme or reason what-so-ever.

2 Likes

As I’ve said before, the way most people describe how they WANT it to work is also how it ACTUALLY works.

First, lets throw out the “hidden” aspect of it. How it works is completely irrelevant to whether you can see it. In other words, they could show us the MMR and do nothing else and it would function exactly the same in terms of ranking. I also ignore this aspect of it because there’s a lot of people on both sides of the debate that agree that having it hidden ranges from unnecessary to counter-productive. It’s not really a point of strong debate and it frequently pops up as a side show in the discussion regarding MMR and SR. See post #260 in this thread.

On to your point; I went over this before around this post (#276) but I’ll gladly expand here: Algorithmic Handicapping (MMR) is Wrong for Overwatch - #304 by OzoneOOO-1681

In a sense, you’re right that matching on SR would be efficient. That’s not the mistake anyone is making here. The mistake is thinking that MMR doesn’t work like you see SR work.

If you haven’t already, read this post (#28): Competitive matchmaking's MMR system is VERY good - #28 by OzoneOOO-1681

I think the trouble some people get into when thinking about this is that they see their SR float up and down and incorrectly think it always goes up a certain amount on a win and a certain amount on a loss. While it is true in practice, that’s not actually the underlying mechanics of it.

It LOOKS that way because you’re almost always matched against opponents that are about as good as you.

But SR isn’t an approximation of your winrate. It’s a description of your MMR (how accurate of a description is irrelevant to my point). That’s why I keep telling Cuthbert that it’s simply factually incorrect to say that SR depends on wins and losses. If your SR was 2000 but you somehow got placed into a series of games at 4500 SR, it wouldn’t really matter how many you lost, you would hardly drop any SR at all, if any. That’s how ALL Elo systems work. Your losses are irrelevant to the system, or at least secondary to WHO the game is played against.

I UNDERSTAND WHY PEOPLE THINK THIS. It’s by far the most common way of ranking teams in sports. If the NY Yankees have a 50% win rate, they’re a solidly average team. How can 2 people in OW have a 50% win rate but have two different rankings? I get it, I really do.

Sports teams play the same number of games and always play games against each other. It’s the same with OWL. Every team plays every other team, is what I mean. The winrate here is an absolute measurement. Someone could win all the games. Anything less is relative to the max number of wins, which is a set, known number.

In OW, or really any multiplayer game, you don’t have the benefit of a set number of games nor the clarity of playing every other player. We’re not going to play a million games each season. There isn’t a max number of wins to compare yourself against. You can’t use an absolute system. You MUST use a relative system.

So you need a DIFFERENT way of ranking people other than the usual win-rate method.

The different way is to rank people based on who they are winning and losing against. Your CURRENT rank is higher than the player you just beat but lower than the player you just lost to. If you have exactly a 50% win rate then your rank will be accurate. If you have less or more than that then your rank is unknown. The further from 50% you are, the less accurate your rank is.

I get that this is wildly different from how people normally think of win rates. Normally, win rate IS the rank in some sense. But that can’t work in a game like this. There are too many variables.

If you matched on SR it would be the same as matching on MMR. Exactly the same. Your SR would be (and is) where your maintain a 50% win rate. The only difference between that system and the current on is the ability to provide punishment, decay, and whatever “feel good” that they may or may not still add (which I’m against, for the record).

Some people (and I believe this includes Cuthbert) think that teams should be randomly assigned regardless of skill and your win-rate (SR, in Cuthbert’s understanding) then becomes your rank.

This would rank people, that’s for sure. I’ll let you be the judge of a system where having a loss rate or a win rate higher than 60% was normal and where you could play 10 games for a 90% win rate and be ranked as high as Jjonak.

I now feel the need to say it every time, so I remind people here that there are several things that could be improved in the system, but you ARE matched based on your rank and always have been, having MMR and balanced matches isn’t what’s broken.

Here's an example I removed from the main text

If you maintain a 50% win rate at 2000 SR, then your SR should be 2000, right? You shouldn’t expect more or less than that. If you maintain a 50% win rate at 4000 SR, then that’s where you belong. What doesn’t happen is that you maintain a winrate of 50% at 4000 SR but have 2000SR. Just watch a bunch of streams or your own games. The average SR of both teams is nearly always +/- 200 SR of yours and generally very close to each other. In fact, before LFG it was generally within 50SR both ways when they had a higher selection of solo individuals to match together.

1 Like

Yes, in elo, you can gain points by losing more than you win, depending on who you play.

Yes, it goes up or down an amount determined by: How far your SR is in comparison to your MMR, How much of an advantage/disadvantage from 50%, and somewhat how much better/worse you did vs other people with your hero.

Then why have both? They can still instill a decay system, where you see the decay as a separate number. The decay from 3000 is to encourage continuous play. Negative DKP minus for not playing.

True, pure winrates is not the answer. This is totally true. But, you have a ladder where in you drop, it should, in theory, be easier to climb back up to the rank you were. Using the 50% condition its not. Its still as hard to climb as it was to drop (especially on the fringes of your MMR to SR) and just as easy to drop further. This is where people tilt.

What Cuthbert is against is if you drop 100 SR from 2100, and get put into a game. You are now at 2000 with a MMR between 2100 and 2000. So it puts you in a game, you are, at times, likely to have a match within 200SR. (using that as a widest margin of SR differential). WIth this, he is stating that you will get a possible 1800 SR (within 200 of 2000) person in your match, and that person is automatically on your team.

So now, you have to do your best and offset the lower end person, because the system has made a match even. But because the game really uses MMR within a SR range, we do not see how far above/below the actual number that was used to make it even.

Yes. I do not believe, other than decay, that anyone will be more than 500 away from their MMR. only because of the SR restriction, you will drop enough that your MMR will be higher than the average MMR in that SR Range.

1 Like

I think ranking up/down would probably take more time, not less, under an impartial matchmaking system.

But for all this talk about win rates not being the best gauge of player skill, the SR system is essentially based on win/loss. Players do not move up in the SR system without winning. They do not move down without losing. All the more reason why the outcome of matches is important. And that is why it’s a problem when we face an uneven playing field, rigged by the workings of big data (MMR) that we do not understand.

When matches are handicapped with MMR, we cannot trust the win/lose result of our matches as proof of our skill. That is why the SR system doesn’t work. As Kapoof commented recently:

So yeah, well put. Getting gold medals in one match makes you lightening rod for MMR to strike in the next, by putting you on a team with the worst 5 out of 11 players available. That is what typically happens in solo queue, and in group play the matchmaking system has other ways of tilting the scales.

1 Like

Look, if a GM and I both have a 50% win ratio then that should be proof enough for you that the system doesn’t use win rates as a gauge of player skill.

I’m not even saying that it’s the best way, but rather the only possible way. WIthout locking in a strict number of played games AND having everyone play against everyone else, using win rates couldn’t work.

In other words, just because it goes up and down on a win or loss doesn’t mean that movement or location is based on a win/loss ratio. Almost everyone has a near 50% win ratio, but we are all different actual skills.

Also, again, it only moves on a win or loss because we’re within a skill bracket where we have a 50% chance to win. If the games allowed brackets into the 90th percentile there’s a good chance you wouldn’t see any movement after the demoralizing spawn camp.

Come on now, Cuth. If you’re going to start claiming that rather than the MMR mechanism doesn’t create accurate results (which you claimed in this very post) but rather that the system picks on people to give them bad players to make you lose, I’m taking my conspiracy marbles and going home.

You’ve rejected this notion before, my friend:

That’s the trouble with you. You like to pull every sycophantic tangent into your orbit as anecdotal proof that you’re onto something.

Be consistent, man. Here you’re saying “handicapping” is only PBSR (“into effect below diamond” implies that it’s equal to PBSR):

Here you’re equating it with balancing matches:

Here you’re DENYING that balanced and handicapping are the equal:

Here you are seemingly understanding how MMR works (if not how it’s used to rank people):

(I actually agree with a solo-only Comp mode, btw. Groups of disparate skill are bad for competitive play because they break MMR, not because they use MMR. MMR would be needed for solo-only mode as well.)

And here you are arguing your best argument:

With your history the answer you should have given to Kapoof is more like “The game doesn’t give you worse teammates as a response to an individual win, it handicaps all your matches so that if you’re the best in the group one or more teammates will be the worst. This is wrong for competitive play.”

Some of these kids look to you to provide answers to their poor experiences. I’m not upset that you don’t understand win-rate rating vs. skill rating. I’ll discuss it with you all day until you understand how it works and then maybe we can discuss implementation details that would make it work BETTER.

But please don’t use your platform to tell these kids that the system is rigged against them in particular. You clearly don’t believe that so have the guts to tell your sycophants that they are misunderstanding you.

I know it’s hard but you’ll feel better in the end.

1 Like

I realized during my last answer that I never responded to your questions in this post. Honestly, I probably just read the first bit and facepalmed so hard I couldn’t see the rest.

Despite you’re little discrepancy there with your message, the rest of the post is fair so I’ll answer your questions. Sorry for the delay.

Almost all competitive environments have some provision for playing vs. people of your relative skill level. You don’t see Little League players vs. MLB players, for example.

Now, could the best college level football team beat the worst pro level one? I really don’t know, but you could call it “unfair” to not allow them to compete at that level. In effect, that’s what I see you as doing here. It’s true that to say “the Cleveland Browns are the worst football team” is unfair because there’s the local PeeWee league that is FAR worse than they are. Hardly anyone would really think that’s “unfair” though.

Furthermore, especially in pro sports, there’s all sorts of “handicapping” that occurs to keep the competition fair. The worst team gets first pick in the draft, for instance. Race cars have strict weight requirements and design requirements where even if you are able to increase power or reduce weight, you’re not allowed. Provisions to make the game or sport “interesting” are the rule, not the exception.

On a different note, the whole idea of drug testing in sports is “handicapping” in the sense that they want everyone to be on a level playing field. They don’t want the use of performance enhancing drugs to be a “prerequisite” for competition even though some of those drugs are completely safe (caffeine, for instance). For someone willing to take the drugs, this is handicapping their performance. https://www.quora.com/Why-isnt-caffeine-considered-a-performance-enhancing-drug

Now, your OP makes great effort to describe handicapping or balance in a way that implies the audience is always the one that’s the best out of the bunch. Surely you are aware that that would only happen 1/6 of the time in reality. If there are potatoes on your team when you’re the best, then occasionally you will be the potato on someone’s team.

Furthermore, you imply, but do not directly state, that the opposing team will all be on your level rather than the average level of your team. To be fair, you do directly state that for every skill level there will be a counterpart, but the tone is that you will struggle to “carry” where the reality is that at worst the “carrying” will be done against a team that is much worse than you. Not ideal, but onto my following point.

Lastly, you fail to mention that the default state is for everyone to have the same skill in the match. While it’s true that finding 12 people of the same skill is nigh impossible within a reasonable time frame, it’s not the case that they intentionally put widely disparate skill sets together as a matter of course.

So my summary of my response to your first question is this: Playing against equal competition is perfectly normal. Putting worse players on the same team as better players isn’t the default mode. When it does happen it doesn’t benefit the weaker players any more or less than the stronger ones in that match but rather gives the equal competition that is normal for sport.

I’ll hammer it home again, as often as you need.

To grade on wins and losses while giving you such strong competition that you can only win half the time would be so obviously broken that you should reconsider that maybe your understanding of it is incorrect.

It’s not designed in the dumbest way possible. You don’t place in some random SR and then stay there because MMR gives you a 50% win rate. A 5 year old could see this would be broken, rather than telling the multitudes of scientists and engineers that developed this type of system that they’re morons, maybe rethink how it works.

You learn more about where you stand in a ranking by comparing yourself to those closest to you in skill. Though I reject your notion of “big data manipulation” as being the case, it still stands that playing against people of varied and random skill ranges would do far less to tell you where you stand in a ranking than playing against someone who you believe to be your skill level. You don’t learn anything by throwing the local Little League team against the NY Yankees. That just tells you what you already know and isn’t pleasant for either team.

True competition is always balanced. To balance millions of players over billions of games you need the computing power to do it. Welcome to the future, my friend.

1 Like

Team sports analogies don’t hold up very well to the subject of handicapping in Overwatch. I quit playing Competitive because I saw my level of skill widely diverging from the skill level of the players around me.

Again, I really don’t see it that way.

The MMR system doesn’t find 12 people of equal skill, and throw them in a match together. It hand picks the best and worst players for either team in order to create an even match. There’s a big difference, and it causes inequitable treatment of players. New and relatively inexperienced players have a huge advantage in that system. Those are the players that MMR seeks to accommodate because Blizzard, like Facebook and any other software company, only cares about maximizing users’ engagement with their application. This is just another abuse of big data, and those abuses are rampant nowadays.

Those are the players who should be most concerned about the problem, so yeah the post is addressed mainly to them. And it’s more nuanced than some worry about one team or the other being assigned “the potato.” There are degrees of difference between players in many categories of skill. We know very little about how MMR discerns those differences and uses the information to form balanced (i.e., handicapped) matches. But we know that the system does so, and it is logically impossible for such a system to be treating us fairly.

Yeah, in Competitive Overwatch you are basically not allowed to outperform the players in your SR bracket, without being penalized. The MMR system uses your stats to engineer your games, giving you a 50% chance of winning wherever you happen to be in the SR system. That’s patently unfair. The only reason for it is to make the game more palatable for casual players.

War is a true form of competition, and it is rarely balanced.

By manufacturing matches to have 50% odds for either team, MMR makes Competitive Overwatch about something other than the individual players’ ability to win matches.

1 Like

I’ve did read all the dev posts. Its also pretty obvious if you play at high sr during off peak hours. The game just throws a bunch of diamonds with t500 to make the sr on each team even, thats it. Gain and loss of sr is always the same too except if you’re high enough to experience reduction that prevents getting 5000.

Its just kawumba’s opinion. I’ve seen all the dev posts and play at a high SR where during off peak hours its quite obvious that matchmaker just tosses people in to make teams have even sr, i.e. 3 diamonds and 3 t500. There is nothing else to it.

2 Likes
Seriously?

If your idea of competition is demonstrated not in the Olympics but rather in the horrors of the battlefield you know neither competition nor war, mate.

You don’t compete for your life or the life of your family. I have to assume you’ve never so much as been trained in the use of deadly force otherwise you wouldn’t make such a sickening analogy, especially after commenting that sports is a poor analogy.

If your idea of “fair” is to do what it takes to kill the person that’s trying to kill you we’re working from an entirely different set of assumptions.

Now that I’ve calmed down…

Quotes are out of order in relation to importance of discussion.

This is our point of contention here. I’ve done my best to show how this actually is “fair” (posts beginning at 249 in this thread, for those following).

Prima facie, I really don’t see how it wouldn’t be “fair” to play against a team of equal overall skill as yours. Honestly, that sounds to me the definition of “fair”. Please elaborate on this keeping in mind my discussions of win% rating vs. skill rating and how it would work regardless of the overall composition and distribution of the MMR inside each team. This is that thing that you just, you know, say without really explaining very well. Convince me.

I’ll give you that it makes it more palatable though. Not just the casual players, but more palatable nonetheless.

I agree that playing against random skill levels using win rate to rank you would also be “fair” in that everyone would be treated the same, not that they would have equal competition. Though that would be fair, it would be a bad experience.
Edit: I mean that overall the game would be fair, but each match would be horrendously unfair the vast majority of the time.

May I remind you of what someone in this thread once said:

What does it matter how the balance is created? We both agree that it’s accurate. In fact, it appears that you think it’s more accurate than even I do, given that you appear to think it’s inescapable. I disagree that we know very little, but again, agree to disagree here. How MMR works is immaterial to how well MMR works to balance matches and we both agree that it’s reasonable effective at doing so. Which brings me to my actual point:

Wait a minute.

We’ve already established that we both agree that MMR is accurate.

Are you saying here that you actually believe that there are, in fact, “[players that are] always the one that’s the best out of the bunch”? You’re intelligent enough to know that not everyone can be the highest MMR in all their games. That would be mathematically impossible. It’s reasonable to think that everyone has 1/6 chance of being the highest MMR person on the team given our mutual understanding that both a) handicapping exists and b) MMR is accurate.

However, here you seem to be saying that you seriously believe that there are people out there being persecuted by Blizzard directly. That there is some group of people that are always* the highest MMR player in every match.
*(or almost always, or disproportionally more likely)

I don’t know if this characterization is accurate, but if it is please answer the following questions to aid my understanding:

a What conditions do you believe occur that separate these people out from the rest? I mean, given that this cannot mathematically apply to everyone (5/6s of people have to be not the top 1/6th), who are these 1/6 of people that is your “audience” that are systematically the highest MMR player on their team?
b What happens, in your model, to the other 5/6 of the people? Does the system work just fine for them?
c Even if there was a person that was always the highest MMR on their team, wouldn’t the opposing team still overall be below that person’s MMR if the teams were balanced (we agree) and MMR was accurate (we agree)?
d Even in the case of a person who always happens to be the highest MMR player, how does affect the skill rating type ranking system as I described above at all? Don’t they still have equal opportunity to win given that MMR is effective at balancing matches?

In the end I don’t really want to debate this because it’s tangential to your point. That we have balanced matches isn’t really in dispute (if I’ve understood you correctly).

I’ll let the reader of these posts determine for themselves which fits the evidence more. You think there is disparate MMR in a group due to selective manipulation by Blizzard. I think it’s due to the physical impossibility of finding 12 players of the same MMR. I’ll agree to disagree.

On my side I present the evidence that MMR disparity varies far more widely at the highest and lowest skill brackets than it does in the common skill brackets and also varies more the more off-peak hours you are playing at. I can’t say what evidence you would present, not that you don’t have any, but that I haven’t seen it presented.

I quit playing Competitive because I saw my level of skill widely diverging from the skill level of the players around me.

The secret real reason for this thread: you can’t come to terms with the idea that you were no better than your teammates, or that your opponents were not stacked against you. It has to be the system’s fault.

2 Likes

The Devs (including Kaplan) has affirmed that MMR exists as well as SR, but you be you. Good day sir.

1 Like

We are never going to see eye to eye on this, because you’re insisting on a fallacy: that all of the players in any given match can/should be of equal skill. But players are individuals, and we have wide-ranging levels of skill.

Blizzard knows this. That is why they instituted the Match Making Rating system in the first place: to ascertain the degrees of difference in skill between players in the same SR bracket, and to handicap their matches accordingly. According to that system, relatively skilled and relatively unskilled players have no chance of being teamed up with their own kind. On a per match basis, they are intermingled to produce a 50% chance of winning for either team. All of that tinkering completely undermines the integrity of a ranking system that is based on players’ record of victory/defeat. It undermines the goal that we are all supposed to strive for.

You’re misinterpreting me, I didn’t say that war was “fair” or “good.” But war is a true (i.e., real) form of competition, which has shaped us as a species. It is inherently nasty, predatory and unfair. And frankly so are we. Even as we speak, the story of the human species is culminating in a mass extinction event of our own making. If you want some perspective on this, I suggest you study history or read some good historical fiction by Ken Follet or Bernard Cornwell.

The fact is that Overwatch, like all multiplayer shooters, is an homage to the ugliest aspect of human behaviour. Multiplayer shooters are far more analogous to war than they are to Olympic and pro sports, formula 1 racing or anything else you’ve posited.

Even in the world of sports, we are focused on rewarding only the most able people. Most sporting cultures are exclusive and derisive of people with lesser ability, and I’ve experienced that as a kid who was always about two thirds of a year younger than other kids in his grade growing up.

Video games like Overwatch should give people a fair chance of advancement according to their level of ability with a mouse and keyboard. Instead we find ourselves hamstrung – handicapped – by the will of a corporation (Blizzard) who seeks to keep us toiling away with their product, trying to reach a goal that they have made impossible for us to reach by their own unethical design choices. We are the proverbial donkeys plodding toward the carrot they dangle before us. And people like you, Ozone, encourage us to be ignorant of that fact.

1 Like

You had a gold emblem but 1700 SR. You USED that to assume that since you thought you were so much better than the 1700-2000 players that you played with, they must have been a lower MMR and thus, being a “handicap” to you. Your entire idea of handicapping this entire time was nothing more than yet another “I’m good so Blizzard gives me bad teammates” topic, and it’s really sad that so many players think like this, and I really wonder how many people in real life think like this as well.

That’s not true, but your willingness to make these things up speaks volumes of your character.

1 Like

https://www.overbuff.com/players/pc/Cuthbert-11649

This was your last known SR according to Overbuff, and you claim you haven’t even played for such a long time. You shouldn’t attack my character when your entire opening post shows your OWN character.

They should just implement WoW style MMR and SR. It’s the best around. Quickly puts you were you belong and you NEVER EVER lose 10 games in a row

1 Like

Blizzard’s implementations of MMR and SR are probably very similar in both games. I think that they impose a double standard that is wrong for either game.

1 Like

Naaahhhh the problem is that they are trying to force win conditions that are 50/50 as talked many times in forums. That is the problem. ONLY random queues should be allowed based on purely SR/MMR nothing else.

Since there are lose and win streaks that can go like 10-0 and then 0-10 I suspect many other hidden factors than pure SR/MMR implementation. Those streaks never ever happen in WoW.

1 Like

I agree, that is the problem.

You mean solo queues? Yeah probably.

There is no valid reason for MMR to exist in the first place.

1 Like