Algorithmic Handicapping (MMR) is Wrong for Overwatch

Yes, there is, the Devs mention both MMR and SR and that SR “chases” MMR. Please look at Kaawumba’s post:

It has references and links for you to see where all of the information has been gathered from.

2 Likes

Like most mainstream game developers, Blizzard considers themselves quite above the player community’s feedback or criticism. They don’t even respond to inquiries about how the Competitive Play system works. And what they tell us about SR/MMR is contemptuously simplistic.

There is no way that MMR is simply SR without decay. Did you read the quotation from Scott Mercer in the original post? There is a lot more to it.

What do you mean by ‘public code?’

Good summary and great question. What do you say FriendlyFire? What’s the harm in trying to know about these things?

Blizzard claims to be protecting the game from exploitative players, by keeping the workings of MMR secret. But they are only protecting themselves from scrutiny. Players will find a way to game any system, no matter if the company tries to keep things opaque.

3 Likes

LOL … They do as good a job protecting themselves from scrutiny as they do redesigning heroes like sym.

2 Likes

What’d they do to sym?

1 Like

Asked someone who clearly doesn’t play this game.

1 Like

Actually Zenren, I’ll side with you on this one but point out that this problem not only happens all the time but is an indicator of how the system actually works. It is my belief that MOST of this argument is really about the question “How does an average winrate of 50% allow anyone to change ranks?”

What you’re describing, an SR that is much lower than MMR, happens with decayed players. A person could have a MMR of 4500 and an SR as low as 3000*. The way they handle this shows what they mean by “chasing” SR. It’s visible and up front. Basically, you are given more SR on a win and less on a loss until your SR is up to the right rank.

If you keep playing at 1400 MMR you will have, overall, a 50% win rate, right? If, somehow** you actually do end up with an SR of 800 then you’ll gain a bunch for a win and lose hardly anything for a loss until it all adds back up to 1400.

This isn’t what actually happens, though, what actually happens is explained in a longer post of mine here:

The system wouldn’t work by holding MMR steady. If you lose enough that your SR goes down to 800 from 1400…your MMR is gonna go down with it, that’s how these systems work. If your ACTUAL skill remains at 1400 then those games will become progressively easier.

Now, there are people that will say that decayed players have special rules, but those people just want to convince themselves and others that they are better than their SR indicates. There is no reason to think that any discrepancy between SR and MMR is handled differently. It’s a rare bird that judges their skill correctly. Try to be that bird.

*(this may be wrong simply due to time constraints of how long it would take to drop 1500 points from decay)
**(I’m not saying this is likely or even possible, but it’s people’s perception so I’ll roll with it)

1 Like

Orrot, you seem like a pretty reasonable person that has a legitimate and common question about how the system works. It’s a question that I also had and that I have researched and answered earlier in this thread and others.

I won’t repeat the posts here, but if you look at the above post of mine (1 or 2 up) I think you may find the question asked for clarity then answered simply.

1 Like

Aaaaaaaand you basically just said you have no clue what is being discussed. Again.

Hey, I’ve got Ivan here who is a pro ice skater. He is going to race against Average Joe, and we want the match to be fair and 50/50. Joe is ok at skating, but not fast and may fall over. To make the match fair we have replaced Ivan’s skates with sticks of butter.

Why is Ivan complaining? It’s a fair 50/50 match?

1 Like

Ivan wouldn’t complain.

In your hilarious scenario the match will be interesting on both sides but no one will say that Ivan is worse than Joe even if Joe wins. The ranking will remain accurate.

This is why I think the discussion about balancing (or handicapping) is really a discussion about how you can determine rank even with a handicapped match.

You CAN rank with a handicapped match, easily, and actually better than just throwing 1000 Joes and 1000 Ivans together without any notion of balance. You actually don’t learn anything by having a fair Joe vs. Ivan race.

It’s just that the system that you need to use is much different from what most people are used to seeing in sports. It’s a relative ranking, rather than an absolute ranking. Absolute can’t work outside a tournament where everyone plays the same number of games.

Do you love making silly comments at the start of a post that make the rest of your comments laughable?

Please go look up the term “handicapping” it’ll help. Reading comprehension helps, but I did decide I wasn’t going to mock you on that anymore.

Pretty much this thread is a lot of Ivans and Joes complaining. But all good, you’ve magically decided that they aren’t complaining.

I never understand your need to bring up subterranean trade routes, which is obviously what you are talking about.

1 Like

Thank you OzoneOOO, I try to be reasonable and coherent. I can see where you are coming from and iterate that this is a discussion on match making and how MMR is used to make 50/50 matches.

I laughed at this.

To be more aligned with a real world analogy:
In drag racing, to be fair to two cars, the slower car starts the match first and the second car starts after a pre determined time.

Example:

Car A has a mean time of 1:12
Car B has a mean time of 45 seconds.

Since the true race of drag is that split second timing from yellow to green and how fast you react. Car B starts his run 27 Seconds AFTER car A started his run. This way the winner is chosen by the person who reaches the finish line first.

I wasn’t aware they did that but I love the concept. Its removing the vehicle aspect and making the final victory more skill based.

Pity Blizzard does the opposite and has a layer designed to offset skill…


Going to finish the next subterranean trade treaty anytime soon Ozone? I’m out of time to respond again, maybe in a few weeks I’ll be back.

1 Like

It’s titled as being about how MMR is wrong for competitive play. The OP doesn’t care if they are effective in using MMR to make 50/50 matches or not. It’s that they do it that he has a problem with. I’ve been discussing this over at least 2 threads and probably thousands of posts. In fact, the premise relies on the effectiveness of their MMR system. The debate from Cuthbert is never about how accurate it is. I’ll give a summary so far:

He actually has two arguments regarding this. The first is explicit and in the OP, but the second only develops later in the comments.

The first argument is that using balanced matches “corrupts” the win result. That is, your rank is meaningless if you always have a 50% win-rate. Simply put, at one point he says that balanced matches are “unproductive by design.” This is both the main hinge in his argument and incorrect as a matter of fact.

I don’t actually care if people call it handicapped vs. balanced. I think the term “handicapped” is a overdramatic and a bit misleading, but I’ll still use it when discussing the issue. However, using handicapped matches most certainly does not corrupt the win result.

The alternative would actually be much worse and lead to more problems that are similar to the more common complaints. This is a technical issue regarding ranking and matchmaking that extends outside the realm of Overwatch into any ranking system and his premise can be easily refuted without using any references to how the actual Overwatch system works. See above posts for refutation. Sorry to make you do the work, but my posts are already notoriously long.

The second argument which has been exhibited more recently is that MMR and the 50% win rate is an intentional ploy by a malfeasant company to ensure more people become addicted to their game. This is an interesting argument to make against MMR as there are many games one could call “addictive” that don’t use MMR. Also, yes, having a game where you are fairly matched is much more fun than the alternative and if you get “addicted” to fun games then I guess I would have to agree that MMR makes Overwatch more fun and therefore “addictive”. Cuthbert goes so far as to propose that legislation be enacted to stop the use of MMR being used to make games more fun…ahem, I mean more “addictive”.

The debate does trail off into several various pet-peeves of various people, from the fact that it’s hidden, the how much MMR and SR can be separated, how accurate MMR is, whether PBSR is good or not, how groups or playing different heroes affect things, etc., but I’ve been discussing this respectfully with Cuthbert for awhile now and am pretty confident in what HE actually believes. None of these tangential issues really don’t make any difference to whether or not a “handicapped” match corrupts the ranking result and is thus “wrong” for competitive play.

What a lot of people like you come here and hear in his initial post and some subsequent posts is something a bit different. Frankly, it’s not as interesting but as it SOUNDS similar to what Cuthbert is saying and it makes people say that they agree with him, even when they really don’t, he tends to encourage it by acknowledging the kind words and welcoming any complaint as if it fits. Take for instance the “hidden MMR” complaint. He’ll act as if it’s you against the world on this, but it’s actually a pretty minor dispute. Very few people will say that it’s great to have MMR be hidden but for some it’s a bigger deal than for others. It’s the same with MMR/SR discrepancy.

Here’s your proof. If Cuthbert wants MMR to disappear, as he’s said clearly multiple times, then whether MMR is hidden or how far it can go from SR really doesn’t matter one bit, does it? Still, rather than telling people to take those complaints elsewhere as they are irrelevant he uses them to grow emotional support for what should be a very technical dispute. The end result is that people think he’s right about his OP when, actually, very few have read and understood the actual message.

To be clear, if you’re concerned with whether the matches are actually 50/50 or how you are supposed to climb in rank with the matches being 50/50, or that MMR is hidden, or that MMR is inaccurate, that is not what the OP of this thread is actually about. Not that these aren’t valid concerns, but Cuthbert’s concern is actually more complex than a lot of people give him credit for, IMO.

2 Likes

In my experience I found that MMR is very stat based.
Guys ALWAYS remember that Blizzard removed bonus streaks and sr loss/gain modifiers based on stats but they NEVER said they removed it from MMR.
Community was blaming Blizzard for streaks and stats modifiers and in my opionion they were always very smart pretending to give to our community what we ask for while nothing really changes.
Guys, atm Blizzard got NOTHING to judge the attitude of a player in cooperation, timings, or whatever else regarding ow as a team based game.
STATS are the individual final results and the only valuable thing they can actually judge as they are comparable in tiers and are a very good way to “understand” the carry potential of a player.
Every time i lose some match with bad stats i find more idiots and tilted players in my matches as if they try to put togheter ppl they think they deserve to go down and viceversa.
I realized I was really able to shorten loss streak durations by farming stats with my best plays instead of filling the classic 2-2-2
Im afraid to tell you that this system actually rewards personal skills but don’t hope they are gonna admit it.

Look at how oversumo works, it can accurately tells you if you are likely to go up or down BY ONLY WATCHING STATS, and in my case it never failed.
The mechanism is very simple, oversumo compare your sessions stats to other players in the same tiers and tells you:

stats under 25% of the rest of the tier = you are going down

stats above 75%, you are going up

stats between 25% and 75% you will fluctuate in the same tier depending on which side you are holding (under or above 50%)

Now it comes my exploit as I think that MMR can move in both direction regardless of the final esit of a match.
When you lose a match you are likely to have bad stats even if you are not that bad but your team simply get stomped.
So, focusing on farming in these cases is the key.
Pick the hero you think is better in that specific match for you to farm stats and if you succeed you will easily see how the MM will be tender in next matches.

Pratic example: you are filling by tanking or healing but enemies are wrecking you with pharmercy and tou know you are a very good hitscan? Screw this team and go for it, you are probably going to lose the game anyway and maybe the rest of your team will see it as throwing, ignore them, your job is raising your stats toward 50% or even more to make your MMR less punished.
It’s the same for other roles, if you feel the lack of healing is embarassing go even for the 3rd healer and farm good healer stats.

Sad truths, you can come here and tell me I’m stupid but Im not gonna stop with this attitude until I see it perfectly works.

Ofc this is not an exploit to raise your rank but it can definitely helps you in reducing loss streak duration and the volatility of your rank, and this means you will be more likely to stabilize at your cap and improve faster (instead of going through an insane and tilt-like loss streak).

p.s my eng is crap no need you let me notice it.

1 Like

I agree. I am so sick of going up and down 500 sr every week. Its obvious the elaborate system that blizzard has tried to make is deeply flawed comlared to the simple elegance of a NORMAL RANKING SYSTEM.

3 Likes

Cuthbert’s stance: MMR used to make 50/50 matches is wrong.

My stance: In turn Doing so only makes the climb, or fall, prolonged so you reach your potential slower than necessary.

This prolonged aggregate is in place so you play longer. Competitive should not have a mechanism to make people play longer just to make it take longer. Getting better should be the goal to make you play longer, hence competitive.

Because it is merely a way to elongate the play time and not the progress of the player. There is no direct product of the design other than to make it take longer.

The word handicap is a sports term. It is where one match is given a level playing field. https://en.wikipedia.org/wiki/Handicap_(golf)

The entire reason the loot boxes are what the way they are is psychological, this is a proven issue that it causes addiction similar to gambling. (https://www.sciencedaily.com/releases/2018/06/180628105014.htm) Are you implying that they would not use similar approaches (To make people addicted) elsewhere?

Name some competitive games with ranking that does not have an MMR system.
Those that do: CS:GO, LOL, World of Tanks, Dota(2), Hwn, and (Other Blizzard games)…

On a side note, I have friends that play some of these games. They all have the same feeling of “Win streak and reach a new high? nope beat down the next few games.”

Problem is the MMR psychology is in the raising and lowering of SR than the “fun” games. You “feel good” when you see your SR climb and climb, you feel bad and then “need to climb back up” when you lose SR.

I love knock down dragged out matches where every bit of it is a close hard fought back and forth and who wins is a culmination of victories… Problem is, I really only should be getting that at my actual rank, and not climbing or falling to it. It makes the matches falsely feel more stable than they should be. (This does indirectly make people think they should be higher when they reach a peak and fall back down.) This is my opinion.

Each of us has their concerns for it. Whether they align perfectly with Cuthbert or not. This thread is a discussion of MMR and its place in Competitive. The fact that we approach them in different ways shows that we are neither sheep following a single minded call nor have the same happy compromises there in.

I have a very weird knowledge base.

1 Like

Start in this thread at post 241 and read our conversation. Also a source for a lot of what follows.

Unfortunately this is not his stance. First, he (correctly) equates MMR as balance/handicapping. He’s fully aware that MMR and balance and handicapping and 50/50 are different ways to describe the practice of making 2 teams of equal overall skill level.

Given it’s the same idea, that means your description of his view would be a tautology and I assure you that though he does think that MMR/balance/handicapping is wrong, his stance is more along the lines of anything other than “impartial” (which in context must mean without consideration of skill) assignment of teams and use of win/loss ratios as the determinant factor of ranking is wrong. (Source: Thread search “impartial”)

It’s a coherent viewpoint, I’ll certainly give him that. If you reject the idea that a balanced match is productive for ranking then the only choice is to remove matchmaking from games. It would technically work too, but we get into the consequences of that in post 241. And how balanced matches are perfectly well productive later in the thread.

It’s often used interchangeably with balanced in this context (see sources below). You’re right, it’s not strictly used properly here.

Cuthbert makes no distinction between 1SR differential or 1000SR differential regarding what he calls “handicapping”. If one player is 1SR up and to compensate the next player is 1SR down, that lower player is being handicapped by the higher player. He goes further, actually, noting that any attempt at balance affects the win loss ratio earned by a person and thus “corrupts” the ranking, which I generously take to mean makes it meaningless. To him it really doesn’t matter to him how close the 12 players are to the same MMR value in practice as he wants MMR thrown out anyways.

To use a racing analogy, that would be like calling the strict vehicle rules of NASCAR “handicapping”. Whether you consider it an even playing field or unfair restriction depends on how invested you are in the win.

I didn’t restrict the list to only competitive games, my man. Just games in general are often called addictive. And they can be, it’s true! I admit I lost a summer once to the Sims… MMR and gaming addiction are not dependent concepts, though.

If we’re going to get rid of every game mechanic that anyone has ever found themselves unable to resist, we’re not going to be left with too many games.

A lot of people think that their rank is absolute, that if you’re a 3000 SR player than you should stay at 3000 SR. The truth is quite a bit different, I’m afraid. Even without your own variability, what SR represents is not absolute, it’s not measured against some independent value like K:D ratio or win percentage. It’s a relative ranking where 3000SR likely means something completely different at 3am on Tuesday than it does on a Saturday afternoon. People have used this fact to exploit the system (see Brazilians) and I understand the desire to have a “yardstick” type of ranking, but it’s called a “ladder” for a reason, 3000 means you’re better than those playing right now that are below you and worse than those above. It’s a ranking of skill, not a measurement of it.

Blizz could probably do a lot with simply educating people on what SR really means more than how it’s calculated, which is what people ask for.

You’re right that this thread has devolved into a hodge podge of unrelated complaints about matchmaking and ranking. The most common one is what I addressed above. I’m not one for pointless whining without movement towards communication and consensus. I attempt to educate.

It’s hard to talk to someone when you’re not talking about the same thing. Meaning is too easily confused in these forums. As long as we dont’ get angry at each other when the misunderstandings happen eventually we can get to the real issues that affect the MM, which are plenty, just not the ones that people get so upset about.

Some more sources (read with context):

1 Like

True there are games that are addictive and not focused on pvp competition. but, when trying to convey that MMR is designed to do so, you cannot compare OW to Tetris. Tetris has no MMR, but is addictive, does not equate to MMR not being used to keep players playing.

It is not absolute in that your rank is fluctuating. This is true. but unless you are gaining skill or, deteriorating, your skill fluctuates just a bit, not the 100s or more per day. Due to streaks on either end.

What times of the day and what day you play can and will alter the types and caliber of people you will play against. These are things that need to be adjusted for in the formula.

As a test is a measure of knowledge, SR is a measure of skill, which is why it is a bell and not a linear spread of people. As in if you are at 3000 you are not better than 60% of the populace, its more ~85%:

In Testing terms, getting a 90 on a test is not a measure of how much you are better than the rest of the class, but getting an A with a score of 70% due to a grading curve, is a measure of how well you did vs everyone else in the class.

I hope I am not in that whining section you are referring to, if you are alluding to that. Merely would like to see changes, even if it is just a bit of a less opaqueness in the MMR for Matches we currently have.

1 Like

Not that I’ve noticed, but it seems the people that believe that SR can be maintained significantly different than MMR fall into 2 camps. Those who think that the system is designed with good intent but in the dumbest way possible, and those who think it’s designed that way because the company is evil.

Both camps fundamentally believe that they would have a higher ranking if the system didn’t drag them down. The first camp can be reasoned with, but both are whining.

There’s lots of questions and issues with the system as it stands. It’s reasonable to focus on the actual problems rather than made up ones, but the actual problems don’t elicit the same kind of emotional response that “the system is holding me back” generates.

What keep players playing games (in general) is fundamentally 2 elements. First, a progression, i.e. always having an accomplishment (that “kick”). Secondly, that accomplishment must be obtainable without too much work otherwise you lose motivation to play. This fits nearly all games, and does of course fit Overwatch in many respects, but actually doesn’t fit MMR that well.
Of course there’s more but they don’t really relate to OW, see: http://www.techaddiction.ca/why_are_video_games_addictive.html

More on this from http://www.slate.com/articles/technology/future_tense/2017/07/why_young_men_might_be_playing_video_games_instead_of_working.html

Work can be tedious or frustrating, and it often carries with it a risk of failure or embarrassment, but in the right circumstances it can be deeply rewarding. When those rewards are stripped of risk, commodified, and offered as entertainment, you get modern video games. Is it any wonder that some people may be choosing them over the real-life analogue?

There was a thread awhile ago that complained that MMR was designed to keep you from going up in ranks. I thought that was a funny complaint, because that’s literally the reason for MMR. You aren’t really supposed to climb, at least not by just playing. It’s not like the WoW progression system, where you play, kill things, win games, and go up in rank, prestige, and power. MMR finds the opponents with which you can only maintain a 50% win rate and makes you play against them. It keeps you at a difficulty where you won’t be able to net gain SR.

In other words, there are no number of games that you can play that will increase your rank. Unless you actually focus on improving your play, AND do so more than others of your rank, you will never, ever go up in rank beyond what random chance provides. So in order to get the reward of higher SR, you need to put in actual WORK. This is work that some people actually get paid for, btw.

That’s not a bug. That’s the design. It also takes away the second requirement to make an addictive game, progression is simply not attainable through gameplay. A point that some people only realize after the anger sets in, unfortunately.

So this “MMR is addictive” stuff rings a bit desperate, to me. Are other aspects addictive? Sure. But a system that matches you with people of your skill level makes the game playable, for most. I’m not very good. If it weren’t for MMR I’d lose a LOT of games and have a terrible time while I’m at it. This is true for anyone below average, which means half the people playing. The other half wouldn’t be challenged and probably wouldn’t enjoy it either. Minimally playable is a far cry from addictive, unless you’re just against video games in general.

Um…I think we agree? The word “measurement” can be a bit confusing, believe it or not. It’s what accomplished people in Universities do everyday. Measurement is harder than it’s given credit for.

Even if everyone gets an A on the test in absolute terms, they can still be ranked on an A-F scale relatively (on a “curve”). So the relative ranking will be in the form of a bell curve as it will have some natural variation. In OW, if everyone below diamond stopped playing for a season, those barely at diamond would drop in rank because there wouldn’t be anyone for them to win against. ALL their opponents would be better than they are. They’d lose a LOT and the remaining population would spread out along the bell curve, from Bronze to GM, but the overall average skill of players would be much better. So your “average” player, now at 2250 SR, WAS at 4000 SR, before the Great Plat Purge of 2018.

There is no way to judge the absolute skill of a OW player (to my knowledge). There’s not a test that is independent of other players that everyone can get an ‘A’ on, so all grading of skill is relative, or “curved”. This is common with some tests too, where the test is so unbelievably hard that no one gets an A. All scoring is done relative to the best performer. Military advancement exams, for instance, are like this, you can do better one year but still get a worse score. I suspect the ACT and GRE are as well, but those are relative to the entire history of the test.

1 Like

I don’t think Blizzard is evil, I do think Activision influences things a bit much and has a more money forward focus. I think I would have the same SR or close to it, I am fairly new and working to improve. I hope to reach Gold by next season.

This is true, there is a lot of people mad at how the system is perceived to stagnate movement upwards. A transparent system would eliminate some of these notions.

I understand that philosophy, I am one of those that in WOW that did research in their class to be the best in the raid I was in. This is not new and I do not expect my SR to increase significantly without learning the game further.

MMR itself is not the addictive point. It is the slow climb and that SR springs and is kinetic. MMR, keeping 50% win condition, slows the climb to virtual coinflips, unless you start playing at a higher skill then the MMR has predicted. It’s the

“That was a close match, lets try another…”

“Damn I lost again…another…”

This season I climbed from around 1400 to just over 1800, peaking at 1935. I have started VoDs on youtube to watch wherever I am.

I think that its not desperate to want to have good games, and if you improve, you are not then matched with a hidden formula. I believe that matching solely on SR should be efficient.

That way, you get better, you climb, you face better opponents.
With Hidden MMR, as you get better, you face equalized matches. you may climb. You may fall before you climb, you may yo yo within 100 SR.

Yes, the MMR may pull you up by giving you more SR for wins and less for losses, but you have to win the matches.

To me having a stagnating SR that floats +/- 50 could be a way to see your skill level. But that may not be satisfying to some.

2 Likes