Why Handicapping (MMR) is Wrong for Competitive Play

I guess I Just don’t understand why you’re comparing it based solely off of a fixed SR range. In order to make scenario 2 faster, you’re imposing unfair restrictions on scenario 1 that don’t have to exist, as MMR and SR are largely interchangeable figures with a slight variance attached to each. If you restrict it to 375 SR, then you end up with 250-500 MMR present in a match. If you restrict it to 375 MMR, then you end up with 250-500 SR. The result is mostly identical, you’re just picking your poison based upon which number you prefer to work with.

Personally, I don’t see a purpose in having 2 filters (adding SR on top of the existing MMR structure) when 1 (just MMR) would get the job done, but it wouldn’t impact my gameplay experience with either implementation. As such, we’ll call it a philosophical difference in placing value on SR vs. MMR and move on. :wink:

2 Likes

I’m sorry but you are now going backward in your thinking. Both had the same assumptions and goals. It was all about which scenario would be faster in limiting the SR range. Please read the scenario post again. Originally the question was:

Would it be faster to limit SR to be closer between all players in a game by decreasing the acceptable MMR range OR BY keeping the MMR as is but implementing a separate SR check?

I stand behind my theory but don’t have any more time for the conversation. It has been polite, thank you.

1 Like

I’m not going backward in my thinking at all, though. I’ve been arguing from a position that “limiting SR” is completely dependent upon the final result being lower than it was prior to the change by a noticeable margin, which involves a value we cannot know moving to another value we cannot know. After you reach a certain breakpoint, two separate implementations could produce the same result, and so we cannot assert that one is better or more likely than the other, nor assert that either is the method they are using.

I will gladly concede there exists a window where your implementation would be superior if the Blue posts we’re discussing meant SR (as they said) rather than MMR. For instance, if we use what we have talked about previously, and if the goal was only to narrow it to 375 SR specifically, what you’ve described would be better. If the goal was to narrow the SR gap to within 500 while also increasing the quality of the matches by a larger degree, then they could simply narrow the MMR window to 375. That means your scenario is better for queue times if and only if the goal was to specifically narrow SR ranges between the values 375 and 474 while changing little to nothing else.

You should stand by it. It’s a good one that could reasonably be used and serves as the better model for some circumstances.

I thoroughly enjoyed our back and forth, although I can’t help but feel I could’ve articulated some of my points in a better fashion. I appreciate your patience and the time you put into explaining your position to me. I hope you have a wonderful day! :slight_smile:

3 Likes

Well, I do hope they simplify the matchmaking system. I 100% agree with the OP.

1 Like

Ok, dude, seriously, I promise this is my last post regarding our example and theories :).

I may now sound a little “teacherish” so bare with me: no, you cannot reason your way out of math. Math is used to prove things when “common sense” is not enough to explain what happens. As long as the assumptions are correct and you have proven your theory the numbers themselves don’t matter. I could present that theory without any numbers only using letters (x,y,z,k,l,m and so on) and it would still apply. What I mean is that when you have a formula built on assumptions you can insert any fitting numbers to it and it will present you the same end results.

Our assumptions were:

  1. There is a limit how much MMR and SR can differ from each other
  2. There are more people that have MMR and SR close to each other than there are people with MMR and SR further apart (like a bell curve having most hits in the middle)

Our goal was:
Find out if it is faster to limit SR to be closer between all players in the same game by:

  1. only decreasing the acceptable MMR range
    OR
  2. keeping the MMR as is but implementing a separate SR check

If the assumptions are correct and we were able to prove that scenario 2 would be faster it is a fact and no longer under debate.

But if you would argument that scenario 1, even though slower, would with appropriate value, limit SR and result in better game quality then it is a different discussion.

Personally I think that they made that change just so it would feel better to the players. That they are playing with others that seem to be at the same skill level as them but under the hood their MMRs still differ. When you change a complex system it may bring out unwanted behavior. IF my theory is correct and they implemented a separate SR limiter that has lead to the people with MMR above their SR being handicapped more often than before ;). Nice bridge back to the topic of this thread.

2 Likes

I’m not reasoning my way out of math. I’m arguing that your assumptions only apply to a fixed range. Also, I’ve realized after reading your post (I told you I wasn’t the sharpest tool in the box) that we’ve been arguing from positions that don’t agree to this (particularly the bolded part):

I have not been arguing from the stance that it is faster to limit SR by lowering MMR (which could be my failure to read properly before embarking on this conversational journey with you). Of course it’s faster to simply cap the SR if that’s the only thing you’re worried about. That shouldn’t even require a mathematical proof to display. That does fall under the umbrella of common sense.

I have been arguing that you could get the same results proposed by the blue post (narrowing the SR allowed in a match) by only tweaking MMR if the circumstances allow for it (ample population size, and a sufficient narrowing of SR values in a normal match). GIven we don’t know what the population is, nor do we know the amount that SR is being restricted, we cannot assert that your theory is what is being put to use.

We can assert that for any given SR limit you want to arbitrarily put forth, your idea is better, 'cause it is, and includes a larger pool of potential players for any given SR range. We cannot confidently assert with 100% confidence that an actual SR limit is what was put into practice, though, which is my point of playing devil’s advocate.

FWIW: Reading Scott Mercer’s update posts about a dozen times to comb through it for context clues and referencing older posts and what not, I feel like it is slightly more likely that what you describe is put into action over the MMR restriction. His use of “SR” throughout the post strikes me as being very deliberate. So, in a sense, you could say that you have won me over throughout our discussion.

I don’t buy into the idea of handicapping in the first place, but it is a nice way to swing it back around. I completely agree on the “feel better” angle being a real possibility. :wink:

2 Likes

To be fair to Blizzard, they might not know what to make of the data they have. They’re already using that data to segregate the best and worst players on a per-match basis. If they think that’s a good way to handle Competitive Play, they could have all kinds of other harebrained ideas. And we have no visibility to the experiments they’re running on us.

When we talk about matchmaking being bad, I think it gets confusing because players are looking for different things in what they perceive as a “fair” match. In fact Match Making Rating exists to ensure 50% odds for either team to win, which is to avoid one-sided matches. Because those are the types of matches that most players would characterize as “unfair.”

To say that “matchmaking” is a problem is not enough. We need to specify what we want from matchmaking, which is impartial, objective, and equitable treatment. We all assume that matchmaking is impartial, objective, and equitable. But in fact it is handicapped, in favor of the weakest players and at the expense of the strongest.

2 Likes

Nicely put.

extra characters

Is it 15? Where did Blizzard disclose that information?

I read Jeff saying they keep your data since the beta…

Could be just a marketing jab to make you buy new accounts though…

2 Likes

Yeah I thought MMR kept a much longer history than 15 games back. Then again, it might make sense if MMR was shorter term because the meaning of performance metrics could shift according over time and changes in rank.

None of those considerations are counter to the argument that handicapping/Match Making Rating is wrong for Competitive Play. But I’d like to know more, and Blizzard should give us complete information about what we’re doing in Competitive Play.

1 Like

What they are doing is a theme park competitive system.

It’s like a theme park MMO, but instead of guiding your progression through quests and gear, they tried to manage the casuals’ ranking experience (and ended up breaking it for everyone).

1 Like

“Theme park!” I’ve thought many times about the staged nature of the experience. Whenever I find myself playing with and against players with less than one star of experience, I feel like an animatronic prop in their tour of Overwatch.

A real ranking system would separate players of greater and lesser experience/skill. Not systematically mush us all together, and arrange teams to create an illusion of fairness.

2 Likes

What I don’t like, and this also play a major part in this discussion, is that Blizzard keeps talking about this ‘Skill Rating’, while it’s actually the hidden MMR that is being referred to. The actual SR displayed to the player is hardly a “Skill Rating”, as it is too dependent on wins/losses, and only factors in a player’s performance in very small amounts.

I hate how the game makes me feel bad for losing by subtracting a load of SR, despite me actually having outperformed my running averages. If I, say, play twice as good as I usually do, but still lose despite this, why do I still lose roughly the same amount of SR as when I played poorly in comparison? If SR is a “Skill Rating”, then why doesn’t the SR actually reflect this?

2 Likes

I understand the frustration, but do you honestly feel you should be rewarded for failure (a loss)?

Of course not, but I feel that, currently, you’re being disproportionally punished for it. Like I said, if I played my best game ever, I’d appreciate it if the game reflected this, and not simply subtract the standard amount of SR for a loss.

In a completely fictional situation, where I’d outperform my average by 100% (meaning twice as good as normal), I’d expect a near 50% reduction in the amount of SR lost. So I still lose SR, because I lost, but it’s offset because I outdid myself. Conversely, if I win, but was basically doing nothing, my SR gains should take an equal hit.

So basically, I want SR to be more centered around actual “Skill”, maybe, like I implied above, in a 50/50 proportion to winning? Though that still leaves the issue of having a personal skill-rating in a team-based competitive mode, but that’s maybe better reserved for another topic.

1 Like

I love how we’re up to the end of Season 9 and this garbage Ranked matchmaking is still in effect…

5 Likes

Having a good game doesn’t guarantee a win, though. We see that across all team sports, everything from pick-up games at a local gym to professional sports.

From what we’ve been told of the system, and I know this isn’t a “feel good” moment, you could take a small amount of solace in knowing that your MMR should not have dropped as harshly as your SR. Thus meaning that subsequent games would correlate to slightly larger SR gains on a win in the near future.

It honestly just sounds like you want MMR visible, with the possibility of throwing out SR in the process. I agree that transparency would go a long way on issues like this.

2 Likes

This for sure! I don’t care if I lose a game per se, I care when games aren’t balanced and there’s steamrolls because people don’t swap, communicate, etc. There are some games that I’m 100% alright with losing because everyone was on the same page, or it was super close, or whatever else. Losing is inevitable. If you have a 51% winrate, you will eventually get to Grand Masters.

2 Likes

Like I posted in Kawwumba’s thread, I think this issue would be resolved if Blizzard was transparent about the MMR system calculations instead of vague or dismissive one liners about being paranoid. The system is obviously abusable and imperfect and a large section of the playerbase is obviously upset about this.

Just tell the players HOW IT WORKS. I don’t understand why this is such a difficult issue. It makes no damn sense to me how as a company being up front with the players is such a problem?

3 Likes