Algorithmic Handicapping (MMR) is Wrong for Overwatch

Blizzard confirmed that MMR does not apply above the rank of Platinum. I don’t have a link though. It makes sense because PBSR adjustment was designed to offset the match-skewing effects of MMR. PBSR is visible, but MMR is invisible, which is why there’s confusion.

1 Like

I really think you’re mistaken on this one. I’m gonna need a source from someone on it before I bite.

It makes perfect sense that PBSR was removed to promote more teamplay at higher ranks. It makes zero sense to have MMR function at the bottom of the ladder and not the top. MMR is the primary driver for what is used to put matches together. If you can source me something that indicates Diamond+ is an entirely separate system of putting matches together (i.e. not using MMR at all), I’d likely switch sides.

Edit: To be clear, I’m aware the SR variance allowable for matches is/was narrower than the bottom, but that is not what I’m speaking of.

1 Like

I can’t provide a link. It is one of the details that has been confirmed in this thread’s history, multiple times by credible sources, but I think the primary source is gone. Some important statements, like the one from Scott Mercer, were made on Blizzard’s now defunct Battlenet forum, so the posts are offline.

1 Like

I was active from Beta and well past the transition to the new forums. There was nothing I recall on the old forums from developers that stated MMR wasn’t used for matches above Platinum. I followed the majority of the matchmaker topics and Blizzard posts religiously. It would almost have to exist on a 3rd party site (like a Reddit post) or be something newer for me to reasonably believe I missed it.

Having 2 entirely different matchmaking protocols that completely segregate the player base around an arbitrary SR figure would be something that should’ve gotten way more attention. Far more than PBSR - which essentially exists as an opportunity to escape what most people would call “ELO Hell” (my view).

Even skimming over your main post and having actually sat down last night to watch the entirety of your video (well done, btw), the fact that you have not used the idea that the populations of the game are divided in experience to the extreme due to the bottom being manipulated and handicapped while the people that luck out of the lower brackets rise to the top to play a legitimately fair and competitive game leads me to doubt your assertion that MMR isn’t used. I feel like you’re glossing over what would be one of the biggest pieces of evidence for your cause if you can somehow source it, honestly.

Either way, you’ve certainly grabbed my attention with the statement and your willingness to double down on the “MMR isn’t used above Plat” idea. So, even if you don’t reply to anything else (since it’s mostly just me rambling in a state of bewilderment and confusion trying to come to terms with how I managed to miss something so huge), I have to ask:

How do you think people above Plat are matched? I know you have handicapping to explain most of the ladder, but I’m genuinely curious of what your understanding is for the top of the ladder with how the teams are decided.

2 Likes

MMR 100% exists above 3000

Search for WyomingMysts comments. You’ll find it.

GM players start EVERY season off at 3900 and have to grind back to the T500 board every season.

To prevent GM players from getting stuck in Diamond and Masters games, the MMR matches them against players with higher MMR.

PBSR prevents me from losing more SR than I would win to push me out of lower Elo faster. Seems like you didn’t read my example.

Unless you’re smurfing, the game isn’t intended to be carried by individuals.

One of the first dev updates I ever saw for Overwatch (before I even played) daddy Jeff states that 6 players on a team is the perfect number to enforce team synergy and cooperation.

Welp…. Their model works too well as individual carry potential, as low Elo players, is nonexistent.

Your argument here isn’t about bad match making, it’s the over arching game architecture. And it appears the community won on this one because the game is going to 5v5.

Uncle Aaron even states that 6v6 requires too much team play and 5v5 will allow individual impact to matter (again, like Valorant :joy::joy:).

Smurfs are the outliers and proof that PBSR works as intended.

Again….

That is clearly PBSR pushing me out of bronze and silver because my skill (as quantified by MMR) drastically exceeds that of my peers in that rank.

The game/software wants to move players to the point where their individual contributions to the game cannot easily define the outcome because the point of Overwatch is to play as a cohesive team, manage cooldowns, and work together to achieve the objective.

2 Likes

Source?

WyomingMyst is just another Overwatch player, not a primary source of information about MMR.

I read it, I’m not convinced.

Then why is team size being reduced to 5, for Overwatch 2?

Team size has nothing to do with my argument.

I don’t see how.

The contributions of individuals will always define the outcome. Individual players’ contributions to the game are never equal.

2 Likes

I was unaware of this until now. Though, for what its worth I wondered why my individual role SR never quite converged. I assumed that over time and playing a variety of different roles that convergence was natural; not instead, that I was a consistently worse tank and dps than healing, for all time (as evidenced by historical fact of my ratings).

That being said, I quit competitive over the lack of map variety and find my QP games far more fun.

1 Like

Thank you for this mega thread. It draws attention to their corporate vision and vision for esports.

For me the rigging is noticeable across any and all accounts. Doesn’t matter where you are on the ladder, you’re not playing for your rank, you’re playing to win a cherry-picked match, rigged to be close based on your recent performance. Climbing is unfun and every win or loss seems predetermined.

Why can’t players just play for their rank, at their rank, and have their rank? We want to compete without MMR. We don’t want manipulated games in silver, gold, plat, doesn’t matter still rigged match on match. Who can honestly defend this model.

4 Likes

Because they are prioritizing another aspect of balance over synergy and cooperation. Namely, tank hero balance.

Edit: which I think also supports the OP’s argument. Algorithms like this can make a lot of sense, but the variation and how far they reach for balance needs to have limits. For example, the algorithm can’t decide a game of 5 bronze players and 1 grandmaster is the same as 6 gold level players.

Tighter limits on this is inherently juxtaposed to longer queues which likely use a greedy method of making the best decision of a localised set of available players. Composition would also affect the queue times, and perhaps removal of a second tank constraint greatly increases their ability to tighten limits as it offsets the queue time increases associated to tighter balance constraints.

Queue times are more closely associated to success and performance of the business than play balance is. So queue times win. This explains the smurf problem (overreaching for balance + greater variation allowed on new accounts), the queue time problem (always long for dps), and the consequent matchmaking problem articulated by the OP. Universal their of everything - thank you S. Hawking.

2 Likes

Dev update from Papa Jefe (I’ll find it later since its old AF at this point).

Incorrect… Forum Technical Support who also has “limited” correspondence with the dev team.

Well you also believe you’re a masters player having never played above platinum. :woman_facepalming:

Because 6v6 requires too much team play to be successful, where as 5v5 allows for individual carry potential

Uncle Aaron very clearly states it in this dev update.

Then you understand less about Overwatch than you think you do.

Because you can’t make a Masters player stuck in Gold at a 50/50 win rate unless they are throwing.

Yes it will… But it is the job of the software is to prevent smurfs from ruining low elo games and shove them up.

Indeed… Which is why I know, NO gold player could be dropped into Masters level games and be successful. To believe so is purely delusional.

5 Likes

You’re very welcome! And thanks for your comments, you speak for me and the majority of players.

Does that mean only 1 DPS would be allowed per team?

I have wondered a lot about how queue times relate to Match Making Rating and group size. It seems plain to see that larger groups are forced to deal with longer queue times, and queue times increase when a group gets on a hot streak.

I suspect that the machinations of the SR/MMR system are largely designed to accommodate groups. But really it makes no sense to allow groups in Matchmaking, when each individual player has a different rank. I think Competitive Play should be solo-queue only.

Please give me timestamps, the video is over 2 hours long. Or quote them yourself.

Okay, but I ran three polls on the Battlenet forum, with the same basic content as the original post from this thread. The polls had thousands of viewers and participants. The post included the quotation from Scott Mercer, which had also been posted to the Battlenet forum. I don’t know why you didn’t see the posts, maybe you were not checking the Competitive section at the time they circulated. But the posts had large audiences, and stayed in discussion until the day Blizzard decommissioned the Battlenet forum.

What you call ‘volatility,’ I call ‘just deserts.’ Under fair, impartial solo-queue matchmaking, players who perform well would be more likely to win games and rank up. Players who perform poorly would be more likely to lose games and rank down. Under Match Making Rating (algorithmic handicapping), the reverse is true.

You are saying a lot of things I agree with, and I just want to expound on this last point because it hasn’t been raised by any other poster that I can remember. You have correctly identified the conflict here, between nature and artifice. That is one of the things that Match Making Rating represents to me. If we believe that nature is generally healthy, then how can we believe in a system that we know to be artificial, and whose details of invention are not disclosed to us? Just a thought. Thanks for adding to the thread.

3 Likes

In statistics we call it variance and confidence intervals. You shouldn’t be punished for a few bad games–especially with the RNG of throwers and leavers. Likewise, you shouldn’t be rewarded for a few excellent games–especially if the enemy team has throwers or leavers.

We expect some stochasticity with win and loss streaks, the player’s rank shouldn’t spiral out of control on that basis. If it did, rank would be meaningless.

That is exactly how the system works. Players who perform well are more likely to win games and rank up. Players who perform poorly are more likely to lose and rank down.

You can’t rationally deny that isn’t the case here.

A high elo player will never be hard stuck in a lower elo. A lower elo player will not randomly climb to high elo.

4 Likes

I’m aware of your polls, silly. I’m also aware of all the competitive matchmaker discussions that tried to dissect what devs did and didn’t say - basically everything from your stance to Kaawumba’s compilations with dozens upon dozens of hot takes in between from random people. That’s essentially the point I’m making - nowhere, in any of that chaos with thousands of responses and views, have I ever seen someone actually use the idea that MMR isn’t used after Platinum as a way to effectively bolster the argument the matchmaker is rigged. This is striking, because it would be the easiest way to argue the system actually is rigged against most of the player base.

So, from where I sit - we know they use MMR , and that SR is different above Plat for a few reasons, but the burden of proof to substantiate the claim that MMR isn’t used above Plat rests on your shoulders. Until that time, the only reasonable conclusion I can draw from this set of circumstances for myself is that you are mistaken and have recently misspoken, and that MMR is used across the entire ladder.

And again - if MMR isn’t used, I would love to hear how you think teams get decided above Plat.

But anyways, that’s kind of a side street from the rest of your discussion, so I won’t continue to chirp on that one too much. I don’t want to create a strawman to detract from the work you’ve put into to promoting discussion on the handicapping front.

5 Likes

Right, pardon me. I don’t think my brain was working in full gear when I wrote this.

It has been a saga, for sure.

Yeah, fair point. I did not know for myself that PBSR does not apply above the rank of Platinum. Had I known this, I might have paid more attention to the fact, because it squares with reports that MMR likewise does not apply above the rank of platinum. It was reported by several different forum users, and I’m pretty sure that it was verified by an Activision/Blizzard rep. But I didn’t record the primary source and I don’t expect you to take my word for it.

This is fair, and if I can win you over by finding the primary source of this info then I’ll try!

I think matchmaking should be solo-queue only, based purely on Rank with no reference to Match Making Rating or any other statistical analysis of player skill. It would be the same from the top of the ladder to the bottom, much simpler than the existing system and it would make matches in seconds rather than minutes.

Not at all, these are important details of the invention I am criticizing. You are right to look for a primary source. I will add it to the original post if I can find it.

I don’t think you’ve done that, it’s nice to get into the weeds with someone who knows the terrain. Thanks for participating and leading the discussion in good faith.

2 Likes

Two quick responses, first I completely agree that matchmaking should be solo-queue only. Second, it is quite possible that an unspoken goal of the MatchMaker™ is to provide intermittent rewards (in the psychological understanding, go look it up, it’s quite chilling in this context) and thus act like a slot machine for players.

1 Like

MMR is still used in Diamond+. That’s evidenced through high-elo SR decay and we know matches are balanced around MMR.

I agree. I despise playing with and against stacks.

It’s called Engagement Optimized Matchmaking Framework. It can still make fair matches, but it also attempts to increase player screen time. Here’s a paper on it:

  1. Chen Z, Xue S, Kolen J, et al. EOMM: An Engagement Optimized Matchmaking Framework. doi:10.1145/3038912.3052559
3 Likes

Hey there!

Sorry about the hiatus. Toggling tasks as it is!

So I went ahead, after a while of thinking about it, and I added party support. Now you are able to directly input local SR for teammates in your party. I left the inputs unbounded, so unrealistic parties can exist, however there is a computation hard limit of 1000 SR range. Anything beyond that in difference will break the matchmaker.

For now at least, you can directly input the SR values you’d like to compare/simulate and see what kinds of players you’ll be matched with. Additionally if you only want to compare your current SR against other potential matchups as a solo queue, you can! Just edit ONE of the direct input boxes!

And, sorry about the initial version of the sim. I decided it was better to allow it to simulate matchmaking surrounding a TARGET SR, rather than allow direct input. Now I’ve added the best of both worlds!

Note, utilizing the party input parameters will OVERRIDE the target SR input. As a result, please make sure your party is within 1000 SR of each other! If the ranges get abused, I may put in hard limits, or simply exclude players outside of the critical range from the calculation.

All in all, I hope you find the tool even more useful! Enjoy!! :grinning_face_with_smiling_eyes:

docs.google.com/spreadsheets/d/1XtVVYyEx0__EwdrGJ0WGbZr_rUI3Up94Wacu-Fi2kRc

Hey Cuthbert!

Sorry, was gone for some time. Wow, really appreciate all of the incredible things you said! Thank you!

I’ve just added a new feature to the sim, which by what seems to be a popular idea, and one I’ve been thinking about since publishing the tool, is to allow players to input SR directly into the model. For some time now, it’s been “simulating” picks from a random pool, so you can steer the matchmaker into a desired direction, but the result is largely a ballpark number. It was designed to show what happens in the general case of matchmaking, much less towards the specifics of individual matchmaking.

That all changes now. I just added a feature which lets you input up to 6 of your own SR values as a sort of “party” and the matchmaker will counter-match your party as best it can to enforce 50/50 win chance!

Additionally, if you simply want to simulate solo-queue, you can input your SR into just one of the 6 available input boxes and it will match your score with others. It’s quite the intrigue!

3 Likes

Your every loss and win below 3000 is affected by PBSR. SR loss or gain has two parts, win/loss part and performance part. If your MMR match your SR, you are badicly gaining and losing 23 SR. If you consistently overperform you will be awarded by PBSR system in both, win and loss. If you win, pbsr will give you more SR, if you lose, it will remove SR from loss. If you overperform or underperform, system doesnt want you in the rank you are in. Thats why there is PBSR.

The way of how this works goes directly against what you believe. You said many times that if you play well and better than others, you are placed in harder games with worse team mates. Well why is there PBSR system to push you from silver if you dont play like silver? It would absolutly make no sense to have two systems fighting each other. One trying to push you out and second one trying to keep you there when you overperform. This is where your whole idea of how you understand handicapping in ow fails.

I can say that some games feel like i am being handicapped by system. For example if my tank is silver and enemy tank is high gold, i dont like it even when its even in team vs team MMR as i have for example higher support team mate vs enemy support. But for me as individual it still might suck. This dont happen often in well populated elos tho and its certainly not done on purpose by design. Only reason why this sometimes happens is not big enough pool of players at certain time or people playing in groups. It might also happen more frequently as blizzard prefer short queu times vs balanced games. So sure you might feel handicapped by having different ranks in certain match but it cannot be done differently unless queue times will go up significantly to make more balanced matches. For example if xou would balance 50/50 roles and not just teams.

I agree that if you are for example diamond smurfing in gold, you MMR is still gold. Well it might be plat if you are high gold. I made many experiments on this and i can pretty much safely say that your MMR will never go too far from SR below 3000. I cant speak for higher ranks. There is 150 SR limit per match and together with my tests i belive that your MMR cant be even higher or lower more than 150-200 points from the SR.

Why? Because as you said, you are compared to others in your actual rank. System cant tell if you are diamond, plat or gm dominating the silver. Just the fact that you dominate silver game doesnt mean that you will dominate gold game so it would absolutly make no sense to give you diamond MMR in gold.

Its honestly good thing, well its double edged sword. On one hand it helps players who over perform to rank up faster or rank up even with 50% winrate, but on other hand smurfs dont climb out fast enough as they should.

3 Likes

This made me think of a question. All the posts I’ve read about the matchmaker expressly described two general rules, 1) groups of players had a max range of 1000SR, and 2) the goal was to find a 50 / 50 probability of win for either team.

We know that rule #1 applies to players and rule #2 has a high priority for the matchmaker, BUT when team A already has a range of 1000SR maybe the matchmaker is allowed to create an opposing team with a greater range then 1000SR in order to achieve rule #2.

Thoughts?

2 Likes

Actually, the way I understand it to work is that having an extensive range of SR in a group actually does more harm than good, as the matchmaker will attempt to even things out by pitting you against a team that is close to your group’s average SR.

So if you have three 2500 SR mates, and three 1500 SR mates, your team SR average is 2000 and you’ll most likely fight against players who are around 2000 SR.

Try it out in the tool! That should clarify some things.

Maximal SR Range Example (1500-2500 SR)
i.imgur.com/6KaTddJ.png
Minimal SR Range Example (2500 SR)
i.imgur.com/dA1zyBl.png
2 Likes