SR, MMR, and Role Queue

[Kaawumba-1133]

When the reference is a timestamp, that comes from the Jeff Kaplan / Seagull interview on July 18th (Seagull & Jeff Kaplan: NEW Hero Confirmed! - YouTube).

The matchmaking and rating system has gone through a year+ long overhaul (00:54:30) to prepare for the new competitive ruleset (Introducing Role Queue - News - Overwatch), and I will describe what we know so far.

Season 17 will be shortened, and end two weeks early. After this, there will be a two week beta season, for the express purpose of testing out role queue (Introducing Role Queue - News - Overwatch).

Players must queue as tank, support, or dps (or some combination of the three). When a match is made, they will be given one of the roles that they selected, and will not be allowed to switch roles during the match.

There will now be three MMRs and three SRs (00:10:00). There will be one MMR/SR for tank, one for support, and one for dps.

@ScottMercer

If you’re a new player, placements are straight forward. You start at a “new player baseline” SR and when you win games your SR goes up, and losing games lowers your SR. New players are also set at a higher variance level, so the amount of SR you gain and lose is multiplied higher. The matchmaker does this because as a new player it knows very little about your skill and wants to determine it as fast as possible. The increased variance slowly decreases as you play more and more games, and the matchmaker is more confident in identifying your skill.

Blizzard has been calculating each player’s role MMR for the past several months (0:08:02), though it hasn’t been used in matchmaking yet. This MMR will be used to seed placements for Season Role-Queue-Beta (00:14:27).

@ScottMercer

To help us seed your role’s SR after initially completing placements, we’ve been tracking your match results as you play the three different roles for several months. We want to help identify your skill at each role, so we can place you at the SR that reflects that skill. If we do a good job at that, you’ll immediately start playing in fair matches. The more inaccurate we are, the more games you’ll have to play on that role until your SR can adjust properly to reflect your skill. This also means that the more games you played on each role, the more accurate we could be with your new role SR.

What happens if you’re a returning player, but you never played one of the three roles? Since we don’t any have data to help us guide what your new role SR should be, we turned to some interesting data science. We looked at the population of players where we did have data for all three roles, and asked questions like “How well does a typical GM skilled player who primarily plays damage perform on the other two roles?” We then duplicated that question across all skill tiers, and the three different roles. This gave us some guidance on how to adjust players who hadn’t played a role yet. We also temporarily increase your variance level for SR gains and losses, so in cases we’re incorrect we can adjust you faster to the SR that matches your skill.

There will be no reset. Blizzard recently did an MMR reset for their internal test server, and it went about as badly as expected (00:15:40).

@ScottMercer

we can’t ever really consider implementing a full SR reset.

There will be five placements for each role (01:10:30). Playing all roles is not required.

There will be four leader boards, one for each role, and a combined one. To qualify for a leader board, a player must play 25 games per role (for a total of 75 if they wish to qualify for the combined leaderboard) (Introducing Role Queue - News - Overwatch, 00:23:45).

SR decay is gone (01:12:20).

@ScottMercer

The results of your matches in the Role Queue Beta are not being thrown out and will be used by the matchmaking systems to help determine your skill in season 18.

Below are some additional interesting topics.

MMR used to be one value per mode (now it is three for modes that have role-queue). SR has not, nor will be, used in matchmaking (00:08:40). However, Scott persists in acting like SR is what matters, even though it is not technically true. Frustrating.

Stats are used to figure out when players are throwing (00:11:40) .

Role queue will be implemented in quick play as well (with separate MMR) (00:31:45).

Don’t take PTR too seriously. It is there to check for show-stopping bugs. Match quality is poor, and MMR/SR is inaccurate (00:52:00).

Applicable arcade modes will have role-queue (00:53:50), but not Mystery Heroes (01:08:30).

Sigma is coming (00:56:40).

Hero bans are not planned at this time (00:58:00).

SR is not used in matchmaking (01:13:30).

MMR goes from -3 to +3 (plus a bit for extraordinary players) (01:13:40). That is, MMR is in units of standard-deviations (a.k.a. sigmas).

@ScottMercer

Something else to remember is that we as humans are not particularly good at identifying our own skill at something. This is referred to as the Dunning-Kruger effect, and if you’re curious to learn more feel free to use your favorite search engine on that term.

@ScottMercer

In the end, the best way for you to improve our matchmaker’s ability to place you with other people of similar skill is just to keep playing. You’ll sometimes have some lucky win streaks, or unlucky streaks of losses, but if you keep playing your SR will continually adjust to you reflect your present skill and you’ll play in better games.

[P00P-11780]

God I’m so happy for no decay. Such a pain that accomplishes nothing.

[Life-22371]

Good job writing that stuff together.

[Kaawumba-1133]

God I’m so happy for no decay. Such a pain that accomplishes nothing.

It never really affected me, because I’ve never been diamond. But it is funny how this would be a huge announcement on any other day. Today it’s just lost in the noise.

[GrizzGolf-1501]

So since I mostly tank in lower levels will this be more beneficial to me?

[Palpatine-21567]

MMR goes from -3 to +3 (plus a bit for extraordinary players) (01:13:40). That is, MMR is in units of standard-deviations (a.k.a. sigmas).

Like it has value of -3 to +3 by default?
Then it means the elo hell many cry about, is true?

[Kaawumba-1133]

So since I mostly tank in lower levels will this be more beneficial to me?

The primary purpose of this feature is to better allow people to play the heroes they want to play without breaking the team composition. Matches should also be more balanced, as even people who are nice and willing to flex are often poor at their non-main role. It should benefit most people. The only real tank/dps/support bias is that tank and support queues will be shorter, and dps queues will be longer. But if a player wants to play dps, and have a good game, they’d probably rather wait longer for that game.

It won’t make players better, and won’t stop trolling. So you’ll still have trouble with dps moiras and anas not healing you, and main tank and off tanks not understanding how to play together, for example.

MMR goes from -3 to +3 (plus a bit for extraordinary players) (01:13:40). That is, MMR is in units of standard-deviations (a.k.a. sigmas).

Like it has value of -3 to +3 by default?
Then it means the elo hell many cry about, is true?

No. Unless your definition of elo hell is a system where the median player is in gold, and we can’t all be GM together, no matter how well we play.

[GrizzGolf-1501]

So it wont base my SR off my tank skills? So even if I play lights out as a tank and we lose Im still going to lose my 20-30?

[Kaawumba-1133]

It will base your tank SR off of your tank skills, your dps SR off of your dps skills, and your healing SR off of your healing skills.

Typical loss/gain is 24. So if you lose less than 24 on a defeat, your performance was good.

[WindowMaker-11972]

Just checking I assume diamond and above are still not going to have PBSR

[Kaawumba-1133]

Correct. Grizzgolf was asking about the low ranks.

[DutchAres-2277]

MMR goes from -3 to +3 (plus a bit for extraordinary players) (01:13:40). That is, MMR is in units of standard-deviations (a.k.a. sigmas).

Like it has value of -3 to +3 by default?
Then it means the elo hell many cry about, is true?

No. Unless your definition of elo hell is a system where the median player is in gold, and we can’t all be GM together, no matter how well we play.

It is the reason why there is so much skill difference in most tiers?
Why the matchmaker fails so much (hope it will be less of a problem with role SR)
He talks himself about playing against a very bad tjorb and the next game a good one.

Ranking most players between -3 and 3. Then i hope certainly not that every one in bronze is -3, in silver -2 etc… I hope it is more like someones MMR = -2.39484 or something. And if you play well then this number will increase (probably compares you with others of your rank)

I would love to see this number after every game. And more info about how it calculated. Jeff calls it an unexcited number… I would like to see this number more then SR. SR is more based on your team performance(win/loss) MMR is all about your individual skill.

[GrizzGolf-1501]

It will base your tank SR off of your tank skills, your dps SR off of your dps skills, and your healing SR off of your healing skills.

Typical loss/gain is 24. So if you lose less than 24 on a defeat, your performance was good.

Awesome! Thanks Kaawumba!

[KSeven7-2463]

Ranking most players between -3 and 3. Then i hope certainly not that every one in bronze is -3, in silver -2 etc… I hope it is more like someones MMR = -2.39484 or something. And if you play well then this number will increase (probably compares you with others of your rank)

I’m fairly certain it does. It doesn’t quite make sense for a matchmaking system to only have only 7 set values (especially when there are 5000 SR points).

I would love to see this number after every game. And more info about how it calculated. Jeff calls it an unexcited number… I would like to see this number more then SR. SR is more based on your team performance(win/loss) MMR is all about your individual skill.

I doubt we’ll ever find out. SR at your true rank is close to your MMR so there’s no need to. They also won’t reveal what MMR entails in order to avoid people trying to abuse and stat pad in order to raise it.

[Mandalorian-11203]

MMR goes from -3 to +3 (plus a bit for extraordinary players) (01:13:40). That is, MMR is in units of standard-deviations (a.k.a. sigmas).

How’s this work, exactly?

For instance, is the matchmaker looking to make a match where each team is 0.

And to do that you’d, say, take 6 players that are something like…

3
2
1
-1
-2
-3

Balancing each other out to Zero? Then have both sides come to about the same number.

[Kaawumba-1133]

@DutchAres-2277

Ranking most players between -3 and 3. Then i hope certainly not that every one in bronze is -3, in silver -2 etc… I hope it is more like someones MMR = -2.39484 or something.

Yes, it is a floating point, not an integer.

@DutchAres-2277

I would love to see this number after every game. And more info about how it calculated.

I agree.

@DutchAres-2277

I would like to see this number more then SR. SR is more based on your team performance(win/loss) MMR is all about your individual skill.

No. This is persistent, but false myth. Both MMR and SR depend more on win/loss than performance (and above diamond, neither depend on performance). See How Competitive Skill Rating Works - Season 17 - #17 by Kaawumba-1133 → Summary.

[Kaaldrin-1481]

Please wait for other posters before replying.

[Kaawumba-1133]

And to do that you’d, say, take 6 players that are something like…

3
2
1
-1
-2
-3

Balancing each other out to Zero? Then have both sides come to about the same number.

No, all players in a (100% solo-queue) match have a fairly tight cluster of MMRs, perhaps +/- 0.08, so it’s more like, 0.06, 0.05, 0.01, -0.03, -0.06, -0.07 for one team, and similar for the other team.

[Vurg-1970]

No, all players in a (100% solo-queue) match have a fairly tight cluster of MMRs, perhaps +/- 0.08, so it’s more like, 0.06, 0.05, 0.01, -0.03, -0.06, -0.07 for one team, and similar for the other team.

I agree.

My understanding is that MMR 0 is basically the middle of Gold, and 1 is above that somewhere, maybe something like Plat/Diamond, with 2 being somewhere around master/GM, and 3 being top 500 or so (not that it matters exactly which maps to which, and this is wild speculation) A better grasp of standard deviation would probably say that I’ve got the gaps all wrong, but which number equates to which rank isn’t the important part.

This means that a particular match is probably something like a 0.85, 0.84994, 0.8501, 0.8489, 0.852, etc. But the main point is that they’re very close to each other, and only the whole population is centered around 0, not the participants in a certain match.

[Kaawumba-1133]

I’ve mixed in the new information from Scott’s recent posts at Role Queue Update.