3000+ Skill Rating Data and Analysis (now including DCs)

Competitive Discussion
1

Summary

  • The default SR movement is 24.
  • Above 4200 SR, there is a significant SR debuff.
  • Recovery from decay is a linear relationship between SR and hidden SR (NOT HIDDEN MMR).
  • Recovery from disconnects is the same as recovery from decay
  • This data says very little about hidden MMR, except for some unproveable speculations.

Introduction

So I went off and recorded 944 streamer games worth of data: SR, team SR, enemy SR, Change SR, Hidden SR (I’ll explain later), and Decay. This covers a wide range of account types: new accounts (Dmum 11, Rainbow 10, Platchat 11), and established accounts (Seagull 10, Seagull 11, Seagull 12, Fistscan/Symscan/OliveToHeal 11). Some accounts have decay, and some accounts don’t. One account (Pandaman 13) is actually below 3000 and not a streamer, but it has a disconnection, so it can be compared to the decayed accounts. The raw data is at Streamer Data - Google Sheets

I’m just going to share the data and talk about it. There is nothing shocking here, but I do this sort of thing for fun, so here goes.

No Decay

First, I combined all the data that had no decay.

From 3000-3600, things are extremely stable, with the preferred change of +/- 24. Low diamond is a bit of a sweet spot, where there is no performance SR, fair matches are easy to find, and there is no SR debuff likes happens at higher SR. So this +/- 24 should be thought of a the default change in SR, when there are no outside influences. The small variations from 24 are likely due to matches that are predicted to not be 50/50.

From 3600-4200, things are similar, but there is a wider spread of possible SR changes, likely because it is getting harder to make matches that are predicted to be 50/50.

Above 4200, there is a sharp knee, corresponding to:

Scott never said exactly how this worked, but now we can look at the chart and read off the numbers.

MMR should not be affected by the SR debuff. MMR should increase above 5000 or below 0 as required and not have any squashing as limits are approached. Otherwise, there would be a wide range of subtle bugs with matchmaking. However, Scott hasn’t confirmed this, so I’m speculating here.

Decay

The situation with decay is more subtle. Here, I essentially guessed what the relationship would be, until I found something simple (that is, linear) that explained all the data that I have.

What I did here is started from the pre-decay SR value and moved it with wins and losses following the formula from the previous section. This value I call hidden SR (NOT MMR). If below 4200, change is +/-24. If above 4200, gain is -0.0184 * hidden SR + 100.4 and loss is -0.0069 * hidden SR +3.5168. Decay (fom the chart) is Hidden SR - Visible SR. I then cancelled decay once visible SR got above hidden SR.

After doing this, I found the linear relationship between Decay and SR change shown in the plot, above.

All the instances of decay have very similar slopes. There is some variation (about +/- 25 SR in x-intercept. In theory, this shows an error in the magnitude of the original decay. But how could this be? With one exception, I knew exactly what the original decay was. My best guess is that this is an effect of a small divergence between SR and MMR, but that is just a guess. Another possible culprit is the uncertainty in my formula for hidden SR movement.

Disconnections

Thanks to Pandaman (Weird SR gains/losses - #9 by Pandaman-21452) I have clean data (brown in the plot above, not included in fits) indicating that disconnections behave the same as decay. That is, decaying 50 SR is exactly the same (as far as SR calculations go) as disconnecting once. Note that it takes about 12 games to recover from one disconnect, and if you disconnect too often you can get suspended, so it is not to be taken lightly, even though it is unlikely that disconnects affect MMR or have any long term consequences on SR.

13 Likes
2

It wouldn’t be all that hard to make the benefits mathematically approach 0 as Sr reaches 5000. What are these necessary bugs that would happen?

3

MMR is used to make matches, in a way similar to (but more complicated than) the Elo rating system from chess. Elo rating system - Wikipedia. Systems like this make the assumption that a difference in MMR represents a quantifiable difference in win probability. If MMR started to get squashed when approaching a limit, this would break that assumption and lead to poor predictions about win likelihood, which in turns lead to poorer matches and incorrect MMR rewards for win/loss.

4

My data from two seasons confirms the default SR change around 24. In my case, I gained on average 23.3 - 23.5 and lost -22.8 -22.6 on average. I wonder, why it is not exactly 24, and why I on average gain almost 1 SR more, than lose, but can never change rank.

2 Likes
5

Below diamond there is performance SR. So it looks like you perform slightly worse than average on wins and slightly better than average on losses.

If you lose just one extra game, that will wipe out 64 games of grinding at 50%. Winning more is vastly more important than performance SR. And since games get harder as you rank up, you can’t really rank up through grinding and performance SR alone, even if you had an infinite amount of time.

6

I always believed the average SR you got from a game was 25 not 24 lol. Of course that doesnt take into consideration if you queue into a plat or masters game as a diamond because then you’ll gain more or less.

7

I wonder, how can this be. Maybe (maybe) it has something to do with flexing into tanks and healers, since DPS is always taken. Stats hunting with tanks and healers is painful.

This is a very sad thing. It means, either I suddenly start playing way better, than I did before (which, ironically, I did), or - playing comp does not matter, besides placement matches. Anyway, we’ll see the results when I place my new acc and play some games.

1 Like
8

I’m not sure exactly what you are saying here, but the way you rank up is by getting better as a player, which allows you win a higher percentage of your games, and then playing enough games for this new win percentage to be effective.

1 Like
9

Don’t stat hunt. Just focus on the game and try to win. Even if your rolling/ getting rolled, just focus on yourself, and don’t focus on trying to get that extra objective time or extra kill when you should be on the point. How i see it that you might be able to climb a bit by cheating the system, but why do that, when you can just work on getting better to climb.

1 Like
10

I looked at Kabaj’s data. Blizzard needs to remove PBSR for lower ranks!!!

11

Define “better”. After that we can continue this discussion.

12

What makes you say that? Did you just look at Kabaji’s stats or does he have some data that he keeps that you looked at?

13

Increasing your skill at the game. Not getting better at cheating the system if that’s what you thought i meant.

14

In a way that allows you to win a higher percentage of your games. I’m going away from keyboard for a while.

1 Like
15

My game data from 2 seasons makes me say it. I can even shout it out loud.
I’ll give you and example.

https://imgur.com/a/yQFA4LX

My games with Dva, same SR, very close performance stats. PBSR bonus completely gone. Why? Because of team SR difference. These SR fluctuations due to factors out of player’s control should go. This is what makes the community so toxic in large part.

16

I think the SR difference should play a part, how big? I don’t know but it should be a factor. But usually its your SR compared to the average SR of the other team. That being said, it looks at your stats PER 10min, so if your stats are similar but if one game was 5 minutes longer, it would make sense that even if your stats were slightly worse you gain more because you had better stats per 10 minutes.
Also, what does the bl mean in 2000bl and 5900bl mean?

17

If you have a look at Kabaj data from Kaa’s spreadsheet, you will see - team sr difference has little to no impact on resulting sr. And this is the way it should be. Think about it: did you choose those teammates and enemies? Did you specify desired sr range beforhand? Did you define grouping between ranks allowance? No. Then why these factors should nullify your personal impact?
2000bl means “2000 damage blocked”, 6900bl is “6900 damage blocked”

1 Like
18

awesome work! thank you. do you have any idea how many points are normally attributed to PBSR. I would say at most 4 points on a 24sr win/loss.

My best guess is, if you have an above average game you’ll get an extra 1 point of SR but if you have a really really good game, you’ll get 2 bonus SR points. Thoughts? thanks

1 Like
19

We’ve talked about expected win percentage and the effect on SR at length elsewhere (How Competitive Skill Rating Works (Season 11) - #61 by Oldphardt-2154 post 61 and following for those who may be interested). I don’t have anything new to say on the topic.

When Blizzard removed PBSR for 3000+, there was a lot of agitation to remove it for everyone. The requests were ignored. But then, Blizzard often does implement what the community wants … eventually, so we will see.

And of course, the community is not of one voice on this. There are certainly people who complain that win/loss is the most important factor in ranking up.

My opinion is that performance modifiers do help move new players where they belong much faster than an Elo type system, so I propose that the performance modifier be removed for Bronze to Platinum players after a sufficient number of competitive games has been played. Blizzard can use their data to determine how long it takes for people for their rank to stabilize, which would be a good place for a cutoff.

2 Likes
20

I have trouble quantifying it, but roughly +/- 4 SR. For people in high bronze to plat, finding fair matches is not difficult, so most deflection from 24 is likely due to performance SR. If you are misranked, I suspect the value can get larger, but I don’t have any data.

1 Like