Season 9 Placements Analysis, High Variance Explained

Competitive Discussion
1

Too long, didn’t read

New Season Start = (-173.7 +/- 20.8)+ (1.0015 +/- 0.0059) * (Previous Season End) + (37.94 +/- 2.23) * (Placement wins)

Unexplained placements far from this occur only for inactive players. Inactivity causes a soft MMR/SR reset. That is, when a player returns from a long break, his MMR picks up where he left off, but it is much more volatile.

What exactly defines an inactive player is ambiguous, but a placements only player certainly qualifies.

Introduction

I’ve been keeping track of placements data since season six, and it is available here: Season Placement Data - Google Sheets

I finally got around to analyzing the season 9 data. Life has been busy, and I’ve been watching too much Overwatch League.

People were helpful in providing new data. Thank you to everyone who assisted. The raw data from season 9 is at Season 9 Placement Results. Specifically, many people included their record from season 8, which I’ll come back to in a moment.

Combined Data

To the data! Here is the combined analysis, for season 6 - 9. To summarize, the independent parameters are placement wins (a draw is half a win) and previous season SR. The dependent parameter is new season SR.

It is a decent fit, but with large number of outliers. With outliers removed (at 2.5 sigma), the fit can be described as:

New Season Start = (-173.7 +/- 20.8)+ (1.0015 +/- 0.0059) * (Previous Season End) + (37.94 +/- 2.23) * (Placement wins)

Basically, for the average player, a win (instead of a loss) is worth 37.94 SR, which is a relatively ordinary (though a bit small) gain from a win.

With outliers included, the new season start has a standard error of 179 SR, which is huge. The corresponds to roughly +/- 500 SR that is not explained by the fit. This confounded me for quite some time, and I came up with various “creative” theories to possibly explain it.

Then, I recalled Scott’s statement from long ago, “Play a lot of games, it (MMR) gets more certain. Don’t play Overwatch for a while, it gets less certain. The more certain the matchmaker is about your MMR, the less your MMR will change in either direction based on a win or loss” (Overwatch Forums). I started to suspect that this was the explanation. And that everyone whining about huge swings had neglected to mention that they only play placements or slightly more than placements.

New Results

So, let’s look at the shiny new season 9 data:

This looks similar to the combined data, but much more sparse, because there is a lot less of it. However, what happens if we plot fit error vs games played in season 8?

Ah ha! If less than 100 games are played error is larger. If less than 50 games are played, error is much larger. If only 10 games are played, error is huge.

What this means is that taking a long break is equivalent to a soft rest of MMR/SR. That is, when a player returns his MMR/SR is highly mobile for a time (though the actual value probably starts where it left off…I’ll try to take data here, but it will take seasons for a robust result).

Aside: This cannot be explained by behavior in quick play. Quick play has no effect on competitive MMR/SR for new (Initial Competitive Skill Rating, Decrypted) or established (Overwatch Forums) accounts.

Aside 2: Those with sharp eyes will notice an outlier in the final plot. A player who played more than 200 games in the previous season, but had an error vs the model of about 150 SR. My best guess is that this corresponds to a player who cranked out all his games early in the season, then took a long break, then did his placements in the following season. That is, that amount of time off is the actual important parameter, not how many games were played in the previous season. I asked this player for a confirmation or denial here, but he did not reply.

4 Likes
4

I feel like you might be overthinking it.

5

TL;DR version up there.

Also, Can confirm. Made it from Bronze to Gold in like 50 games. Almost all placements, over several seasons, playing with my 6-stack in quickplay and getting significantly better…now I’m on my way back down because I haven’t been playing much.

I might get an alt called NightOwl so I can play tired and not feel bad. The game’s too fun to take so seriously; on the other hand, I’m not trying to throw by playing after a long day, then stomping on a day I’ve gotten some rest…not really relevant to the thread though…

Good work again, Kaa.

1 Like
6

I have an alt account for practice, messing around with friends and what not. Best $20 I spent from a gameplay perspective honestly.

7

Great post, as usual.
I also think, that the rule of “no less than 100 games played” applies not only to placements, but to season SR, as well. In my personal experience, SR movement in the first 100-150 games is awkward.

8

Since this is an old post but if you are reading this I’m just curious how that happened. If you were in fact getting better and there was a soft reset then shouldn’t you be moving up or at least stay the same rather than moving down? Thanks.

9

Skills degrade without use, my friend. Though I finished placements near my career high for this season, very surprisingly given that I no longer know the full capabilities of half the cast.

When playing QP but allowing the “uncertainty reset” to occur for comp the skills are maintained or improved. When not playing at all the “uncertainty reset” drops you like a rock.

I think I benefited from being matched with people that were basically in the same situation as I, my placements had almost no ranked people even with only 17 days left in the season. That, and I was able to create a decent group for several games with the new LFG.

1 Like
10

I think he’s putting just the right amount of thought into it.