How Competitive Skill Rating Works (Season 11)

so how long do you have to play without leaving to go down one tier. If I leave once, I’ll get a 10 minute ban. How long, or how many games do I have to play for my next ban to also be 10 minutes? Same with 8 hour ban?

Once I get an 8 hour ban, how long or how many games do I have to play for my next ban to be 8 hours and not 24?

Blizzard hasn’t said, and testing would be painful and probably against the terms of service.

1 Like

You certainly put a lot of words in that reply…
I don’t know, why do you disregard applying trends to data. This is what they are for - to counter randomness and highlight TREND, no? Even if you remove the trend from graph - the tendency is still there.
And - think about it, please - teams are not composed randomly. You have MMR, players’ SR range, ping and probable win chance for every team. This is as far from random, as it can be. And the teams are constructed, having in mind a goal - to respond to player’s MMR vs SR progression.
And - and it is totally my personal observation - my fall of 500 SR is constructed by placing me on teams with less average SR.

Blizzard doesn’t publish detailed information about how the Competitive Play system works, because they don’t want players to realize that it is handicapped with Match Making Rating. Nothing in Overwatch’s user-interface even mentions MMR.

Blizzard would rather grant special permissions and authority to forum community randoms like Kaawumba, than to publish information like this themselves. That is so they can avoid the liability of misinforming consumers and deceiving them about their terms of use for Competitive Play. Frankly, this should be illegal.

Not really, no.

The correct trendline is great, but using the wrong one or one that doesn’t fit the data is really, really bad. You can’t just throw a line in there and call it good. Besides, the moving average isn’t a trendline.

Only 60% of the games fit this characterization and you were only 5 SR less on average. You can’t seriously think 5 SR is a meaningful number.

But if you feel persecuted by the Blizzard team then there’s likely nothing that can be said to stop you from feeling like that.

My “special permissions”, that is, my forum level 3 privileges, were achieved by automatic algorithm (basically being active on the forums without being suspended) and are accessible to anyone. No Blizzard employee has given me special treatment or endorsement.

Scott has published this kind of information himself, his most recent post being here: Groups and Matchmaking in Overwatch. My effort is to curate and organize various developer statements, as well as gather and analyze data to expand upon what developers have said. But if you ignore everything I’ve said and just read Scott’s last post, you’ll get most of the system.

Well, it’s not handicapped. So nothing they say is going to confirm that it is handicapped.

Just because you don’t like something, doesn’t mean that it should be illegal. The government should focus on important and serious problems that it is best equipped to handle, not video games.

There is no model or program that would be able to catch the “rigged” matches in Overwatch. In order for a pattern to be flagged as something strange it would need to be completely out of the ordinary, here’s an example:

wwwwwwwwwwwwwwlllllllllllllllllllllllllllllllllllllllllllllllllllwwwwwwwwwwwwwwwwwwwwlllllllllllllllllllllllllllllll

and even with the above data it might not catch it as “out of the ordinary”. On the other hand, the artificial win/loss data below would not be caught as fake:

wwwlwlwllllwllwllwlwlwlwlwlwlwlllllwllwlllwlwlwlwllwlwlwl

even though I just played the drums on the “w” and the “l” key randomly. Someone presenting data that they ran through a program and saying “everything checks out” means almost nothing. Your best bet for understanding how the matchmaker works is to observe it with an open mind and look for patterns that are obvious. When they stand out then obvious is obvious.

You are claiming the special ability to detect patterns that no scientific analysis can, while humans are notorious for detecting patterns where there are none. Color me skeptical.

With a 500SR swing, do you mean 500 up and down, or 250 up 250 down?

Which section or post are you referring to?

I looked at this again, and realized that even though there aren’t very many games here, it did look suspicious. So I ran it through the analysis described at Overwatch Forums and found that this data definitely gets flagged as rigged. Specifically, there are way too many streaks of length 1.

Win/Loss Simulation and Data - Google Sheets shows real and realistic simulated data.

Your simulated data looks like this:

More than 60% of the streaks are of length 1, while it should be 20-30%.

This excessive number of short streaks shows up here as a excessively high tendency to lose after you’ve won, and vice versa.

1 Like

The video game industry is one of the biggest entertainment industries in the world, if not the biggest. Lack of regulation has consequences for all of us. And Competitive Overwatch isn’t Blizzard’s first crime against humanity. Just look at the billions of hours (trillions?) that people have wasted on playing World of Warcraft. All of Blizzard’s games are designed to be as addictive and time consuming as possible, regardless of the experience they’re delivering.

SR accuracy

There are number of ways to approach this question. One is to start a completely new account, and then play 100 games on the new account and see how it performs compared to the old one. This shows that SR can vary by 1000 SR in extreme cases, and 500 in normal cases (27). There is some evidence that reroll experiments show less variance at higher ranks (28) likely because there are less random variables, such as smurfs, throwers, and inconsistent play.

Next, any player can see how his SR changes during a season. A range of 500 is completely normal here.

Off-topic, but…

To be fair… I do not want the government to over-involve itself with video games (again). Some parts of the world are kinda cucked from playing certain games because its not low enough on a rating to pass and what not. I’m all for being somewhat anti-lootbox shenanigans and scams coughBattlefront 2cough, but yeah…

+/- 250 in normal cases. +/- 500 in extreme cases.

That’s interesting, in that context, do you have any idea/explaination for my experience? Being that every season I have played seriously I would slowly climb up to 3400 or so only crash down to 2200 or so, and the cycle would repeat again.
My SR diagram looks like a sawtooth.
I do tend to climb just a little higher every time, pushing my ath just a little bit every cycle, but such huge SR waves can’t be good right?
To me that just signals that the SR in this game is wildly inaccurate.

I would really like to know what causes this behavior, or at least understand it.

1 Like

The amount of swing varies based on play style and hero choice. Your profile is locked, so I can’t see your hero choice. However, it is common for players with such high swing to be Mercy mains. Mercy is a problematic hero, because she depends so much on her team. Carrying with Mercy is difficult at best. This means that most games you are flipping a coin to see what teammates you get and whether or not you will win or lose and you can only occasionally flip the outcome with your own skill. This leads to higher variance in skill rating.

As to early season being easier than later season, I have noticed the same thing, though my swings are less dramatic. I have theories, but I have no way to prove them.

  1. One possibility is that casual players play early season till they get frustrated and quit. This leads to skill creep over the season.

  2. Another possibility is that high level players play their mains early in the season until they reach their skill cap, and then they boost up smurf accounts. That means that anyone who belongs in plat, for example, will have a significant head wind.

  3. Late season people have reached their competitive point goals, and are more likely to screw around, leading to more randomness in matches.

Whatever is going on, two month seasons reduced the problem from where it was with three month seasons.

1 Like

I’d like to add something about the difference between accuracy and precision, if I may. The common usage mixes the terms but to discuss what I want to discuss I need to separate the ideas.

If I were to tell you that my location was California, that would be accurate, but not very precise. Similarly, to say that your skill is between 2200 and 3400 also is likely accurate, but not very precise.

You are talking more about precision. The question then becomes one of the interface between you and the measuring system (SR). If you were to try to find me, I could likely give you my address and that would be sufficiently precise to accomplish your task. What I couldn’t do is to just give you my zip-code. You’d never find me that way. That wouldn’t be precise enough. On the other hand, I could also give you a 12 digit latitude/longitude coordinate, there are literally billions of those coordinates that would accurately describe my position and any one of those would work as long as I didn’t move.

If I was walking around my house, or even doing yardwork, by the time you got to the 12 digit coordinate it would likely be wrong. You’d be better served by the address.

Kaa’s experiment in reference (27) assumes that the precision is 4 digits and thus discusses accuracy in terms of the fact that the two accounts were 476 SR different. He’s not wrong (I mean, SR is 4 digits, it’s most reasonable to conclude that that precision is meaningful) but he would be able to say that the system gave the same (accurate) results if both accounts ended up in Gold tier. If we consider that SR is likely over-precise then the question of accuracy become a bit different. How different is 476 SR really? The answer probably depends on where that 476 range is on the overall scale.

This is where your hero and playstyle come in (and perhaps the other factors Kaa gives). If you, personally, aren’t that consistent or you play in a way that isn’t that consistent, or you play a hero that relies on factors beyond your control, then what you are doing is very much like moving around your house while someone is looking for you with a 12 digit coordinate. You keep updating the number, but a few games later it’s wrong.

Measurement systems should never be more precise than they are accurate. You wouldn’t want to use the 12 digit coordinate to find me in my house. It would only be right by sheer dumb luck. If you look at coordinate x,y but you only find empty space, you are simply inaccurate. Wrong. You of course want them to be meaningful, using a zip code wouldn’t work, but generally it’s important to match your precision to your desired accuracy.

The SR system fails dramatically in this respect. I’ve never heard someone that claimed they could tell a difference in skill level below a 200 SR difference. There are 5000 SR levels currently, so we could just take 5000, divide it by 200, and get essentially 25 meaningful tiers of game play in the MOST PRECISE version. According to Kaawumba’s experiment, we should take that 5000 and divide it by 500 to 1000. Consider if it is meaningful to have an SR difference of 1. If not, there probably shouldn’t BE the possibility of an SR difference of 1. It means absolutely nothing.

In a sense, we already have these tiers. If you forget the over precise 4 digit number and understand your rank only in terms of tiers, I think you would find your results to be a bit more accurate but THEN you have to consider not just YOUR consistency, but what “2500” or “Platinum Tier” even MEANS. The 3 possibilities Kaa just posted are all variations on the same theme, which is that the skill level that “2500” corresponds to may potentially change throughout the season. It’s a ranking system. A RELATIVE number. You’re not a Platinum player, you’re a player higher than the people in Gold but not quite as good as those in Diamond. It may seem like a pedantic difference, but it will help you to understand.

So even it the system was perfectly accurate and precise to 4 digits and you played perfectly consistently, it’s still possible for your SR to go up and down through the season. Not that it necessarily would so dramatically, but it could certainly be a factor.

Hope that helps you to understand it.

I play almost every hero, my most played hero is phara though.

SR is very accurate. On my original account (this one) I have only made it above <500 once. On my other account I am high silver low gold at the start of the season. So within 2000 SR means it’s accurate, right?

1 Like