How Competitive Skill Rating Works (Season 9)

The part Moocow is talking about

People who fake data without any prior mathematical knowledge of odds tend to stop runs at 4 or 5 H’s or T’s. In reality, the longer you keep flipping the coin, the more and more likely it is that you will get longer and longer streaks. A run of 200 coin tosses reduces the probability of NOT getting a run of at least 6 to only 3.47%. The professor simply divides the papers into two piles depending if they have a streak of 6 or more and those without any streaks. To a high degree of accuracy the ones without streaks are fake data, and the ones with streaks are real data

It actually says the opposite of what you claimed. The chance of not getting losing streaks is much, much smaller than what most people intuitively believe. That’s not quite saying

but if people underestimate the chance of streaks in 200, they’re also underestimating the chance of streaks in 20.

The thing about science, and those that are scientifically literate, its that, contrary to popular belief, WE WOULD LOVE TO SEE DATA THAT SHOWS OUR THEORIES TO BE INCORRECT because more than anything we want to know what is real.

If you, or anyone, has actual data that shows anything but what the devs have so far claimed that would be welcome news.

So far the only people that have the data reinforce this thread, not the others that disagree with it. They claim all day their experience is different, but oddly no one writes it down for independent verification. I dunno, maybe Blizzard knows who is scientifically illiterate and punishes them with SR losses.

It’s not that it’s well organized or wordy. It’s that it’s backed up with data that is referenced for anyone to verify. That’s the key to believing what is true vs. what is false. Verifiability. Not feewings. Especially not your own subjective experience.

1 Like

Bamping again lol…

Data, statistics, evidence?

/gasp

but muh teammates / bliz fixes muh gaemz / i dersrv GM / etc.

Blizzard should hire this man, srsly.

1 Like

Quickplay appears to be designed for…

Quick play. There seems to be “tiers” that people get placed into with a very broad range of player ability. Grouping up appears to make it want to mirror the group mismatch (Master with Silver, for example, will attempt to find another Master with Silver). But as for SoloQ completely, I noticed while in low Diamond I’d have a couple high plat, a few diamond, and lots of random master/GM players. The games were definitely not as balanced as comp… but then again, QP invites lack of team comp, team play, etc. and with say, a GM Tracer, that’s like an open buffet.

On this account, I’ve noticed my QP, without groups, cuts off at high diamond, and I see gold sometimes. It’s mostly Plat/Diamond. Master+ seems to be groups only currently.

If Jeff has confirmed QP doesn’t influence MMR for comp, then it must mean it’s extremely accurate on predicting your MMR within those 10 games … which seems unlikely? I got placed high plat/low diamond on this account (my main is high diamond/low master). Is our individual metrics that predictable of rank? Or is it taking a guess based on QP stats to set an initial “test” for placement matches and moves you < A LOT > based on your individual metrics? Or is that whole “QP MMR =/= Comp MMR” include ANY data from QP?

I’m interested.

Your conclusion “then it must mean it is extremely accurate … within 10 games” does not follow from your premise “If Jeff has confirmed QP doesn’t influence comp”.

The reverse is frequently observed. Players that belong in bronze are placed way too high initially. Players that belong in GM+ are placed way too low.

I’ve done two reroll experiments. In the first, my rating jumped substantially from my main to gold (my main was silver at the time): https://us.battle.net/forums/en/overwatch/topic/20758807010

In the second, my rating stayed about the same in gold (my main was gold at the time): https://us.forums.blizzard.com/en/overwatch/t/initial-competitive-skill-rating-decrypted/31877

I’ve edited the original post. I’ve added the analysis for season 9 placements (Season 9 Placements Analysis, High Variance Explained).

I’ve added a section to popular myths on “SR is taken into account on matchmaking”.

I’ve added references to the the summary section, “Summarize matchmaking, rating, and progression for me”.

Both threads could be easily resolved if Blizzard just released the series of calculations they use for the MMR system. No more no less. No more weird conspiracy theories or whatever. Just tell the damn playerbase how it works. end of story.

I don’t understand why this is so difficult for Blizzard?

That’s the thing though.

People don’t even believe what they have given us. The threads against this system all boil down to one thing…don’t believe what Blizzard tells us, they’re a company and companies are evil.

For Diamond and above, it’s actually a pretty clear system, though they haven’t put it into one long sheet and answered questions about it. That’s why Kaawumba put this guide here.

You’ll note that there are 47…47 references. That’s not exactly keeping it a secret. They note changes. They tell you that it finds people with similar MMR. They tell you that MMR is primarily calculated on wins and losses.

People climb, both Bronze to gm AND just plain everyday folks getting better.

In the end, looking deeply into the MMR system is a curiosity for some and a scapegoat for others. It’s really not relevant for any other reason.

The problems with comp aren’t that it can’t find people of similar skill. It’s that it doesn’t find people to make a cohesive, coherent, and cooperative team.

1 Like

If the matchmaking could identify players that prefer certain roles (and actively utilize them to uphold their current rating) matches would be a whole lot more balanced on average. The reason people are yelling out for role queing is because it gets tiring to see 3-4 people on a 6 man team prefer the same 1-2 heroes, or role.

This essentially makes a team an underdog regardless of what SR people are at.

That’s a big “IF” when you’re talking about human behavior. This is not a problem for an algorithm. It’s a market problem. Trade-offs will have to be made and no one will be satisfied with a computer making that decision for them.

Some people will be perfectly happy waiting 30 minutes to play DPS. Others just want to get in the game. Some people will do both at different times. The trade off is simple, but a system needs to be designed that allows us to make that trade.

I’m certainly in favor of more transparency and a simpler system.

However, evidence suggests that we would still have conspiracy theories. Take a look at Forced Losing Streaks are Real - #624. Prophet has carefully constructed a non-falsifiable hypothesis. Even if Blizzard open sourced their code, I expect he would just claim that Blizzard was using code other than what they released.

I’m an example that the system behaves the way its “supposed” to. I was sort of okay at the game. Spent awhile getting better. Slowly ranked back up to silver, then gold, then played through platinum and touched diamond. Took me seasons from deep bronze like 500SR.

Proof the system doesn’t work - grouped with friends - one of us got gold, one of us got silver, one of us got diamond (this was in 6v6 competitive arcade). Blizzard, do you really just have a random number system? Because this points towards that strongly. You need to RESET everyone’s SR. If this 6v6 three tier difference placement isn’t evidence you guys are doing it wrong…

I don’t really follow the arcade competitive modes, but:

What are these three players SRs in normal competitve?
Are they active players in normal competitve (at least 50 games a season)?
What were their starting SRs in 6v6?
Did you play all the placement 6v6 matches together?

My goal here is to understand the system, including finding problems when they can be verified.

1920, 2215, 2700 (a fair note - the 2700 received his rank grouping with me - I dropped from unfortunate, but decent, matches).

They are all active in competitive, well over 50 games a season. The starting SRs were 1980, 2174 and 3101.

Yes, we were grouped.

I will gladly answer any other questions you have. Hopefully Blizzard will take notice for once.

I’ve had a similar situation occur, where I solo qued to and from Plat, and deranked to mid gold, then got destroyed in placements awhile back. I had a lot of leavers - and this negatively effects everyone else. Fair? No. Is there a fix? No. Does it happen to us all? Yes. Should it? No. I’m not one to complain, and it took me 9 seasons to say anything, but this is becoming ridiculous.

2 Likes

This implies that the 6v6 rank is seeded by normal competitive. The 2700 -> 3101 is a bit odd though. Maybe a performance modifier effect?

Assuming that I’m right that 6v6 being seeded by normal competitive, it would likely be because it is very difficult to find people’s true ranks after a full competive reset in only a few weeks of play.

I don’t think it’s that odd honestly. If it’s anything like capture the flag was, it basically takes your MMR from comp as a starting point, but without the certainty level attached to it. So, the fluctuations can be pretty heavy. Had one account place 4-500 above rank, and another that went below by nearly as much.

1 Like

That would make sense. Thanks.

2 Likes

So we have an answer:
MMR. Now, if only they could show us our MMR and what “rank” that stands for… that way we can avoid confusion in the future. To keep the system fair, every so many seasons, MMR should be reset, given that the majority of players are in Platinum and lower. Diamond consists of 10%, Master 5%, and GM 1% of Overwatch Players. I found this under the forums posted by a Blizzard employee, for reference (it won’t let me put a link in for some reason).