SR's biggest problem: A Mathematical Proof

Competitive Discussion
1

Ok, so first off, I am a PhD math student, so I may have a unique perspective here, but hear me out.

The SR system may have a fatal flaw – SR can be thought of as an economy, where it is traded back and forth from person to person. However, like any economy, it must have ways for SR points to enter the market (like printing money) and ways for SR points to leave the market (like old, or otherwise destroyed dollar bills). These two forces must be, on average, equal for the currency to maintain its value.

Consider SR:

  • How does SR leave the system? i.e. how can you lose SR without some else gaining it? Simple: you leave a match either accidentally (DC) or on purpose (throwing). AND high-end SR decay over time.
  • How does SR enter the system? i.e. how can you gain SR without someone else losing it? Placement matches. Draws used to do this, but not any more.

Now, if we are to assume that we all place properly – i.e. at the SR we deserve – then we cannot consider placement matches to be SR entering the system since it also signifies a consumer entering the system who, rightfully so, owns the SR and should not, on average, lose it.

Now, consider the usual random walk – this is a stochastic process where X(n) randomly walks along the integers. One particular realization of this process may be that X(0)=0, X(1)=1, X(2)=2, X(3)=1, X(4)=0, X(5)=-1, etc… The idea is that X(n+1)=X(n)+1 with some probability, and X(n+1)=X(n)-1 with some complimentary probability.

There are a number of analytical results one can prove regarding random walks, they can be found in almost any stochastic processes book (Ross writes some good ones). Namely, if P(X(n+1)=X(n)-1)>P(X(n+1)=X(n)+1) – i.e. it has a higher probability of decreasing than increasing, then the random walk will, eventually, tend to -infinity and with probability 1, X(n)>k will occur only finitely many times.

Now, consider the total SR in the competitive system (i.e. the sum of all players rank). Let n index all matches beginning from game launch till now. i.e. game n+1 is the next game played after game n. Then SR(n) forms a random walk over the integers with irregular steps. Since there’s no way for SR(n+1)>SR(n) – i.e. SR cannot enter the system – then its trivially true that this random walk will tend to -infinity. However, it’s bounded below by 0, therefore it will tend to 0 and reach 0 in a finite amount of time (remember finite can still be large, so its the trend we are more concerned with).

What happens if SR(n) tends to 0? It would mean that we all, on average, derank regardless of skill or match record. Now, as any good statistician will tell you: averages say something about a population (i.e. OW players) but say nothing of individuals (i.e. a particular OW player). Just because we are, on average, deranking does not mean its impossible to rank up. It merely means that the odds are stacked against you AND that constant performance will not mean constant SR.

Thoughts?

10 Likes
2

Leaver penalties.

But, you have a bigger fundamental problem with all the math that I glossed over because, well, this flaw underminds your own thing.

It isn’t.

SR distribution is on a bell curve. Where is tries to group everyone in the middle… and the outlayers (really bad and really good players) are pushed out to the edges. If it was an economy, it would be possible to break the caps in both 0 and 5000.

The other problem, is that SR is gained or lost on an individual basis. Someone can gain 100sr, but that doesn’t mean someone else or the enemy team as a whole lost -100sr to make up for this, nor does it mean their teammates get less SR.

SR and MMR is a placement system designated for the UI to display a number as to where in the community said person places in regard to skill. If the computer gets confidence that they are say, better then all gold people on avg… they will place the person high until they get to people where they will start losing to more then they win.

In a REAL competitive system, there would be an SR economy. There would be games that are really close, and the player wouldn’t gain any SR… and blowouts one way or another would adjust things dramatically.

OW competitive, like QP is just simply a matchmaker that is designed to find fair and equal games. It is based on in-game stats, a lot of which do not explain roles and plays in the game that determine win or losses.

It is a highly flawed system. If math could figure it out, it would have been cracked over a year ago. But it can’t because it is an arbitrary number that doesn’t mean anything except thats where it placed you.

6 Likes
3

There. Post above explained.

Blue team SR wins =! Red team SR losses.

Math fist anyway!!!

4

  1. Empirically, there is no significant inflation or deflation of SR, so your model must be wrong. For season 3, the rank distribution was Bronze: 6%, Silver: 22%, Gold: 34%, Platinum: 23%, Diamond: 10%, Master: 3%, Grandmaster: 1% (Overwatch Forums). For season 8, the rank distribution was Bronze: 8%, Silver: 21%, Gold: 32%, Platinum: 25%, Diamond: 10%, Master: 3%, Grandmaster: 1% (Competitive Mode Tier Distribution).

  2. You’re making a lot of mistakes on how the system works. See How Competitive Skill Rating Works (Season 13) for a more complete understanding.

Specifically:

The real rating is MMR, not SR. Decay does not affect MMR. It is quite likely MMR is always symmetric (one person gains what another person loses), like it is with Elo in Chess, but this has not been confirmed.

When recovering from decay, winners gain more SR than the losers lose. Leaving may behave similarly to decay (I don’t have data here).

Initial placements add MMR to the system, but that is not the relevant quantity. MMR per person is. MMR per person does not appear to be changing, so it is likely that new players are put in in such a way to prevent inflation/deflation. At least to first order, new players are put in at the median rating (Initial Competitive Skill Rating, Decrypted), but it is possible that Blizzard messes with this number to deal with people quitting the game and throwing the MMR per person off.

Blizzard could also readjust the MMR economy at placements each season, but I’ve seen no statistically significant evidence of this (Season 9 Placements Analysis, High Variance Explained).

8 Likes
5

Your theory leaves out PBSR and only works above diamond. And, yeah, at diamond+ you trend towards deranking as you need at least a 50% WR to maintain your rank

6

I’m pretty confident that draws never altered anyones SR.

1 Like
7

Wealth is also distributed normally. Actually, according to the central limit theorem, anything that is the result of Bernoulli signals (yes/no) will be normally distributed in the limit. This includes SR yes, but also economic wealth. The distribution of the individual was not the point, but rather the limiting behavior of the system.

Boundedness need not play a part, since economies need not be unbounded in general. Actually, economies in practice are trivially bounded both from above and below by the containing counties debt and wealth – if this does not work as a bound, then consider the worlds debt and wealth. Surely this bounds the individual and is finite – do not worry so much of the 0 and 5000 numbers, while these are bounds they are simply parameters that do not affect analytical results, provided that those analytical results take into account boundedness, as I did.

This may well be true. I find I usually lose more SR for a loss than I win, but
this is anecdotal and not necessarily true for everyone. Perhaps SR can enter the system this way: by giving more to the winners than taking from the losers, on average. Point well received.

This assumes EOMM (Equal Opportunity Match Making) – there are a number of papers published on this. For Overwatch, it’s likely not the case that it uses EOMM. This is because probabilistic studies show that EOMM is the least optimal match maker when seeking to optimize player engagement (play time). Just google: 1702.06820. look for the first paper link, entitled: “EOMM: An Engagement Optimized Matchmaking Framework”)

Mathematicians love arbitrary numbers – its what we wake up to study most of the time. :slight_smile:

1 Like
8

It is actually more likely that it is not. Mostly because MMR is hidden, there probably isn’t and increase or decrease that is symmetrical, because the stated purpose of the matchmaking system is to find equal games.

This can be seen on the anecdotal evidence we’ve gotten from several people at how uneven SR gains are and change.

If you play at ranks higher or lower then your SR… your SR and MMR will gravitate toward those SR’s as the game loses confidence into where you place. This happens frequently in groups. The purpose is so that the game will place you closer to your friends. If you can’t carry your low friends, it will reduce your MMR to reach that of your friends.

When you stop playing with your friends and go back into solo q, your MMR is all off… and you either start winning or losing heavily.

The system also artificially tries to pull everyone toward 2500. The avg gains get less and the avg losses more the higher above 2500 you get. This is built into the base SR number before the MMR calculation is taken i to account.

I believe there is an additional ratio built into the SR gains/losses depending on it trying to pull your SR toward where the game believes you should be according to the population of other players.

This is why players that have high win rates will see very drastic changes as it tries to keep you down to where your MMR is… until you start beating people the MMR doesn’t think is possible. Then your MMR will shift as it tries to find out where it should place you in regards to others playing the game.

You can really mess up the system and keep your MMR in place when you beat people it thinks you should lose to, and lose to people it thinks you should beat, but you end up having a higher overall win rate… so you SR starts to climb because it has to for the UI, but it gets far away from your MMR.

This also explains why people can “feel” like they are getting bad teammates and the system is trying to make them lose games. If there is some level of the SR number actually playing a roll in the matchmaker, but the confidence number is wrong… it will somehow try to find a way in the game to make a fair individual match, while also still showing your number being high. It might also be placing you with people who’s confidence rating is off. As a way to try to fit you into where it should place you.

These could be people on losing streaks… and/or people that are on winning streaks/smurfs. This is probably why the game can feel very streaky for some people.

However, if you quit and regroup to prevent a bad day from turning into a matchmaker issue, then you won’t run into a streak issue of bad teammates.

1 Like
9

I agree with your math that the matchmaker system is horrible and rigged.

10

That’s a bold claim…
Doesn’t seem rigged to me, and the only people I see complaing about it are quite bad at the game (as shown by their low rank).

3 Likes
11

You can agree with this or not, since there is no proof. However, the system does somehow prevent inflation/deflation. There are only a few choices on how to do this:

  1. Adjust MMR of people entering the system.
  2. Have balanced MMR changes in each match.
  3. Globally adjust MMR to keep it from drifting.

You’re just making stuff up here. The SR debuff doesn’t happen till 4200. 3000+ Skill Rating Data and Analysis (now including DCs).

I’m not sure what you’re saying here, but it sounds like you are in conspiracy theory / rigging land.

12

You’re overlooking a much more obvious problem with the Skill Rating system, which is the hidden Match Making Rating system. SR cannot function to rank players when every match is being handicapped by Blizzard’s proprietary “match balancing” algorithms. MMR does not exist to make matches objective or fair, it exists to make Overwatch as addictive as possible.

1 Like
13

Here is the problem with the difference between wealth economies and the SR finite issues.

There is a finite number of points in the total system of say money. To say that, you would have to say SR is a currency in that model.

However, SR is NOT a currency. That is where you have errored in your model. You would have to show that SR (or more accurately the hidden MMR) is a currency… which there is no indication that is true.

A finite currency would mean that in any given game, the SR would be equal. However, that is not true. At the farther ends of the boundries, you see especially at high ranks where pbsr modifiers are absent, that the total SR is actually a negative number. Players who win, always gain less the the losers, lose.

This means that SR is not finite. The currency isn’t finite. What IS finite is the boundries.

Since you can’t say that it is the same as a tax system… where the high level taxes are then given to the low end of the bell curve. You can’t say that because you would have to have an equal number of games on the low end of the curve.

If there are more high level, or low level games then the other… then there will be a shift in the SRs and infation or deflation will occur.

Instead, it is far more probable that the game purposfully intends for the percentages given… where 30% of the playerbase is in gold, 25% in silver and plat… to be the case and will adjust the active player base into that bell curve.

if SR was finite, you would see a number of players be capped at 5000… and greater number of the playerbase entering gold. Much like finite economies like real world currency where there is a small portion that have a large and capped stash of currency.

14

Hmm, perhaps I should have covered this more. Placements do ‘reset’ the system. This is why when I conclude that the sum of all SR will reach 0 in finite time, I assume that this finite time is large (and thus beyond the season time limits), making this fact irrelevant. The trend matters, even within seasons–though I do agree that seasons reset the system.

This is a good work, I suppose when you say:

If an account is inactive for a while, after the player returns SR/MMR gains will be exaggerated again for a time (48). For this reason, players that only do placements each season will tend to see high volatility in their rating each season.

This could contradict my findings since it introduces a way SR can enter the system – through players who do not play very often, winning games. If there are other possible contradictions, do let me know.

I do agree that MMR is generally considered the better measure of skill, all-be-it invisible to us. However, I was modeling SR, not MMR.

1 Like
15

I rank around 3300 which is better than 95% of players, I have dipped into masters as well, and this is 100% solo queue never cheating (like MANY of the DPS in these ranks) or getting boosted. I also use skill hero’s, not brig, moira, mercy or stuff like that.

16

Whoa there cowboy! Do you have like… A source for your claims?
The MM has always been really nice to me, even if some times I couldn’t see it.

The only time I got bad teammates was when I was throwing a lot, and even then I get bad enemies too.

17

Yeah, the game’s own designers. I’ve quoted them in my thread on the subject, please go read it.

18

This is likely true.

I would reference [arXiv:1702.06820] with abstract:

“Matchmaking connects multiple players to participate in online player-versus-player games. Current matchmaking systems depend on a single core strategy: create fair games at all times. These systems pair similarly skilled players on the assumption that a fair game is best player experience. We will demonstrate, however, that this intuitive assumption sometimes fails and that matchmaking based on fairness is not optimal for engagement.
In this paper, we propose an Engagement Optimized Matchmaking (EOMM) framework that maximizes overall player engagement. We prove that equal-skill based matchmaking is a special case of EOMM on a highly simplified assumption that rarely holds in reality. Our simulation on real data from a popular game made by Electronic Arts, Inc. (EA) supports our theoretical results, showing significant improvement in enhancing player engagement compared to existing matchmaking methods.”

This serves as proof that this practice is likely done in
other game titles – though it would be suicide for Blizzard to say that they do this, as the community would be in outrage. It is worth noting that the paper claims, and shows, that equal match making is the least optimal of all match making schemes considered. Perhaps Blizzard is being least optimal – you decide.

1 Like
19

Not to be that guy but these statistics are also congruent on them being accurate… I find it incredibly hard to believe that only 8% of players were in bronze considering.

20

I don’t agree with your conclusions. You post has many assumptions and your conclusion seems biased.
You should check kawuumba’s post. He used TONS of data and the correct method, then formated and presented it.

You should check it out!!!

Btw do you remember metzic? He was a true believer in your post. I added him, we played some customs, I reviewed a few videos… Didn’t work.

Then he dropped the attitude and worked to improve and he CLIMBED. And he stopped being bronze, and if you give him a bronze account he climbs easy out of it now.

Just food for thought.

Btw I really like you (Eddie Dean is my favorite in the whole series, I had to stop reading for like a month after algul siento).
Just apply the gunslinger oath to your game sessions and you will climb.

1 Like