Thats the point of matchmaking. If the game perceives you as good, you will meet better players and opposite
3 Likes
I feel like the thing is probably too entrenched in some people at this point. It’s become part of their identity, and that gets really tricky. There may have been a moment early on where things could have gone differently (but I think you guys did an excellent job at the time- I certainly don’t think I could have done any better.) But at this point people have put so much time and energy into convincing themselves and each other that there’s real investment.
Hell, there’s a culture that’s developed. And that’s hard to give up- even if there isn’t some sort of financial incentive involved.
It’s really interesting to study though. That’s probably, more than anything, why I still participate in these threads at this point.
I was really fascinated to learn, for instance, while perusing those 2018 threads that the theory has not been modified to address some of it’s fundamental incoherences at any point over the last 4 years.
There’s a whole ‘do we want random or do we want skill based matchmaking’ contradiction that was present in those early days, and is still present in the theory today. We seem to want random (because we talk about how the most skilled players should have really high win rates), but we also say we are going to make matches only between people of similar skill ratings (which would preclude those really high win rates we seem to want to see happen- unless skill rating is just a name and it has nothing to do with the actual skills of the players). It’s a pretty fundamental contradiction that’s just baked into the theory.
And that’s what I was trying to get at earlier with my unicorn believer analogy- people should be able to recognize, on some level, when this stuff is pointed out that they do not have really solid reasons for what they are saying, nor do they have a really solid theory that stands up to scrutiny.
If they still want to express a preference (even if it isn’t the most well thought through preference)… whatever- folks prefer what they prefer and think what they think.
It’s just that when they purport to be experts or have a really solid understanding of what’s going on, when it’s clear they don’t, it evinces much less self-awareness and it makes it much harder to have an honest and productive conversation.
1 Like
My favorite is when they complain about a forced 50%, but then say to create games based on SR, not realizing that the complaint and the solution are the same basic system.
As far as I know, I’m the only community member to try to develop a system to address the perceived problems. It became outdated with the introduction of LFG and Role Queue, but I posted the link in an edit to my last comment, if you’re curious.
1 Like
Bro…. I for one, am grateful for the amount of effort you’ve spent trying to clear things up…. But I’m not entirely sure you realize how zealous some of these people are.
P.S. got rolled by GMAT last year when I applied to grade school 
still got in though.
Some guy around here named Nano, not 100% aligned with the rigged cultists, tried to write a simulated SBMM in Python (I think) about a year ago.
It was okay. But not very complex. And cutbert and receipts tried to skew the presentation (as expected).
1 Like
No, normal distributions would occur from randomly selecting players. If you have an algorithm it will have a bias.
How do you know someone belongs in a certain rank if you modify the difficulty sufficiently to prevent them from winning more than 50% of their games?
1 Like
less chance of a flat tho!
2 Likes
So, I’ll go through how the GMAT does it again (just so we can see that this isn’t something that only OW does, it’s something that all sorts of competitive ranking systems do- the GMAT has no reason to try to rig the system against anyone and it’s accepted by all the best universities as a metric for ranking people according to their skill):
When a test taker sits down to take the GMAT, the algorithm selects a question of medium difficulty to start. (This is the equivalent of being given a gold match when you play your first comp game.) But every question after that first one is selected based on the test taker’s past performance. If the test taker gets that first question correct, they are given a harder question. If they miss it, they are given an easier question.
How does the GMAT know what difficulty of question to select?
It uses a whole host of criteria (that are tracked from the test taker’s prior performance in that Quantitative reasoning or Verbal reasoning section), but at it’s most basic level, the GMAT is trying to get the test taker to have an overall 50% accuracy- the test wants to attempt to get you to miss half of your questions.
And it’s pretty good at it. It has a huge question bank of questions at a wide variety of difficulty levels it can select. Roughly 80% of test takers will end up with an accuracy of approximately 50%. Only the top 10% of the ladder and the bottom 10% of the ladder will deviate significantly from that 50% win rate.
So, to your question- how does the test know someone belongs in a certain rank (or should have a certain score in this case) if it modifies the difficulty sufficiently in order to push test takers away from answering more than 50% of their questions correctly?
Two points:
The first is that any given test taker can answer more than 50% of their questions correctly. (I usually get between 80 and 90 percent of the questions correct when I take the GMAT, for instance- that’s enough to place me well within the top 1 percentile of test takers on that test.) And any given OW player can win more than 50% of their games. (Players at the very top of the ladder will do so. As will most players who are currently ranking up.)
The second point is that the GMAT is selecting questions in order to learn more about the test taker. (Just as the matchmaker is selecting matches in order to learn more about the players in that match.) And it learns something different if it is able to push a test taker to a 50% accuracy using only Easy questions than it does if it has to pull out the hardest questions in its question bank in order to push a test taker toward a 50% accuracy. And something else again if it is never able to push a given test taker toward a 50% accuracy, even using the easiest or the hardest questions in its question bank.
Consider the following scenario:
I ask you to rank an OW team. These are a bunch of amateurs. We first pit them against a team of low bronze players. Our team wins. So we pit them against a team of high bronze players. Our team wins. So we try them out against a low silver team, and they lose. We try another high bronze team and they win. So we try them out against another silver team. etc. Our team ends up hovering around 50% accuracy when we pit them against teams in the high bronze to low silver range.
We now run the same set of matches with a different amateur team. But this team wins all of those silver games as well. They do not hit a 50% win rate until mid Masters.
Could we not, then, say that the first team is somewhere about the bronze silver border, but the second team is somewhere in mid Masters- even though we manipulated the difficulty of their matches in order to push them toward a 50% win rate?
Just as the GMAT does.
1 Like
That doesn’t work for an individual who is solo-queuing. Every game they change teams. So your entire premise is moot. There is no way an algorithm that ranks teams can rank an individual within that team. Period.
Consider the confounding co-factors:
- Team composition and role flexibility.
- Skill breadth (can your teammates switch to a hard counter? will they?) Skill depth (how much experience do they have on any given role, and how many roles do they have that skill depth)?
- Variance. Some days you or your team mates might be in-the-zone, or they could be tired or not sufficiently warmed up.
- Sociability. More sociable people are more likely to be on comms and more likely to group up. Groups who are on comms are more likely to win. That is not an individual skill, but a team skill.
- Complexity. There are 31 roles with 4 abilities each, and many of these abilities have synergistic or compounding effects. To build an elo from the combinations of these would be next to impossible.
- False metrics. Have you ever won a game, and the enemy team got all the cards, PoG and mysteriously 150% of the eliminations? I’ve been in games where we won simply because we had a teleport from spawn to half way to the point, giving us a 6 second advantage in reinforcing our attack. The enemy team got more elims, more ultimates, did more damage, but we won effectively through strength of numbers.
Another game we won because I spent the whole game as DVa shooting Sym and Torb turrets, and not much else. I got no medals, no card, no endorsements. It was QP, but if it was comp, I would have received a lower SR bump, simply because sometimes the things that win you games are not rewarded. And sometimes the doing the things that lose you games ARE rewarded. For example, doing a lot of damage, but not many eliminations builds the ultimate of the support roles. You can have gold damage, but lose the game for your team because Moira gets ultimate every 30 seconds.
The SR doesn’t measure skill or elo the way an actual game does. It measures something else, something for a new generation of gamers. A little bit like WWE.
This is a good critique of the overall problems with ranking individuals in a team based game. But this critique would be equally valid no matter how our matchmaker worked. It’s just a fundamental complexity that is baked into the system.
At the end of the day, OW will always be a team based game that nevertheless seeks to rank individual competitors with its competitive mode.
None of your critiques apply any more to a matchmaker seeking to push players toward a 50% win rate than they would to a matchmaker that makes matches more randomly. The only critique that you are making in this post that applies more to the current matchmaker than something like the most commonly proposed matchmaker (which does not use hidden MMR or performance based metrics) is your point 6. But even then it is a critique of using performance based metrics rather than a critique of seeking to make 50/50 matches.
The question you posed was originally- how does a system such as the one OW uses (or the one the GMAT uses) know where to place people when it pushes them toward a 50% win rate? That was the question I was answering.
Do you get how that works now?
2 Likes
As a reminder, PBSR used to be much, much stronger, so we know how tuned down it currently is. Most normal players aren’t getting much. You have to take stomp to see the effects you used to see, and as far as I can tell, do so on a regular basis. As in, one really good game isn’t affecting your SR.
I will caveat that I only think this from seeing other parts about SR gains from people that really are playing outside of their rank. It’s a small sample size and has the same anecdote issues as most accounts.
But, I do know PBSR isn’t nearly as strong as it used to be.
2 Likes
I’m going to go ahead and address the performance based metrics here as well:
These are really just in the game in order to better address players of very high skill and get them into their correct rank faster, while ameliorating some of the effects of smurfing and similar.
There are two main components of this one is the higher/lower SR bump from a win/loss that goes to players who perform significantly better or worse on their heroes than others at their rank. There’s a win streak/loss streak component here as well (as there is with the GMAT).
And this is really about more efficiently sorting players.
I don’t really disagree that some players might lose out in this system. But that loss would mostly be a slower rate of SR gain/loss. And it would go to players who were contributing significantly less damage/healing than other players on their hero at their rank.
And the truth of the matter is that you can still get a lot of damage on the enemy while doing things like shooting down turrets or setting up a TP. If you are getting significantly worse damage than someone else on your hero at your rank, you cannot assume that they are not setting up useful TPs as well. They are simply doing so while getting in their damage.
But we could imagine a player whose value add to their team is something that most other players at their rank on their hero are not doing. This would generally be something in comms- some way in which they are coordinating their team and enabling the other players on their to perform better than they would otherwise. Something like shot-calling/play-calling, etc.
And someone who is really good at that, but not so great mechanically might be a loser in the current system. They would rank up more slowly than someone who is better mechanically. I think we overestimate the impact that the actual performance based metrics OW uses has on all of this though. If it’s impacting anything like a large-ish percentage of the players, it would more than likely be impacting me. My mechanics are not great.
But I’m not too fussed about it.
If I win my matches, I’ll still rank up. And if I’m not winning my matches, I should have no reasonable expectation of ranking up. Further (and this gets a bit complex, so please bear with me), this is one of those fundamental incoherences that’s baked into this whole “rigged matchmaker” theory that I mentioned above.
If my performance based metrics are low (which is the issue brought up in the earlier point 6), the “rigged matchmaker” theory posits that I would be given teammates whose performance based metrics are high in order to “force” me to win 50% of my matches. So even if I am being “punished” for my poor performance metrics and awarded less SR after a win, I am also being given better teammates making it more likely that I will win. So why am I worried about a system which will give me better teammates?
Conversely, if I am one of those players with better performance metrics, I am already a winner in this scenario as I am given more SR for my wins and lose less SR for my losses. So why am I worried about this system?
The “rigged matchmaker” theory wants to say that each of these players should be bothered- the player who is performing better on the performance metrics, because he will be given “worse teammates” (though that is a really poor understanding of how this all works) and the player who is performing worse on the performance metrics, because he will win/lose less/more SR from his matches. But it doesn’t really make sense to say both of those things at the same time. If players performing better are losers in this scenario, then the players performing worse are winners (and that is usually how I see the “rigged matchmaker” theory framed.)
But point 6 suggests that players who are performing worse in the performance metrics are the losers.
So which is it?
In truth, the vast majority of players are going to perform roughly within some similar range on their heroes at their approximate rank. Nothing I’ve seen suggests that these performance metrics have any significant impact until someone is a clear outlier.
And that brings me to my next point. Dealing with smurfs:
People will often complain that the smurfs are always on the other team and that there is nothing they can do. But these systems work to address this in a couple of ways. The first is that by increasing the SR gain that performance outliers receive, smurfs are ranked up more quickly and out of those lower ranks (at least until diamond). The second is more interesting to me. There was a dev video at some point many years ago, in which they said that their performance metrics were pretty good at identifying people whose skill was significantly better than their peers at their current rank and that the matchmaker was aware of that.
That makes sense. If you are measuring performance metrics it would be pretty easy to tell a GM Widow from a Silver Widow, for instance. But if you watch someone like Yeatle do an unranked to GM run, he’ll point out the smurfs on the other team as well. He’ll say something like, man that low account Soldier is really good for a Plat player. And what you’ll find is that Yeatle never gets the smurfs on his team.
They are always on the other team.
Why is that? It’s the matchmaker using it’s performance metrics to push for a 50/50 match. It knows that Yeatle is smurfing, so it does it’s best to find comparable smurfs to pit him against (which isn’t always easy, it’s a small pool of available smurfs and Yeatle is really good.) You could do something similar using only things like wins/losses and win/loss streaks but it would be a lot easier for the smurf to wreck that system- they’d just have to maintain something like a 50% win rate to stay at the lower rank and to confuse the matchmaker enough that it couldn’t balance the smurfs on each team.
2 Likes
I agree. I think it’s more a boogeyman than anything else at this point. It’s just really easy for people to suppose that something that only impacts players who are clear outliers is impacting them/is a major issue for many players.
2 Likes
Ok, so now that we’ve agreed on how the matchmaker works on a fundamental level (there are many additional “features” added to Overwatch that I’ll leave out of this discussion) I’m going to explain what ELO hell is, I’ll give you a hint, it’s not a psychological disorder related to dunning kruger affect which is an excuse developers use to avoid facing the fact that their systems have horrible side effects.
Let’s say you have a Master tier player who logs onto an old account that was previously stuck in gold tier. For the first several matches the Masters player can win easily because the matchmaker was creating matches based on the assumption that this person plays at a gold tier level.
After winning several matches the matchmaker will catch on that this person is playing beyond their tier level and since the matchmakers goal is to create a 50/50 match it will look to offset his skill discrepancy by either placing significantly weaker players on his team, or by placing other players it suspects are playing at a masters level on the opposing team. Basically, it creates a 50/50 match based on how the players have been performing over the recent matches. Since smurfing is allowed, and because high skill players are stuck at lower tiers they don’t belong, the matchmaker has no problem finding someone just as good as the masters player to place onto the other team. Now the masters player, who is playing gold tier, is actually getting matches where they have only a 50% chance of winning, even though they are better than everyone in the match except for the other masters player on the opposing team. This is what keeps them from ranking up as fast as they could and creates a perception of “ELO hell.”
I mainly solo queue and have seen the above situation countless times, but at times I group with friends who are also masters or GM. They purchased a new account and want to rank it up, so we start playing and land in plat to start with. The matchmaker has evaluated our skill and it knows that we aren’t gold or plat players, the matches are high diamond tier or master level games being played in plat tier for the sake of a 50/50 match.
This is why skill-based matchmaking is a good idea in theory, but in practice creates an awful experience. A possible solution to the above scenario is to lower how good the matchmaker is allowed to assume a player is based on their performance. In other words, if a GM player is playing in bronze the matchmaker should not be able to assume he is GM caliber (or even plat or gold) based on their performance, it should only be allowed to assume they are a high bronze player and nothing more to avoid ELO hell. It wouldn’t resolve all the issues but would at least help.
Okay, but just like the GMAT, when the matchmaker recognizes that it needs to find tougher opponents in order to push that player toward a 50/50 win rate, it ranks them up.
That’s the way the system operates.
It assesses a player’s skill level by determining which opponents are needed to push someone toward a 50% win rate. If one player is matched against bronze players in order to achieve a 50% win rate, that player gets a bronze rank. If another player gets pitted against masters players to achieve a 50% win rate, that player gets ranked masters.
Again, that’s how the GMAT works too. It’s a well designed system that is used in a variety of fields. It’s not just some crazy thing that OW does.
And it’s not like there are that many Smurfs running around. When GM players stream those unranked to GM streams, you can see the matchmaker really struggle to balance out the teams. It’s really only once the streamer gets into Diamond that it is able to begin to find some people that are sort of close to their level.
A Masters player is not going to struggle in Plat. There just aren’t players of that caliber playing in Plat. But people continue to make this stuff up, because they “know” that the “rigged matchmaker” theory is true. So it’s fair game if they make up evidence in order to support their position.
All that does is make it difficult for players to understand what is really happening in their games.
1 Like
Just so you know, because GMAT works like this doesn’t make me believe it has added credibility. I don’t give credibility to something because it’s used by major institutions, I give credibility to something if it makes sense and works properly.
What do you mean “it ranks them up?” They have to win to gain SR and if they are in 50/50 matches against other master players then they’ll be winning around 50% of their matches and so it can’t “rank them up”.
…unless you’re aware of how Overwatch matchmaking system can quickly shift someone’s rank up by placing them with other suspected high tier players and grouping them together to get them to their tier quickly. But I don’t think you’re aware of that and it’s an entirely different discussion altogether, I’m talking about how typical modern matchmakers function.
This, by the way, is already how the system works. If the matchmaker assumed that someone in Bronze was actually GM, it would give them GM games. And it would adjust their SR accordingly. It does not actually do this though.
This is about the equivalent of suggesting that a kindergartner is actually operating at a grad school level and therefore we will give them grad school level work but keep them in kindergarten.
No one would ever do this. And there’s no evidence that this is happening here.
Instead, the matchmaker has some limit as to how much higher (or lower) it can rate someone’s skill than their current SR. And if someone is at that limit, they will get more SR per win and lose less SR per loss, so they can more quickly rank them up to that limit. But once they are at that new limit, the matchmaker can then posit that they are actually higher than their new SR and begin the process all over again until they rank the player up to their true skill level.
1 Like
If someone wins 50% of their matches against Bronze players they stay in Bronze. If they win more than 50% of their matches against Bronze players they gain SR and are given harder opponents. That’s the ranking up that I mentioned.
The idea that there are GM players stuck in Bronze is absurd. Have you ever watched an unranked to GM stream (or a Bronze to GM stream, which I think is more ethically problematic, but they certainly exist)?
How difficult is it for those players to breeze through Bronze?
Waitaminute… is your notion that those players in those streams are matched with other great players on their teams? Because that’s not at all what the vods show. How would that even work? How would the matchmaker know which GM players to give the great teammates to and which GM players to give the sucky teammates to if they are all GM players stuck in Bronze? You are now suggesting that the matchmaker does the opposite thing for different sets of players who look the same in terms of their skill and performance metrics.
That’s like… peak conspiracy theory at that point.
1 Like
It does give them more difficult games, maybe not GM but you’re playing probably plat tier matches in bronze and since the teams are balanced for 50/50 there is little you can do to change the outcome. I’m not speculating on this, I’m telling you this is what it does. There is plenty of evidence for it, it’s called eye witness accounts of thousands of players describing the same thing, not just in Overwatch but in other games as well.
This is probably where we part ways. You lack the experience in playing and experiencing the system that it takes to really understand how it works. How things are designed to work and how things actually work are very very often two different things. Who knows more about the car, the mechanic or the person who races it every day? The mechanic can tell you how it’s supposed to work, the driver can tell you how it is working. I can tell by your posts that you simply don’t have the hours put in on multiple accounts to understand some things I’m explaining. That’s the difference here, you’re talking from a “this is how it should work” I’m talking from actual real life experience that you don’t have.
So one of the issues here is that most people are really bad at this sort of evaluation. Take any skill-based competition and most people watching cannot tell which players are better than others. They lack the expertise to make that sort of assessment.
And people sort of naturally over-estimate their own ability.
So, someone playing a game with their friends or with people in their neighborhood will pretty much always say they are better than most players- that’s kind of our default setting, to assume we are better than most at whatever we like doing.
But that cannot be the case. Most people cannot be better than most people. So players in Bronze will naturally assess themselves as better than average, sometimes significantly so. And they will construct various theories that allow them to continue to believe this rather than re-evaluating as new information is provided.
Someone who is good at making this sort of assessment operates differently. They are more dispassionate. They use new information as it is provided. They are less likely to make strong assumptions like ‘I am better than most.’
But watch Bronze vods and GM vods. Compare them. People in Bronze do not simply have worse aim. (Though they do.) They are not even noticing the same things that players in GM notice. It’s a stark contrast. It’s not even close. But the worse someone is at a game, the less able they are to see these differences. Just as someone who has never played tennis will be worse than a tennis pro at noticing the differences between players of different skills.
And, again, consider what you are actually saying here. If the matchmaker gives hardstuck Bronze players (who are really as skilled as GM players) crummy teammates and matches them against the other GM players, but the matchmaker simultaneously gives other GM players the best teammates and matches them against crummy players, the matchmaker must actively decide what teammates to give someone based on something other than performance metrics and other than skill.
You are suggesting that the matchmaker literally decides to rank some people up and keep others in Bronze.
But that begs two questions. The first is why would the matchmaker do this? Because you are not at this point suggesting that this is simply some emergent property of a complex system that seems like a good idea on paper but is actually terrible in practice (as you earlier suggested). You are now literally suggesting that the matchmaker actively screws some people over and it decides to do so.
Why?
The second question is how does the matchmaker decide who to screw over? Why pick some people to win and some people to lose? It cannot be because this makes for an enjoyable experience for the player base. People really complain about this stuff (as you well know). Why would a game company want to actively piss off its players? Why would it alienate many of the people who might otherwise give it money? Surely they would prefer to keep players in order to maximize revenue.
None of this makes any sense on the most basic level. It only makes sense as an attempt to allow people to maintain their self-image of being better than most at the game even though they struggle to win most of their matches.
3 Likes
Every time I see this suggested I know I’m dealing with a special type of person, one who believes everyone is so much more stupider than they that they think they found a flaw in “the system”, rather than just realizing that even a half-braindead developer would immediately recognize this, which for normal people would cause them to consider that perhaps they are wrong about how it works.
Like, yes, if they are always making even matches but only adjust on a win or lose, you’re right that it would be random and also very, very stupid.
So maybe that’s not how how it works? Nope! I have the secret sauce and it’s everyone else that’s stupid!
1 Like