You just explained matchmaking in a way that a undergrad in computer science could tell you was basic, incomplete, and several decades behind current technology.
You ever heard of Dunning Kruger? You know so little about this that you think you’re brilliant.
Your system has problems. Problems that are far worse than the supposed “problems” that you think currently exist.
You likely don’t have the intellectual curiosity or ability to identity those problems, but others have already done it for you. You can start with that video you linked.
You’re like a 5 year old that thinks square wheels would be better.
I explained proper matchmaking perfectly, I didn’t try to explain the way current matchmaking works so of course I left out those details, on purpose.
Every matchmaking system has some issues, but nothing holds a candle to the issues Overwatch matchmaking has had since day one. I’d like to hear what problems you think there are with my matchmaking. I know you think complicated is better, that’s how peoples brains like yours work, overcomplicating the simple, never stopping to ask “yeah it’s doing more stuff but is it better?” that’s why things get confusing to you, you are literally confused that something so simple could be better than something so complex.
Here’s another MAJOR problem with current matchmaking. When you practice at a game enough to get significantly better than the average person, you have EARNED the right to join a match at any given time and typically be the best person in the match, to dominate the opponents. That’s part of the enjoyment of gaining the skillset, so you can dominate, so you can be the star of the match. Want to know why smurfing is so prevalent? It’s a side effect of matchmaking that has taken away the satisfaction of being the star in the match, when you smurf you can start at a lower tier and be the star again. It’s nature reacting to un-natural matchmaking. With current matchmaking there’s no enjoyment anymore in becoming better than the average person because no matter how good you get you’ll just be placed in a match with people of similar skill, so no matter what you do, you’re just like everyone else in your match, who cares??
Your brain thinks “fair games are better” and “nobody wants to get beat down by people who are much better than they are,” but that’s where it requires FAR greater understanding of things that you can’t learn in a classroom or read in a book, people want the adversity to drive them to become better than average, they want to see that person in their match who shows them how good people can get at a game, none of that happens anymore.
One day this type of matchmaking will fall apart, they’ll talk about how “we found out that it provided a bad user experience for xyz reasons” and they’ll have lots of data and charts and books to support and show why it’s bad and that’s the day that someone like you will believe and understand it. I am the person trying to tell you the Earth isn’t flat before it’s been discovered, of course you’ll fight it.
So much of this is the same people. You said hundreds of thousands of people reporting “the same thing” and what you were describing was that you have a masters account, but also a gold account that you struggle on. You then said there were hundreds of thousands of people describing the same thing.
And then you linked to a search result. A search result. I’ve used the term “rigged matchmaking” in my posts describing a problematic theory that harms the OW community. That does not mean that I have reported the same BS that you are.
There is no attempt in your posts to actually understand the underlying systems. Are you aware of the history of Elo systems? Are you aware that things like hidden skill ratings (in addition to the publicly visible ladder ranking), pushing for 50% accuracy, accelerated score gains, etc are used in computer adaptive testing for graduate school entrance exams?
Do you understand that these innovations did not originate with OW?
Why do you suppose these systems were developed in the first place? Because it wasn’t to rig OW matches.
Absolutely, I posted a video to someone giving a keynote for students. I know it didn’t originate with Overwatch but Overwatch has expanded upon this matchmaking in a way that makes it worse, not to mention 6 v 6 with complex hero’s that all have vastly different skillsets makes Overwatch inappropriate for modern skillbased matchmaking. For games like modern warfare it can work significantly better, look at kill death ratio and a few others and you’re good, Overwatch can’t work like that.
Simply put, the complexity of having tanks, healers, dps, and each hero with different kits and utility and skillsets makes algorithmic matchmaking especially awful for Overwatch.
So this is a profound misunderstanding of how 50/50 matchmaking works (and why, for instance, the GMAT- a test used to differentiate among grad school applicants- also pushes for 50% accuracy when it selects the difficulty of the questions its test takers face), and why 50/50 matchmaking gives players the most opportunity (not the least, but the most opportunity) to impact their matches.
I’ve written about this fairly extensively when others posted this same misunderstanding. But I’ll give you the short version. I’ll also give you the short version for why hidden skill ratings have utility and why they were first proposed in order to correct a problem with Elo’s original system.
The reason hidden skill ratings are useful is that our algorithm operates better if it does not assume that smurfs and boosted accounts (or even simply someone who has significantly improved since last they played) have equal skill to their current ladder rating. It is useful to assign a separate skill rating apart from ladder ranking in order to better account for these discrepancies.
Having a single skill rating (as you propose) makes players whose current skill is much higher than their ladder rating take longer to climb, and it makes players whose current skill is much lower than their ladder rating take longer to rank down.
We don’t want them to take longer because it harms every match on the way down/up.
The short version for why 50/50 matchmaking is preferable is because it allows the game to learn the most about every player in that match. If a player pops off in a 50/50 match they can swing it in their teams favor. In a 70/30 match or an 80/20 match or a 90/10 match it is much more difficult for anyone to alter the outcome of that match. And for that reason, we learn less about everyone in the match. (And this is particularly true if we are only tracking wins/losses as you suggest.)
In a 70/30 match, we could replace a player on either team with a player of significantly higher or lower skill and still expect the 70 team in that matchup to win. So how would we know the skill levels of the players in the match if we could replace one of them with a player of much different skill and get the same outcome?
Do you see the issue there?
In the 50/50 match, though, if we take any one of those players and replace them with a player of much greater or lower skill we swing the match in the favor of one of the teams. The implication of that, then, is that we can actually tell if one of those players was more or less skilled than we thought they were.
Do you see why that is beneficial if we want to rank the players according to their skill?
So. You aren’t wrong that this poses challenges. OW is incredibly complex. You are wrong, however, when you suggest that these issues would somehow disappear if we use systems that are less able to differentiate among players of different skills.
The issues with OW’s complex design would always make ranking players a more difficult prospect than they would be in a game in which every player played the same hero (we could have everyone play S76, for instance) and all matches were on the same map (King’s Row, maybe).
50/50 matchmaking still gives us the best opportunity to notice players are a different skill than their current ladder rating would suggest. Performance metrics still allow us to better deal with smurfs, boosted accounts, etc. Having a hidden MMR is still more useful for noticing changes in skill than having a single skill rating, etc.
These tools are still better than the alternative.
The complexity of OW is a separate axis that poses challenges regardless of the system we use to try to rank the players on the ladder.
You sound like you understand the real way the matchmaker works (although your faith in it is misguided) and are ready to have an honest conversation so I’ll continue with you because other people in this conversation clearly are either being dishonest about how it works or they just simply don’t know. I do understand the purpose of a 50/50 match and know why matchmakers use MMR and SR, you would be surprised what I know about how matchmakers work, even more so than the programmers themselves who only know how it is “supposed” to work and don’t realize some unforeseen, unexpected side-effects that are occurring. My point I am making is that it creates an awful user experience. The supposed problems it solves are much more easily solved in other ways and the problems it causes are immeasurable (literally it cannot be measured by running simulations or pulling data).
There’s a problem with this statement. You’re saying 50/50 matchmaking is preferred so algorithmic matchmaking can work properly. Well yes, of course, that’s like me saying 50/50 matchmaking is not preferred because algorithmic matchmaking provides a bad user experience. You can’t use your own viewpoint as an example as to why your viewpoint is preferred.
But let’s ignore all that and simplify. In order for me to explain why the “algorithmic” matchmaking is bad compared to the matchmaker I explained I need you to answer this question (I already know the answer, I need you to answer it so we can continue knowing we are both starting from a place of agreement):
In a 50/50 match when the matchmaker is trying to, as you put it, “learn the most about every player in the match,” what types of things would a matchmaker look at in order to “learn the most?” You can guess, or give specific examples. Answer that and I’ll continue
I mean, it depends upon how the system is set up. In your proposed system, for example, in which the matchmaker only tracks wins and losses and only uses a single skill rating, and always awards 1 pt for a win and removes 1 pt for a loss, it would be incredibly important to set up 50/50 matches to the extent that one wants any sort of integrity in the ladder rankings.
The underlying reason is the same as I laid out above: the matchmaker best learns if one or more of the players in the match is actually more/less skilled than their skill rating would indicate more easily the more evenly it makes the match.
So if you remove things like performance based metrics and the hidden skill rating, the only thing you’ve got left is the outcome of the match. In that environment, making a match that deviates to any significant degree from a 50/50 match ensures that the system is least positioned to determine if anyone in that match is better/worse than they thought they were before the match began.
Removing 50/50 matchmaking in that system, then, would mean that we could not award SR accurately unless we accidentally made a 50/50 match. Your idea of always awarding/removing 1 point of skill rating, regardless of the match made would be utterly terrible unless you designed the system to only make 50/50 matches. Awarding a point to the winner in both a 70/30 match and a 50/50 match undermines honest skill ratings. Worse still, awarding the same point to the winning team regardless of whether they were the 70 or the 30, undermines the accuracy of the ladder.
The only way for your proposed system to have a hope of accurate ladder ratings would be if it leaned super hard into 50/50 matchmaking, and even if it did that, it would still make it easier for smurfs, boosted accounts, etc to manipulate the system, and the system would have a harder time accounting for new players who were significantly worse/better than the average player.
In short, it would make for a worse player experience. (Unless, of course, you were boosted or wanted to smurf or whatever- those players would have an easier time.)
In a 50/50 match when the matchmaker is trying to, as you put it, “learn the most about every player in the match,” what types of things would a matchmaker look at in order to “learn the most?” You can guess, or give specific examples. Answer that and I’ll continue
Your proposed matchmaker is only looking at wins and losses. And, if I understand you correctly, does not factor in things like win streaks or any performance based metrics. The GMAT, for instance, looks not only at questions answered correctly vs incorrectly, it also tracks the difficulty of the questions answered (and pushes for 50% accuracy) over time, and adjusts the difficulty of the questions asked based on things like win/loss streaks and the test taker’s performance earlier in the section.
Um, if you’re talking about the video above, that’s at GDC (Game Development Conference) and it wasn’t to students, but to other experts in Josh Mercer’s field, the same Josh Mercer that worked on the OW matchmaking system (among many others).
Which…I thought you knew, but just in case you’re not talking about a different video, just thought I’d let you know.
Not sure if you’re purposely avoiding the question or not, I’ll rephrase it using your own comments so it should be impossible to not understand. You said:
“50/50 matchmaking still gives us the best opportunity to notice players are a different skill than their current ladder rating would suggest.”
Explain how the matchmaker would “notice players are a different skill than their current ladder rating would suggest.” Again, I know the answer already, I’m asking you to to see your answer.
Good job man, you got me. It wasn’t in front of students it was at the GDC, great job on your “gotcha” moment. This adds nothing useful to the conversation whatsoever, you’re just saying stuff to pick apart things because you’re desperate for some little win. Maybe later on I’ll do a typo and you can say “they’re not there” for another sweet gotcha moment. What a waste of time.
Dude. Calm down. IF you were making a mistake, it was a reasonable one, but also kinda important in the context of how seriously you took the video.
I wasn’t being a jerk here, it was a honest correction based on something you actually said, and I tried to be careful in stating that you likely knew it. And you COULD have been talking about a different video.
Do you always look for the worst in everything you hear from others?
We learn that a player is more or less skilled than we previously thought because they are able to change the outcome of our pre-match prediction. Our pre-match prediction is based upon our current understanding of the skills of all the players in the match. And our ladder ranking (in a competitive ladder system) is intended to reflect the actual skills of the players in the match.
If we want to know if any of the players in the match differ from our current understanding of their ability, we need to give them the best possibility of upsetting our prediction. (Because, again, our prediction is based upon our current assessment of their skill.)
When we make a 70/30 match rather than a 50/50 match, we are giving the players less of an opportunity to upset our prediction. So we can see if our prediction is really wrong (if the 30 team wins the match), but we cannot see if it is barely wrong. Which lessens our ability to learn about the players in the match.
In the 50/50 match, though, we maximize the chances that our prediction is wrong. (We essentially ensure that it must be a bit off, because one team will win.) This allows us to see finer gradations in skill among the players (rather than simply noticing when we are really wrong.)
Consider the following scenario:
We make a match between 2 teams. I’ll use only a single number to represent the skill of the players in this match. Team A has the following 6 players: 105, 95, 99, 115, 120, 111 for a combined skill of 635. Team B has the following 6 players: 55, 85, 90, 73, 67, 60 for a combined skill of 430.
We would expect the first team to win that match the majority of the time. How much SR should we award them if they do, in fact, win? It should be based on a better understanding of their skill than we had before the match. (That’s the whole reason we make the match in a competitive ladder- to better rank the players according to their skill). But how much did we learn by making this lopsided match? That first team would, more than likely, still win if we replaced one of those players with a significantly worse player. We could take any of the players on that roster and replace them with a player who had a skill rating of 30 less and that team would still win.
So we do not know if those players are as skilled as we think they are.
That’s the key point. By making the lopsided match, we cannot tell if any one of those players is much worse than we think they are. Our rankings have a high uncertainty.
If, instead we make a 50/50 match:
Team A has the same 6 players: 105, 95, 99, 115, 120, 111 for a combined skill of 635. But now, Team B has these new 6 players: 101, 99, 103, 111, 113, 118 for a combined skill of 635.
Now who wins? We honestly do not know. But it’s probably the team who has one or more players who are a little bit better than we thought they were. We can tell players who differ from our current understanding of their skill to a much finer degree. Now if we replace any of these players with someone a bit better or a bit worse, we impact the outcome of the match.
That’s why we learn more about the relative skill levels of the players when we push for 50/50 matchmaking.
tale and I both have other posts that describe this. A while ago I made a post that linked them here so you don’t have to wade through a bunch of links.
My god. I just spent a good 30 mins scrolling back through some of those posts. You and others really did explain all of this back in 2018, didn’t you? There were some really good posts with really nice explanations of how these systems work. And there were several of you at the time using various language and analogies and ways of understanding this that worked together to give an excellent break down of how various competitive ranking systems do work, could work, have worked and how they each relate to the system that OW uses.
Good stuff.
And already present in the discourse way back in 2018. I feel like anyone who had an honest desire to understand probably could have at the time. That’s why it’s so mystifying that this whole “rigged matchmaker” and “forced 50% win rate” narrative persists.
I don’t know. I’m pretty sure I’ve been less patient in a post or two than I could have been over the past month or so. When you are teaching (at least when you are teaching adults who are paying you to teach them) people pretty much want to learn and understand that they have things that they do not yet know.
This sort of forum discourse is different and you never really know if someone is simply trolling, is honestly struggling to understand, or if they have some other weirder agenda at work. It makes the whole conversation harder.
If you follow my later conversation with Lhun you see the agenda change. At first, he’s legit asking questions, but he ends up going off the deep end.
You can both not understand and have a scapegoat, but not everyone who can understand can find a way to cope without the scapegoat.
That’s why you see me try to redirect to more plausible causes of their ire. You seem to do it simply because it’s true, but I really, really focus on the idea that yes, your teammates can be bad and that’s ok, here’s how the system works (and how it fails!) so you can stay calm about it.
Maybe you can figure out how to do it better than I, as making people feel better isn’t really my skillset.
ETA: I don’t know what I did with the original 9 page document, but before role queue, LFG, and the endorsement system I created a hybrid matchmaker and posted it on Reddit. I’m sorry I deleted it, but you can look at the comments and get the gist: