Did ANYONE archive Overwatch's BATTLENET Forum?

Competitive Discussion
182

So to be clear, your PRESUMPTION (and yes I’m calling it a presumption, because no, Tale, I’m not going to unquestioningly accept that every claim you make is de facto true or applicable to OW, no matter how much experience you have with other sorting models in other contexts. Indeed, you could be a Nobel prize laureate in this field and that still would have no bearing on your claims IN THIS CONTEXT on whether your conclusions are conjectural) is that it is NOT POSSIBLE that grind or engagement based mechanics exist in Overwatch?

To answer your question, “well designed” systems are predictive to reduce variability and accelerate the sorting process.

Not sure I agree here. The ladder ranking seems more like a point-in-time snapshot reflecting prior player interactions. Your statement here seems to apply more to MMR?

You cannot make statements like “it says such and such.” I’ve already stated that my conclusions about the matchmaker are inferences. Your statements here, however, seem to assert facts, claiming to know what it’s doing and trying to do, and so you take on a higher burden of proof. What you describe here is the working of a SBMM and not an EBMM. EBMM would take other values and variables into account, and would bring matches together based on factors beyond the assessment of raw skill.

This seems to defy logic. If the predictions are wrong, and those predictions directly affect matchmaking actions, then the predictions could and certainly will be determinative in some cases: file the result under “unintended consequences.” If I incorrectly believe to be Joe a great baseball player, and I put him on team A under that presumption, and he’s actually a bad player, that prediction/conclusion about Joe absolutely has consequences for both teams. The expression of skill of every player on each team IS affected by Joe’s performance while playing, and his placement on whichever team was a result of my prediction. My prediction for better or worse, certainly has consequences which could affect the outcome.

To agree with this conclusion, I’d have to agree with your premises, which as you can see above, I do not.

Your explanation makes a lot of assumptions. It also doesn’t address a phenomenon I see quite often in my games, which is loss streaks following win streaks. Considering the reliability of this phenomenon, I no longer consider it possible to be a statistical anomaly.

Also, unlike the GMAT where a more knowledgeable, higher percentile test taker still “wins” even though their score is considerably lower than it would’ve been on a non-adaptive test, a 90th percentile OW player suddenly has the same odds of winning their match as a 10th percentile player. We know in a random match, the 90th percentile player only has about a 10-15% chance of seeing a player on the other team who’s about as good or better than they are. Under the current model, that number is conservatively, what, 70-90%? That player should also expect to see a MUCH higher than normal chance of being grouped with a bad player – the BETTER his results – whereas this is not true for a 10th percentile player.

In scenario A) we take two top high school varsity basketball players, Steve and Alan, and these two players are playing on teams otherwise composed of 5th graders. By random drawing, Steve and Alan can either be on the same team or on opposing teams. Steve advances a grade every time he wins 5 matches (6th, 7th, and so on).

In Scenario B) we ensure that Alan, again, a player equal in skill to Steve, is always on the opposing team, playing against Steve each and every match until the experiment is concluded

In scenario B) we should expect Steve’s winrate to drop considerably, as he will lose many more matches.

In scenario A) we would expect Steve to have a relatively high win rate until he reached players of his approximate age and skill.

What am I missing here?

Yes, and GMAC’s goals are almost certainly very different than Blizzard’s goals, which would logically lead to the two organizations designing/wanting different systems. GMAC wants to sort the best students from the worst and to place them along a spectrum. Blizzard wants to make as much profit as possible which – does not require – cleanly and accurately or efficiently sorting players by skill. It only requires that they keep players playing for as long as possible. Extensive research has shown that players exposed to engagement based matchmaking play longer, are less susceptible to churn, and therefore show higher engagements metrics in every category that matters. So I’m not sure why you believe that GMAT algorithms are relevant to another industry with different goals and different business models. It seems logical that there might be some overlap, but it also seems logical that there would be considerable differences.

See Steve and Alan scenario above about win rate differential between random groupings and 50% groupings. You also cannot say that a random matchmaker “serves no competitive ranking purpose” since the process of winning and losing is obviously a stratification mechanism. Your presumption is also that there is no intentional grind for the purpose of engagement, even though it’s a known fact that the gaming industry commonly institutes grind mechanics to boost profits and engagement.

Yes or no question: is having a 1000 SR spread in matchmaking likely to increase grind?

Yes or no question: is having no SR restriction per role, such that one team can get plat tanks and silver healers, and vice versa, likely to increase grind?

Yes or no question: is there any correlation between EBMM and profit/engagement?

What is your evidence or what leads you to be believe that Blizzard employs no EBMM in its matchmaking algorithms?

The planet would’ve burned to a crisp by now if we operated under the assumption that a group of people (all with serious biases/predispositions towards one outcome over another) performing in a highly unscientific, unsupervised environment had discovered objective truths. There are so many ways that this “test” could’ve gone wrong or been fudged or proved inconclusive that it’s really not worth mentioning.

2 Likes
183

Tale, one of the problems I’m having with your communication style is that you don’t address things point by point. I literally reply to each and every point that you make, but you sort of do this thing where you reply to my posts in summary and leave out responses to my points that I consider to be problems with your argument. Yes that’s easier for you, but if convenience is the ultimate goal, it’s way more convenient to not reply at all. Not really interested in a fragmented conversation revolving around your whims.

In essence, I’m putting a lot more effort and specificity into my posts than you are. Further, because you don’t quote anything, and don’t reply in a way that’s interstitial you can simply pick and choose not to respond to various points being made in lieu of what you prefer. I’m sure you can see how that’s untenable from my perspective.

I’m not saying it’s intentional or bad faith effort because I don’t think it is, but I question whether it’s worth tackling a topic with this many sub-aspects with someone who treats some of them as though they didn’t come up.

You know what you’ve signed up for when we talk, so I’m going to have to ask you to address each point being made and to quote replies, or it’s just not worth it. It’s like a good portion of the conversation just drops into the abyss.

4 Likes
184

Given that literally every argument can be refuted with the epistemological nihilism that you exhibit, I think no reply is not only more convenient, but just as effective.

If you want assurances that Blizzard isn’t cheating you a priori, you’ll never get it.

That’s ok though, because nothing is real anyways and you can’t prove otherwise.

2 Likes
185

Show me one example of where I engage in “epistemological nihilism.” And if it’s just as effective why are you bothering to reply now?

Lol, I see you’re fast and furiously googling philosophical terminology. A priori has several meanings, how are you using it here? Don’t use $20 words you barely understand when a $2 dollar word will do.

Regarding the matchmaker topic, I’m careful not to declare anything as true or untrue or to align a conclusion with a fact if I can’t support it with evidence. I’ve never once said “this definitely exists in the matchmaker” or “this doesn’t.” And if you don’t understand why you shouldn’t let people make unsupported fact assertions without evidence then you don’t understand debate.

You seem bitter.

3 Likes
186

I’m afraid not. That post was real though.

That is, you didn’t imagine Scott Mercer’s post and explanation.

2 Likes
187

Where it at though? I doubt WyomingMyst would discriminately keep different posts.

3 Likes
188

Sadly when Steve and Alan are on the same team against 5th graders, we can not count the win towards their advancement because they are expected to stomp those games. (We gain no information!) That in turn means that half of the games Steve and Alan play are totally pointless, so they have to play more games, making the system more grindy.

In scenario B every win counts.

Steve and Alan would however be on the same team regularly, because we don’t actually put them against 5th graders. We give them opponents of roughly the same rating as they are.

It is likely to increase grind! However, we want players to be able to play with their friends, so we have to allow some deviation. This is restricted to within 1000SR though.

Towards the higher end we restrict this even more. Once you get to Master it is reduced to 500SR. At GM it is reduced to 350SR and you can only group with one other player.

This is a compromise between match quality and being able to play with friends.

Also Silver and Platinum are only 500SR apart, even less if you account for the fact, that you can lose up to 5 games before dropping out of Platinum, which is more than 100SR. So if you have a Silver player and a Platinum player in your game, they might be within 400SR of each other.

How would you modify the skill based match maker and make it engagement based? And how would that increase either engagement or profit?

3 Likes
189

What you are bringing up is the fact that it is difficult and time-consuming to address multiple errors. It is far easier to simply throw lots of things at the wall that have no real understanding behind them.

In your posts that I address, I routinely have to address the same errors over and over and over again. And every time I do, you throw more things out there, but each of the things you throw out there are predicated on fundamental misapprehensions of how competitive ranking algorithms work. You are still not understanding how Elo-based systems use predictions.

I’ve gone over this I don’t know how many times at this point.

This is the same difficulty that evolutionists have had in debates with young Earth creationists. The young Earth creationists can throw out so many different things because none of them have to be based on anything real and none of them have to cohere- they can, in fact, throw out mutually contradictory things because there is no real underlying reality being described. (This is the issue I’ve been having wherein people will start by saying that the system is inaccurate, but once I demonstrate that 50/50 matchmaking is more accurate they will pivot to the supposition that it’s forcing grinding. Those are diametrically opposed complaints- the more accurate the system, the less grindy it is. But the people making the complaints don’t have to make complaints that cohere. They can simply make multiple mutually contradictory complaints, because the whole thing is just an attempt to explain a nebulous feeling that something is wrong. But that nebulous feeling is precisely what one expects from a highly competitive system- that sort of thing is stressful! Of course it’s gong to feel bad to be asked to perform at a high level every match if you want to rank up. That’s just the nature of competition.)

The evolutionists, then, have to pick and choose what they address. They have to respond in a way that discusses complex and nuanced systems, because they are attempting to convey something real. And doing so for any given point takes time and energy- they cannot possibly give thoughtful replies to every point, because it would take forever.

There is an asymmetric burden then. I can give you bullet point answers to everything you say. But some of what you say interacts with more complex systems. Some of what you say requires answers that will be difficult to convey in bullet point form. But mostly- it’s just that the most fundamental bits (why predictions? why 50% win rate? why accelerated SR gain? why separate ratings for MMR and SR?) are more fundamental questions that get to the heart of the matter and require complex explanations. When I use shorthand explanations, I simply get the response that you don’t think it works that way. And there is always, always the underlying issue that your replies indicate fundamental misunderstandings with the points I listed above. What should I do then? Ignore those fundamental issues to address something that you’ve said that is clearly based on one of those fundamental issues, or try once more to cut to the heart of the matter?

What is most frustrating is that I have composed multiple lengthy and thoughtful explanations for each of those questions, but no explanation can compete with “but it feels like it’s unfair to me and unless you can show me Blizzard’s proprietary code, your professional experience is equal to my anecdotal experience.”

What you are asking for, then, is something that is impossible. No one will ever be able to show you Blizzard’s proprietary code. Just as no one can show anyone the GMAT’s proprietary code. It will always be possible to interpret public dev statements in the worst possible light with no regard to the historical context of how competitive matchmakers are designed. It will always be possible to assume the worst if one does not understand why competitive ranking systems prefer to set up 50/50 matches or why accelerated SR gains are preferable when there is high uncertainty that hidden MMR matches publicly available SR.

But unless and until some basic understanding of how MMR and SR function, why 50/50 matches are preferable, etc exists, there is no point in addressing anything else. Those are the fundamental topics at issue in the discussion.

Once people realize that more random matchmaking results in grindier, less accurate rankings, then a real conversation can be had. Until that point, there is no basis for a real conversation. We’re still at the ‘maybe the Earth is 6000 years old; how could you know it’s not? you weren’t there’ phase of the conversation. Any interesting or broadly useful discussion of evolution cannot really take place until that phase of the discussion is over. Just as any useful discussion of the OW competitive ranking system cannot really be had until some basic understanding of how the matchmaker works is achieved. Then you can have a meaningful discussion of more pertinent issues. But one of the biggest is the one that Jeff brought up in a quote that Cuthbert described as an obvious lie: No system knows what weird choices the players in the match will make.

6 Likes
190

You are ranked (whether on the GMAT or in OW or in other competitive ranking systems) at the level at which you hit a 50% win rate. That is what you are missing. Almost everyone who takes the GMAT gets roughly half their questions right. If the GMAT has to give you really hard questions in order to get you to that 50% win rate, then you score really well. If, conversely, the GMAT has to give you really easy questions to get you to hit that 50% win rate, you score really low.

Just as in OW, if you have a 50% win rate when matched against 4.3k players (like Emongg, say, who even though he goes on long win and loss streaks still, as he loves to point out, hovers around 50% win rate over the long haul) you get placed around 4.3k. But if you have a 50% win rate when matched with Bronze players, you will rank somewhere in Bronze.

This seems to be a fundamentally poorly understood topic in these conversations. Again, the competitive system of MMR and SR is constantly testing itself in order to revise the rankings. The way that is accomplished (based on dev statements and on a good understanding of how similar systems operate) is through an ongoing comparison of match outcomes to predictions about those matches prior to the match taking place. If the current understanding of player skills is correct, everyone’s win rate will hover around 50% but those win rates will occur in very different matches. Both aspects of that are important to understand.

And the reason I keep bringing up the GMAT is because everything the devs have said about the OW system sounds like a description of the GMAT system. They are simply describing how well designed competitive ranking systems work. But people keep taking those dev statements (the ones that apply equally well to the GMAT) and assuming that they are saying something nefarious. Those statements are not evidence of what you think they are- they merely describe how modern competitive ranking systems (in multiple fields) work.

And, to illustrate the point further:

In your preferred scenario (the one in which high skilled players have a win rate of 90+%), most of those matches have no use whatsoever in allowing us to rank their skill. We could predict that they will win the match, but (and this is the important part) we could just as easily predict that someone much less skilled would win the match. So we would not be able to differentiate between a player of extremely high skill and a player of moderately high skill unless and until we got each into some matches that were closer to 50/50 outcomes.

Consider the scenario in which a diamond player and a GM player each are placed in different matches. The diamond player is in a lobby with 11 other bronze players and the GM player is in a lobby with 11 other bronze players. Both the diamond and the GM player will win their matches easily. How do we know who is diamond and who is GM at that point? That match is useless for allowing us to differentiate among them.

But, wait! you say, they aren’t always going to get matches like that- sometimes the diamond player will be in a match that is hard for them but would be easy for that GM player and we can then differentiate among them.

Yes! And that is precisely what we accomplish by making matches with a predicted 50/50 outcome- if player A might be diamond or GM, but we currently think they are diamond, we put them in a match that diamond players will win 50% of the time, but that GM players would win 90+% of the time and we see what happens.

You could do it more randomly but it would, in fact, be grindier. The random system ensures that we make matches that are not helpful in terms of better understanding the relative skills of the players. So it takes longer for us to do so (and, indeed, we probably won’t ever understand them as well as we can with the 50/50 matchmaker), because we have to wait until we randomly get a match that lets us test those fine gradations in ability level.

And, again, players tested this on ranked. We don’t have to think about this theoretically- they literally manipulated things so that the matchmaker could not make 50/50 matches for them. The result: those players ranked up indefinitely but at an incredibly slow pace. Their ladder experience was inaccurate (because they could not lose a match and could not rank down) but really grindy (because they only got 1-2 SR per match won.)

5 Likes
191

Bro you’re over here writing 1000 page novels full of very explicit and clear cut information.

  • It can’t possibly be this difficult to understand from a 3rd person’s perspective.

At this point you’ve written your explanation in at least 4 different ways on this forum from the 3 different threads I’ve seen you comment on.

I have to admit, your tenacity is superior to mine.

Appreciate you Boo :kissing_heart:

Edit: You tell good tales, Tales

5 Likes
192

Ask WyomingMyst. I don’t speak for him, but I read the Scott Mercer post the day it was posted.

Meanwhile, there is indirect evidence that such a post existed: Like `https://variety.com/2018/gaming/news/overwatch-matchmaking-myths-1202856486/

It’s a taller order to find a journal-quality reference with the complete content and relying on memory from an event 4-5 years ago is also questionable; but nothing Cuthbert quoted seemed out of place.

1 Like
193

Sure we can, which is why I stated that after 5 wins they progress to the next grade (one would expect 6th graders to be better than 5th graders, and so on).

See previous response. Each win places each player closer to their natural skill tier, so yes we do gain information by virtue of each win. And yes, the initial sorting process takes time, but it takes time in every scenario. Consider how many tens of hours a player must play in OW in order to get to their true skill. It seems to me that there are two tracks here: sorting (information gathering) and win rate. And now that I think about it, perhaps a third: how much OW conforms to expectations with respect to “real world” competitive environments.

My point is that Steve eventually gets to where he should be, but his win rate on the way there is drastically altered based on the sorting model.

So the solution is to implement anti-competitive practices into competitive environments? This is the crux of this debate: it hinges on the question of what is a competitive gaming environment supposed to be. There are countless game modes where people can play with friends who are much worse. There are tons of game modes that aren’t competitive at all. I don’t think it’s unreasonable to have at least ONE game mode that is a pure and competitive environment for players who want such an environment. Allowing players 1000 SR apart is quite frankly awful for competitive matchmaking, particularly because Blizzard is so liberal in how it goes about accounting for that spread on the opposing team.

This is one way of looking at it. Another way of looking at it is: it’s a profit-driven decision that makes the gaming environment worse, grindier, and more frustrating for players who don’t come to competitive to socialize. Both sets of players paid good money for the game, and as stated, there are a ton of game modes where you can be social. Competitive OW is disaster these days, and every single person I know who hates the game mode does so because it feels like a collection of manipulations and compromises. There should be a purely competitive mode for players who want that experience.

You’re answering a question with a question, and seem to be ignoring the context of the original question.

But to answer, I wouldn’t make it engagement based. Chess, tennis, basketball, hockey, and half a million other activities are not “engagement based.” They offer inherent challenges, rewards, and satisfaction by their nature. I think engagement is the natural consequence of a well designed game. I think Blizzard, out of greed and self-interest, institutes various forms of competitive manipulation into the equation to maximize profit, and it does so due to the findings related to the profitability of engagement based matchmaking. And they’re obviously free to do so, but it doesn’t mean I can’t criticize them for it.

2 Likes
194

No, that’s not what I’m bringing up; I’m bringing up your reluctance to delve into a complex multi-faceted topic on its merits. And since I’m raising objections to all the points/errors you’re making, I’m practicing what I preach. If you think the proper course is to ignore points of disagreement, you don’t understand how debate works.

If you want easy and quick, why do you keep replying to me?

I was going to type something snarky here, but I stopped out respect for the discourse. It’d be a shame for this devolve into insults and whatnot, but if that’s the road you want to go down, I can do that too.

You seem to have this apprehension that I follow you around, reading wherever and whatever you post, essentially collecting your thoughts on various topics wherever they’re scattered throughout these forums. And even if that were true, you’re providing your perspective (I know you do not merely consider it your perspective, but rather the dispensation of Ultimate Wisdom) in the context of conversations with OTHER people. Do you understand that circuit debaters discuss the same topics over and over and over with different people? Do you understand why? Do you understand why they don’t stand up at the podium, say “Please refer to my body of work on these topics,” and leave?

YOU replied to ME, I didn’t reply to you. I just stated last week that we have fundamentally opposing views on the matchmaker, so I’m not sure how useful these discussions are. But I don’t mind thinking through these matters and writing about them as it seems more useful than popping on Netflix. If you consider it a chore, you shouldn’t do it.

Again, you’re asserting that your positions are unassailable dispensations of ultimate truth. They are not. I find your some of your explanations either flawed or incomplete; and I find some of your pronouncements not universally applicable (what makes sense in one context may not work in another). Even if some of the things you say do apply to other domains/situations, you’ve still got work to do to prove that they apply here.

And I’ll restate the same point I’ve made many times before too. People have the nebulous feeling that something is wrong because every competitive or skill based experience that people have in real life, feels and IS fundamentally different than it is in Overwatch. You study more for a test, you immediately do better. You add 15 miles an hour to your serve, you immediately win more games. You come up with 10 new plays as a quarterback, you win more games. Lag and lead indicators in real life are very different than they are in OW. People simply do not often have and have never had the experience of practicing X amount of hours at something to reach skill level Y, going into a competitive environment, and then having another person already at skill level Y selectively dropped in front of them. So yes, people are going to have a nebulous feeling that “something is wrong.”

And if your arguments are so air tight, there should be no room to question them. I have seen exactly one irrefutable argument from you, and I changed my position. Everything else has been questionable, or you make claims about how things work that you cannot possibly know (since your claims would absolutely require knowledge of the OW code base).

This analogy doesn’t work. I also reject the idea that a discussion about complex and nuanced systems should be devoid of, you know, discussing complex and nuanced systems.

And it doesn’t take “forever.” Like I said, I reply to every single paragraph you or anyone else writes – it’s really not that hard, and doesn’t take that long. Maybe look into a typing class?

Of course it would be easier to cherrypick or ignore various points, but this isn’t about convenience. It’s about having a full and complete conversation. And it’s ridiculous to essentially deride someone on the basis “well gee, you want to me to actually be ready to defend EVERYTHING I’m saying? Can’t you just do me a solid and blindly accept 30-40% of it.” No. I can’t do you that solid.

If what you say about your background is true, I’d imagine amassing that understanding took considerably longer than the 10 minutes it takes to read one of your posts. You have to get it out of your head that people are replying to be difficult. You should also maybe do a better job of considering counter-arguments and working them into your replies. And some of the things you say, just fail to convince, or make fact claims that you cannot possibly support. Really. Quite literally, you’d have to see the code base to make some of the claims you make with the degree of confidence you make them. And I know you have not, so I HAVE TO reject those things as unfounded. I make inferences. You routinely make fact-claims, and with them always comes the burden of proof, which you cannot produce.

To be honest, I think you hold the quality and comprehensiveness of your explanations in higher regard than is warranted.

No. Just no. There are claims you – cannot – make without seeing the code. But you never let that stop you from making such claims. And when confronted with this inconvenient fact, you want to fall back on your experience in a related field, or other post hoc fallacies. You will make a claim like “The matchmaker does thing X because of reason Y. What people fail to understand about the matchmaker is thing Z.” The problem is you want to sound authoritative about a topic for which you have insufficient evidence, not because you have no prior experience, but because it violates how logic works. And you make all sorts of logical errors. You’re frustrated by the fact that people can always reply “you cannot know that to be the case” in response to – things which you cannot know to be the case. If you want to stop running into these types of burdens of proof, stop making fact-claims. It’s that simple.

You know what propositional logic is. “P is true.” “P implies Q.” Etc. You constantly make claims about what is true, without seemingly understanding that you have no proof for those claims (you frequently confuse evidence with proof). So when you say “P is true” based on speculation, based on reasoned analysis, based on, I don’t know, some strongly held belief that you can reason the specific from the general, expect to run it to people saying “hey, you can’t know that!” It’s just how it works. Stop making so many fact claims and start supplying inferences if you don’t want to be bothered with producing proof commensurate with your claims.

So the solution is simple: stop making the kinds of claims that require seeing the code base.

And it will always be possible to assume the best intentions if you completely ignore the existence of all order of manipulative and shady practices frequently instituted by developers.

I think you fundamentally misunderstand that deductive reasoning can lead to false conclusions. And as I’ve already stated, I believe 50/50 matchmaking may potentially have some sorting advantages, but that the bad outweighs the good. I value transparency over expediency. You don’t. What 50/50 match making gains in expediency it loses in transparency. Get rid of MMR and with it you are guaranteed that EBMM matchmaking cannot be part of the equation. I cannot help that you value this aspect of competitiove integrity differently than I do, and vice versa.

2 Likes
195

Curious. Do you get this level of pushback from your GMAT clients?

I’d like to think people taking the GMAT would be better than this, but that may be wishful thinking on my part.

2 Likes
196

All right. So, it appears that you believe that getting rid of 50/50 matchmaking and getting rid of MMR are both useful things to do in some way. That there is some disadvantage in having them (although there might also be some advantage in having 50/50 matchmaking as well), and that we would be better off without them.

What you would get, then, is a version of the original Elo system that was developed around 60 years ago by Arpad Elo (although, even then people understood that random matches would be counter-productive- the earliest conversations about the Elo system in the chess world were about how best to make matches between players so that you could achieve more accurate rankings). You would be removing all of the advances over the intervening 60 years. But let’s play that scenario out, so that we can better get at what those changes would accomplish:

  1. A player enters their first competitive match. They are matched randomly with 5 other players on their team and against 6 random players on the opposite team. As our system is just starting out, we have no information about any of these players. I will also assume for the moment that we are not factoring individual performance into this streamlined system, because there is no hidden MMR that could be used to factor that data in.

Our player wins the match.

What do we learn? Not much, but we have to give this player a rating because they completed this match- even if there is no SR rating until placements are complete, we must have some sort of internal rating that changes after this match, otherwise we accomplished precisely nothing in making this match.

So, we assign every player who won that match a rating of… I don’t know 2200… and every player who lost that match a rating of… let’s say 2100.

Now, we make some more matches. These matches are also made randomly. So we have some of our 2200 players (bear in mind that these players could actually have anything from Bronze skill to high GM skill- we really have no way of knowing at this point) in matches with several other 2200 players on their team and some in matches with no other 2200 players on their team. Similarly, the opposing teams could have any mixture of player skill.

So we have 2 matches under our belt with players that could have been any skill level against others who could have been any skill level. What do we know at this point? We’ve actually got something like 4 skill levels now (again, if we aren’t tracking a hidden skill rating that differs from our publicly viewable ladder ranking- if we are we have much more information and more skill bands) with which we can rank our ladder- folks who won both matches, those who lost both matches, and some variation among the people who won one match but lost the other (we might give more value to the second match because we have some pittance more information at that point).

How does this look in a world in which we make 50/50 matches and track a hidden MMR (which factors in things like personal performance within matches that you win or lose) in addition to the publicly viewable SR?

Our first competitive match is made in a world in which we already have a great deal of information about the players- that’s why they have to play enough matches to reach level 25; it primes our hidden MMR.

So we aren’t making matches between GM players and Bronze players. Instead, our top tier competitors are matched against each other and our lowest tier competitors are matched against each other right out of the gate. In this way, we are able to meaningfully test our assessment of their relative skills. Players who beat GM players can be much more easily sorted than players who beat some random assortment of players. We can much more easily determine whether a top tier player deserves to be Diamond or deserves to be GM. Similarly we can much more easily determine whether a low tier player deserves to be low gold or bronze.

And this happens after every match. Each time a match is made in this directed fashion we gain much, much more information than we would have if we didn’t track hidden MMR and if we did not strive for 50/50 matches.

So our system is less grindy (it takes people less time to reach an accurate rank- again, because we are learning as much as possible with each match made, rather than the tiny bit we would learn with random matches and no tracking of skill apart from the publicly visible ladder ranking) and it is more accurate. We are more quickly able to form accurate predictions about the outcomes of the matches we make (which is how this stuff is measured when you, say, write a paper exploring different competitive ranking algorithms- how quickly can system A accurately predict the outcome of a match vs how quickly system B can. The prediction compared to the match outcome is the test of how accurately your system is ranking the participants’ skills.)

Those are the benefits of tracking a hidden skill rating and of making matches with a predicted 50/50 outcome (a less grindy experience and more accurate rankings). What are the benefits of getting rid of them? As far as I can tell the only real benefit you’ve offered up is that some people will have higher win rates (and others will have lower win rates). But that’s not a competitive benefit. It makes it harder to assess the relative skill levels of the players in such a system. And it makes the whole system grindier- because we actually do not know whether someone is diamond or GM until we can randomly find some matches that they will lose. And that takes a lot longer in a random system.

So, while you might want a higher win rate (or others to have a lower win rate), it hinders the entire purpose of the competitive ranked mode- accurately ranking the players according to their skill.

And your notion of EBMM seems to assume several things that are counter-factual:

  1. As I have demonstrated, and as we have seen on ladder- 50/50 matchmaking makes the system less grindy. How is that EBMM? Your proposed system means that many players (particularly those at the top and the bottom of the ladder) would have to play many, many more matches in order to attain their true rank. That sounds more like EBMM- if your changes were implemented they would lead to a much grindier experience for players to reach their true rank.

  2. There are already systems in place to drive player engagement (and there almost certainly will be more in OW2). They are things like seasonal events, mini-events, arcade loot boxes, etc. Looking for engagement driven systems in the competitive mode is largely barking up the wrong tree.

  3. The competitive mode sheds players, and largely didn’t appeal to the majority of players even when OW was more popular. 50/50 matchmaking makes for an incredibly competitive experience- most people don’t like those. They, like you, want to win 90% of their games. So the 50/50 matchmaker does the opposite of what you suppose it does- it makes things frustrating for all but the most competitive minded of players, rather than driving players to engage more in the mode, they opt out.

  4. They could easily have all sorts of engagement based systems in place without a hidden MMR or 50/50 matchmaking. When OW2 comes out with whatever integrated battle-pass or similar system they implement, we will all see what engagement based game design looks like in the OW context. My prediction is that little or none of that will force people to engage with the competitive mode. That’s not where the money is. People will still be able to play quickplay or arcade modes to check the boxes on whatever system they use and get the in-game rewards.

Competitive mode is for something else- that’s why it uses the very systems you find frustrating. It doesn’t care if you are frustrated, it mainly cares about trying to assess the relative skill levels of the players in the competitive mode.

I do agree with you on some points, though. I’ve told you before that I think the 1000 SR difference and the way grouping is handled are compromises that undermine the competitive integrity of the ladder. (That is, of course, why they have different standards at the top of the ladder- there is more of an emphasis on competitive integrity in Masters and GM than there is on the rest of the ladder.) I think a real conversation could be had there about the values of the community as a whole and to what extent people feel that those compromises are permissible in order to allow folks who aren’t really pushing the top of the ladder to play with their friends.

It’s just hard to have those conversations when people obsess over things like hidden MMR and 50/50 matchmaking.

2 Likes
197

Like I said, WyomingMyst did not make a comprehensive archive of the Battlenet forum. It does not include most of the Battlenet forum contents, and it does happen to be missing the statement from Scott Mercer that I excerpted in 2016. That doesn’t mean WyomingMyst is a bad actor. It also doesn’t mean that Mercer never made the statement, or that I made it up. Feel free to jump to any other conclusion.

198

Honest question: Are you Mhz, from the original forums?

1 Like
199

There are no discussions held on these forums that can reasonably be called necessary

wanted? yes

enjoyable? sometimes

necessary? never, to the best of my knowledge

2 Likes
200

This is something I’ve tried to get across before. It’s not that I trust Blizzard to not use a matchmaking algorithm that prioritizes something other than matching skill level, it’s that the frustrations people express wouldn’t exist on a “engagement” system and we don’t see the problems that would exist if it were based purely on ping.

Now, maybe some people LIKE the pure ping based approach. If you’re in the top 50% of players but not in the top 25%, I can see how you’d have challenging matches interspersed with lots of wins. Below that is miserable stomps, above that it’s boring games.

An EOMM could give that for nearly all players, reducing a lot of frustration. The frustration is itself evidence that they don’t use this model. Maybe they should! From the paper i read there’s really no reason to not intersperse fun “exhibition” games into the mix, as long as the overall win rate stays near 50%.

I can buy that Blizzard is a bad actor, sure. But I have a hard time believing that they have some complicated system that produces the results you expect from TrueSkill, but deliberately makes it a worse experience, especially since it’s not clear where the profit motive would be here.

Doesn’t it make more sense that the system and it’s results are just as simple as we describe it, with it’s faults and all? Maybe it is a Cartesian demon choosing winners and losers based on behavior in a past life, that ALSO fits the evidence we have, but is it more likely than a TrueSkill type system?

I mean, if everyone had a 75% win rate THEN I’d be very suspicious. Everyone having a 50% win rate is normal and expected, isn’t it, given the system we describe?

3 Likes
201

Why are they playing against 5th graders in the first place? Are they higher in reality but the system thinks they are 5th graders? In that case: That is pretty much how the Overwatch matchmaker works, so no controversy here.

Or does the matchmaker know they aren’t 5th graders? Then why would they “rise” to face 6th graders next? Aren’t you proposing completely random matchmaking?

It can’t be both.

If you want to call it like that, maybe. But fact is that the developers and probably a large portion of the player base have a different opinion. This is still a game and the first and foremost objective is to have fun. Yes, even in the most competitive mode! The ability to form groups is not really debatable. There was already quite a bit of backlash when Blizzard implemented the group limit in GM. Also the impact on lower ranks is small anyway.

If you think competitive should be solo queue only I can relate to that. This would make matchmaking the easiest. I think in LoL there are two different queue modes? (I don’t play the game.) Apparently that creates different problems though so they are now investigating a solo only mode.[1]

If anything competitive should be full-stack only. That was Jeff Kaplan’s original intention and I hope there will be some kind of team queue or tournament mode in OW2.

I think you misunderstood the question here.

Imagine I am Darth Kotick, CEO of Death Star Entertainment Inc, and have just run my massive yacht a-ground. Now I need more money to buy another, even more massive and less sunken yacht.

We have recently released our new team shooter, where you can play your favourite storm troopers! Sadly our developers have created a very good skill based matchmaker, that is apparently not generating enough money.

Luckily, I have made you, the brightest of Titans, the new head of the nefarious activities department. Your job is now to alter the matchmaker to make the maximum amount of money in the shortest possible time, so I can buy that yacht. I don’t care about the long term impact, since we’re going to be bought by Mordor next year anyway.

I want to get to the bottom of what exactly it is, that makes the matchmaker “engagement based” vs “skill based” so we can see if the Overwatch matchmaker actually does any of that. And we can get closer to the issue by imagining what Blizzard could do if they wanted and what that would have for consequences for the matchmaker and for us as players.

It could for example put a player who bought lots of skins in every match to encourage the other players to also buy lots of skins.

Exactly this! I find it highly unplausible Blizzard and especially the Team 4 devs would create a system that makes playing the game less enjoyable for some vague monetary gain while directly driving players away. The players who would actually spend the money on a great game.

Accusing the developers of this would be very disrespectful without a lot of solid evidence.

[1]: https://www.leagueoflegends.com/en-us/news/dev/the-state-of-competitive-spring-2022/

2 Likes