OW's feature that can break the MM system

General Discussion
1

Avoiding players.

I’ve learned that video games nowadays match players up by something called EOMM, Engagement Optimized MatchMaking. It’s goal is to minimize players’ “churn risk”, meaning how likely they are to take a break from the game. That’s why we are all experiencing so many lopsided games that are essentially rigged by the MM. The MM’s main priority isn’t match quality, it’s “player retention”.

Players who have gone on mad winstreaks are less likely to quit the game in a short-sighted perspective, but it obviously has to be compensated for. How to avoid the losingstreaks is speculative. My theory is that you should take a break after losing, that way the system learns that you are not a player that it can send on mad losingstreaks without you quitting out.

EOMM really sucks for the players in general, but OW has one unique feature that makes it fail spectacularly: avoiding players as teammates.

When a player avoids another player, it becomes the MM system’s highest priority. It can never ignore this factor, it has to adjust everyhing else around it. When many players are avoiding each other, it breaks the MM’s ability to rig who’s gonna win and who’s gonna lose. This becomes especially true at offpeak hours, when the MM has no choice but to keep putting the same players in the same games.

I have experienced this firsthand, as many of you probably also have. I’ve been on both the losing end and the winning end of the equation.

I only play open queue competitive, never quick play or any other mode. I normally hover between dia5 and plat2. I could possibly climb higher if I only played my mains and really tried. But with how the stupid MM system works, that doesn’t feel too appealing, since you’re rigged to be losing sometimes anyway, no matter how hard you try.

But one day when I had already climbed to dia5 and was surely due for a losingstreak, I noticed a massive troll in my game, so I avoided him and queued up again. This was at nighttime which meant that as usual, we were all the same players in the next game. The troll was on the other team, and was still trolling just as hard. I kept queueing up and so did the troll. Eventually I ended up winning 7 games in a row for free. This propelled me to dia3 for the first time, I was stunned by how easily I had managed to abuse the MM system to gain a new highest ranking.

So the next day when I came back to play, also at nighttime, what do you think happened? Some supergood player started avoiding me for some reason. It was after a game we had won, but I guess he somehow identified me as the weakest player and was an experienced abuser of the system. He played Soldier every game and was easily the best player in every game. I became tilted and just like the troll the day before, I kept queueing up and kept getting wrecked, except I was actually trying. I ended up going 5-14 and dropped back to platinum.

This particular case is not a big deal to me, I’m just telling the story for the purpose of illustrating how avoiding players breaks the MM system. Because the MM system being as bad as it is, is a bit of a bigger deal to me.

5 Likes
2

It reminds me of this sentence someone said months ago, when I was playing Overwatch before my big break.

The MM spends his time abusing you, the only thing you have against that is to abuse the system yourself. Nothing else will work.

It was months ago. Nothing has changed.

Nice post though.
Too bad, it will probably be ignored.
Like all those topics who goes a little further from the usual whining and tries to raise the level of analysis.

1 Like
3

Very interesting conclusion. I personally don’t play a lot of comp, especially in Plat, where I’ve heard matches hurt the most, so it doesn’t affect me much, but I do appreciate your thought-out conclusion. I wish the MM system was better as much as the next person, and I can only hope they do better in Season 9.

4

This isn’t actually true though. If you read the EOMM paper that everyone bases these theories on, long win streaks increase churn. In fact, there’s only one scenario that puts a player more at risk of churn than a win streak and that’s a losing streak. What you want in an EOMM is to have really close matches. Draws are the best outcome.

So if you are looking for evidence that OW uses an EOMM, what you should look for is more draws and close matches than you would expect and very few win/loss streaks.

5

As far as I can conclude, winstreaks are pretty good for minimizing churn risk, but they’re not optimal.

In the study, they initially determine how often the last 3 games makes a player take a 7-day break from the game.

This is what the table in the study looks like. I added all the hyphens (----) in an attempt to make it look better on the forum.

D = Draw. L = Lose. W = Win. x% = Churn risk.

Last 3 Outcomes --------------- Churn Risk
DLW | LLW | LDW | DDD — 2.6% - 2.7%
… …
WWW ------------------------------ 3.7%
… …
DLL | LWL | LDL -------------- 4.6% - 4.7%
WWL ------------------------------ 4.9%
LLL -------------------------------- 5.1%

The data in that table isn’t what they use for their case study though. They create their own “Churn prediction model”, which is much more complex. As far as I can tell, they don’t actually disclose what kind of W/L/D patterns minimizes churn risk according to their model.

If I’m missing something here, I’d be very happy if you could point it out. Because this paper is not an easy read.

To perform their study they then go on to apply four different matchmaking algorithms to their churn prediction model. SBMM (Skill-Based MM) and their own EOMM (Engagement Optimized MM), are among them. They conclude that EOMM outperforms the other algorithms when it comes to churn risk.

Of course it outperforms them, it’s sole purpose is to minimize churn risk. That is not the purpose of the other algorithms. But it still doesn’t outperform them by much.

When their sample size is 500 players, EOMM retains 261 players per round of MM, while SBMM retains 259 players per round of MM.

When their sample size is 100 players, SBMM actually wins. EOMM then retains 51.9 players per round of MM, and SBMM 52.5 players per round of MM.

Those numbers are for every single round of matchmaking, so the difference accumulates the more rounds are simulated.

The fact that SBMM is better for player retention in the 100-player sample size, is the only issue that they raise in the paper’s discussion. To me, this study has lots of other issues that should have been pointed out. There are more inconsistencies in the numbers. But the most important thing to point out would be that this study was performed on data from a 1-vs-1 game from EA. They factor in “goal difference” in their churn prediction model, so I assume it’s a sports game. That’s a far cry from other types of games, such as Overwatch. Perhaps that kind of analysis falls outside of the scope of the study, but I still think the discussion-section would be the proper place to point that out.

I fear that the results of this paper have been applied to a far too great of an extent in many games where it’s not a good fit. If developers really choose to throw higher match quality out the window, in order to increase short-term player retention by a small amount, it’s surely a very bad decision for the game in the long-term.

I’m not allowed to link the study directly, so I have to link you this reddit-post: Reddit - Dive into anything where you can go on to click on a link to the study itself.

6

Short-term decisions are that characterized the Kotick era.
With the requirement maximum profitability with minimum investment.
Also : automate everything that can be automated, without taking care of the quality (or lack of) of the result.

7

I do that by paying a friend to throw games. Doesn’t sound like a MM issue

/s

8

You’re obviously trolling, but I’m gonna respond anyway.

If you can pay a friend to avoid him, and have him throw games for the opposing team. Then that’s a huge MM issue (if you consider the “avoid”-function to be part of the MM).

9

Yeah. Look how nice draws are. Draws are what reduce churn. If you are optimizing for engagement you want even matches. You can absorb wins more than losses, but the ideal state is draws + wins matched with losses.

Neither loss streaks nor win streaks will optimize for engagement. It’s even matches that optimize for engagement. And loss streaks are the worst. You also want people to be trending upward, so a win after a loss or two isn’t too bad but a loss after a couple of wins is pretty rough.

So, again, if OW is using an EOMM, we’d see draws and even matches, but almost no loss streaks and the occasional win streak. You’d also have people’s win rates trend upward if you could. But this isn’t really what people complain about. What people complain about is streaks (particularly loss streaks), stomps, and their win rates either remaining low or trending downward.

And that makes sense, that’s what the data tells us. The data tells us that people would prefer more even matches. And the data tells us that people hate loss streaks. And the data tells us that people would prefer their win rate to trend upward. So it makes sense that people complain about these things.

What makes zero sense is when people suggest that Blizzard is trying to increase engagement by skewing the matchmaker such that players get more loss streaks or more stomps or a decreasing win rate. If Blizzard were doing that it would be the opposite of an EOMM, it would be a disengagement optimized matchmaker. And that would go against their financial self-interest.

People are just frustrated for all the reasons we would expect them to be frustrated and want to blame that on something even though their theory makes zero sense.

10

It’s SombrasFeet9.

None of his posts are serious, but is not a troll, rather a forum animator based on jokes.
As soon as you understand that, all these posts are a treat to read.

1 Like
11

When it comes to the stomps/lopsided games, I definitely think any implementation of EOMM would create a lot of them, since the goal of EOMM is to create a certain pattern of wins, losses and draws in your match history. That would naturally make the system rig stomps to achieve that kind of match history pattern. It would only attempt to rig a draw when your current match history is set up for it.

A system that instead focused on actual match quality, like SBMM, would not create as many stomps. Since it’s main goal is the quality of every single match, rather than creating a certain match history pattern.

In that small table we’re provided with L, D, W patterns and churn risks, one thing that all the cases with the lowest churn risk have in common, is that they all end with a win, except for the DDD-case. And all the cases with the highest churn risk end with a loss. That’s a more obvious trend in the data than draws being beneficial. Although I do agree that it seems to like draws. I really wish they had disclosed the data from their more complex churn prediction model.

The win- and losingstreaks is another thing than the stomps. I think it’s connected to EOMM, but it’s not as clear cut, only judging by that table. In OW, players avoiding other players likely plays a big role in it too.

To summarize, I don’t think the conclusion that EOMM would create more even matches is correct, I instead think it creates a lot of lopsided games. But I do agree with you that the data in that paper does not show that EOMM prefers streaks.

12

EOMM wouldn’t work in this game, not even SBMM works well. circumstance is too varied and too big of an influencer. end of story really.

1 Like
13

But why? This is like saying that if one wants to maximize their income they’ll try to be unemployed. All the data we have suggests that close matches are the best way to optimize engagement, and that loss streaks are the worst way to optimize engagement. You cannot get draws with lopsided matches. You would need to minimize lopsided matches. What you are suggesting runs directly counter to the data.

Like, it’s interesting that draws are the most represented outcome among those with the lowest churn rates. Why do you suppose that is?

14

Well, It seems correct, systems adapted to 1vs1 mirror, it does not work very well with team vs team and different characters that we can change several times during a match.

15

Put simply.

The matchmaking system is desinged to enforce minimum and maximum win rate thresholds…

The reason for these thresholds is based on Market data, which shows that complexity is the enemy of profitability in the free to play market…

Overwatch is a complex game that doesnt look complex at face value.

The matchmaking system is basically designed to embellish the experience for players who would otherwise struggle with or be driven away by the games learning curve…

Based on Market data those are the people who are most likely to spend money. So the matchmaking serves to improve the experience of those players, with the expectation it makes them more likely to spend money…

Then relying on sunk cost fallacy to retain them

This extremly simplified…

16

Which is ironic, because you would think better match quality would lead to better retention.

1 Like
17

Let’s talk human psychology. By simplifying obviously, we are not PhD psychologists.

Human beings are very sensitive to change, and invariability bores them.

If you only play balanced matches, you end up not realizing that they are balanced and appreciating that they are.
At some point you have to experience “stomp” matches to rediscover the pleasure of balanced matches.
If you only have balanced matches, you get into a routine and you will quickly look elsewhere for something that will allow you to break the boredom of the routine.

Balanced matches are good. Impossible to say otherwise unless you’re a masochist.
Too many balanced matches have a counterproductive effect, and instead of retention, it can encourage people to look elsewhere.

18

You agree then that in an EOMM what you would want would be mostly balanced matches and very few loss streaks? You just think it couldn’t always be a super balanced match. That makes sense. It’s just that what people complain about when they start hypothesizing that OW uses an EOMM is that there are two many loss streaks and too many stomps.

But that’s what you’d get when things were not set up for engagement optimization. It’s literally the opposite of engagement optimization. That’s why it’s so bizarre when people use EOMM theories to complain about things like loss streaks or stomps or whatever.

If people are complaining about EOMM it would really only make sense to complain that there are too few stomps and too few loss streaks. Like, if someone posted a thread that said, “I’m tired of balanced matches and I want more loss streaks- I really think they’re using an EOMM and I wish they would stop,” that would make sense.

But I’ve never read that thread. People who theorize Blizzard is using an EOMM and complain always complain about the opposite of what an EOMM would promote.

It’s just weird.

19

We’re both basing our assumptions on that same table of W/L/D = x% churn risk, since it’s all we have. But it’s good to keep it simple to understand the fundamental principles.

You’re arguing that there is a correlation between draws and low churn risk in the table, which there is.

But trying to set up a draw is risky in any game, there’s a big chance that one side is going to lose.

If a certain player needs a win to avoid his churn risk going up too much, the system cannot risk trying to create a draw, it has to try to give that player a win. Only when all players involved are already in a “safe state”, can the system attempt to set up a draw.

Therefore, when you apply this to a game of 5on5, very rarely will all the players be in a safe state. The players who’s churn risks are getting high, simply needs to be put on the winning team and the players who are in a safe state on the losing team. How the match actually plays out is less important, as long as the MM system gets the result it needs to ensure that all churn risks are in check.

Now in reality, Blizzard has definitely got a much more advanced way of defining churn risk than that table. In fact, it could very well be something that is continuously evaluated by an AI. The AI might define every individual player’s churn risk separately based on their historic behaviour, and on other players that it deems to be behaving similarly. It could take anything into account, from amount of words typed in chat to eliminations, as long as it finds a correlation between it and the player’s churn risk.

For example, a certain player might tend to keep on playing even if he loses, meaning that losing impacts his individual churn risk less than others. These players would be like golden nuggets for the MM, they can safely be sent on losingstreaks. Then when they’ve ended up in lower ranking, they can be the carry who carries the players that need a win.

Now this last part is pretty much a conspiracy theory, but it’s my take on why the streaks are allowed to happen.

Why EOMM ends up with many lopsided games is pretty clear, even if it likes draws. The streaks are harder to explain.

1 Like
20

You are hypothesizing a system that no one uses, by the way. When you start discussing individual players’ tendencies you have to realize that you are so far beyond anything currently in use that it doesn’t really have any relevance in a discussion about what is actually happening in game.

You can certainly imagine what might be possible years or decades from now, but a discussion about what is happening currently is a very different sort of thing.

I also think it’s important to recognize when one is reasoning backward from a conclusion rather than reasoning forward from evidence.

Here’s what we know:

  1. Various devs including Morgan Maddren (who currently heads up the team that works on the matchmaker) have described the OW matchmaker over the past 7 years. They have repeatedly and in every instance described a matchmaker that is not an EOMM. They have, in fact, said that OW does not use an EOMM.
  2. There is one paper that had nothing to do with Blizzard that described a hypothetical EOMM and suggested that they could increase player engagement using their hypothetical EOMM over the traditional SBMMs that were in use at the time.
  3. The paper used data from an Activision game.
  4. The paper found a statistically significant increase in player engagement using their hypothetical EOMM. (We’re talking a percentage point or two).
  5. The paper found that churn risk was highest when players repeatedly lost.
  6. The paper found draws to be the most common result when churn risk was lowest.
  7. The paper used a matchmaker that did not create individual profiles for specific players but rather used the same data set to determine what outcomes were most desirable for any given player.

Now, you can assume all sorts of things that go against the above 7 points. But, at that point, you are inventing things in order to support a pre-existing conclusion rather than looking at what is known in order to understand the current state of the game.

I think it’s worth asking yourself just how adept you think folks like Aaron Keller and Morgan Maddren are at repeatedly lying about something like this in a way that they never slip up, and just how likely you think it is that some disgruntled employee or other would not at some point even hint that everything they’ve ever said about the OW matchmaker has been bald-faced lies.

One has to believe a lot of really weird things in order for this hypothesis to seem plausible.