Mercy is being ignored?

[Arceus-2238]

It’s a fact that she’s underpowered as of now. Statistics is our evidence. Blizzard is most definitely wrong.

[Hollypop-21276]

Unreliable information… I know there are people claiming Overbuff is still reliable, but their data set is an example of sample bias because of the private profiles. A factor that until we can figure out what the leniency should be, we cannot use as reliable information.

[GTIX-21225]

she doesn’t need to be priority…
but saying that ‘the support changes’ were succesfull also includes the bad decisions they made.

they had well over a year to listen to us and they did nothing with it, nerfing her healing (which has been buffed before because it wasn’t enough) is a lazy fix for the real problem thats still present

[InTheDesert-1983]

I get what you mean, you’re saying since I think it wasn’t successful for mercy I think they’re ignoring it. But I still think they were ignoring the problem and ignoring the biggest topic on the forums lol like to me, the problem is that they were so vague about it. They didn’t make it clear that they meant it was successful for every support so it leaves us wondering are they thinking of changing mercy, are they saying she’s fine but being really vague about it

[Hollypop-21276]

Decisions you personally perceive as bad. They also don’t have to listen to a word we say because we may not actually be right - never mind that our voices are divided.

The data that Blizzard have is more reliable than we can pull from Overbuff, and while we’re able to suggest ideas, we are not owed anything to have them used by the devs. It’s their game, they’re the professionals who know what they’re doing. They take our ideas into consideration, not as gospel.

[Arceus-2238]

Except after the private profile patch, the statistics barely changed. Also, someone made a long post explaining why it’s still reliable:

@Agnostika-1802

I’ve seen a lot of people claim that Overbuff’s numbers are meaningless due to private profiles (or due to the fact that even before private profiles Overbuff only received statistics from a subset of the player base). This claim is false. You absolutely can make very good claims about a large data set by sampling a large piece of it at random.

Here’s an example. A very big bag is full of blue tokens and red tokens. The bag is shaken thoroughly to mix up the tokens. Then you spill 100,000 token on the floor. Of these, 60,000 are blue and 40,000 are red. The remaining tokens stay in the bag (in the Overwatch context, these are the “private” tokens). What percent of the tokens in the bag are blue?

You can’t know for sure without looking at every single token. What you can do is make claims, and then give probabilities that they’re true. The bag should contain about 60% blue tokens and 40% red tokens. If you claim that the bag has exactly 60% blue tokens, then you are probably wrong. If you claim that the bag has somewhere between 59%-61% blue tokens then you are almost surely right (with a probability that can be calculated and shown to be very very close to 100%). The more tokens you draw from the bag, the closer the percentage should be to the actual percentage in the whole bag. In math, we call this the law of large numbers.

You end up replacing a certain claim (exactly 60%) with a more nuanced one, together with a measure of how sure you are about it. This shouldn’t worry you. Your entire life is lived like this. For instance, your brain processes information from your eyes at a certain frame rate, and your eyes themselves have many missing pixels (so there are “private” pixels and “private” moments in time). Everything you see at all times is based off of probability. That’s the way almost all of human experience works.

The situation in Overwatch is almost identical, except we’re not sampling entirely at random. For instance, we’re only sampling people with public profiles. The good news is that there’s excellent reason (supported by data analysis) to believe that private profiles aren’t correlated with, say, skill at playing Roadhog. So the conclusions still hold.

So, now on to the main question that I wanted to discuss. How much should you trust Overbuff data? Like in the bag of tokens example, it really depends on how many games Overbuff records. If you drew 2 tokens out of the bag, your conclusions would be off. If you drew a trillion, they would be really good. So, how many games does Overbuff record?

We can’t know exactly without someone from Overbuff chiming in, but we can get a very good estimate. The thing that comes into play here, and will be important for the discussion later on, is a beautiful theorem called the central limit theorem. I won’t state the theorem itself, since it’s a bit too technical for a forum post, but it deals exactly with the kind of situation we have here - repeating a random event over and over again and trying to understand its averages. The central limit theorem is the source of the bell curves that you might have seen pop up in all kinds of aspects of your life.

Suppose you record N games in which Mercy was played (I told you there would be a little bit of Mercy here). Suppose that Mercy is balanced such that she should win about b% of the time (and b% is pretty close to 50% so the standard deviation for one Mercy game is close to 1/2), but that in those games she won about a%. How far apart can we reasonably expect a% and b% to be? One thing that the central limit theorem (and its companion, the Berry-Esseen theorem) says is that you should expect about 66% percent chance for a% and b% to be within about 100/2 sqrt(N)% of each other (i.e within one standard deviation).

So I looked at the time period between late June (when private profiles were introduced) and early August (when the support balance patch came in) and found several characters whose pick rates were stable (Mercy was one of them). For each day, I found out how far the win rates for that day were from the average win rate during that time period. Then I found the percentage that 66% of the results were smaller than and solved for N.

Now, this is only an estimation but the upshot of it is that Overbuff
records around 80,000 games a day. Using Blizzard’s statistics about player distribution (yes, I know that player distribution isn’t the same as the distribution of the number of games per day, but they’re close) we get about: 6,400 games in Bronze, 16,800 in silver, 25,600 in gold, 20,000 in platinum, 8,000 in diamond, 2,400 in masters, and 800 in gm every day.

Now back to the central limit theorem. Say we are trying to understand Mercy’s win rates in all ranks. Mercy’s pick rate in all ranks is currently 7.32% so she appears in 7.32% * 6 * 80,000 = 35,136 games. The central limit theorem tells us that we have about a 66% confidence that her average is within one standard deviation of the actual average. This means that we have a 66% level of confidence that the average in overbuff is at most 0.26% away from the actual average. It also gives about a 90% confidence that the average is at most 0.52% away. That’s about the level of trust we should have for a daily win rate in all ranks in Overbuff.

In diamond for instance with a pick rate of about 4%, we get a 66% chance that her daily averages are within 1.14% of the correct average and a 90% confidence that they’re within 2.28%. That’s not amazing. But, we can improve it by quite a bit. The first thing we can do is to limit the direction - we have a 84% confidence that the correct average is not MORE than 1.14% above the overbuff average for that day (in this case I don’t really care about it being less) and a 98% chance that the correct average is not more than 2.28% from the Overwatch average.

Better, but still not great. So, what do we do? We take more games. We calculate the average win rate over a whole week (if you want to be accurate, you should weight each day by the pick rate). This ends up being 7 times as many games so the accuracy improves. When you do this, you get a 84% confidence that the actual average is not more than about 0.43% from the Overbuff weekly average in diamond, and a 98% confidence that it’s not more than 0.86% more than it. And if this accuracy isn’t enough, you can average over a whole month. This will give you an 84% confidence that you’re not more than 0.2% too low, and a 98% confidence that you’re not more than 0.4% too low. Not bad. In all ranks, the monthly average has about a 84% chance of not being more than 0.048% too low, and a 98% of not being more than 0.096% too low.

In Masters and gm, you really shouldn’t be paying attention at all to daily numbers - only to weekly and monthly averages. Especially for characters with low pick rates. Mercy’s pick rate in gm is about 2%. That means it only measures about 96 Mercy games a day. That means that you only have a 66% confidence to be within 5.1% of the correct number (or a 84% chance to not be more than 5.1% too low). That’s really bad. However, if you average over a week then you are 84% confidence you’re not more than 1.9% too low, and if you average over a month then you’re 84% confident you’re not more than 0.93% too low. Similarly, in Masters if you average over a month you’re 84% confident that you’re not more than 0.53% too low. Note: you have to be careful when averaging over a month. The days with higher pick rates should count more than the days with lower pick rates because they represent more games. And if this level of confidence isn’t enough? Wait another month and average. This will divide your 84% and 98% confidence numbers by about 2.

The numbers given here are just guidelines. They’re based off of estimates and approximations. But the math behind them is very real. They’re good estimates and guidelines. There are other places to go from here that can increase the confidence levels even further. For instance, if the character you’re interested in spends more day below a certain number than above it, that increases your confidence that the number is low. Or, If the character you’re interested in spends most of their days very close to a certain number, that increases the likelihood that this number is close to the correct average.

I hope this helps people interpret data in future discussions (and not only Mercy based ones).

[GTIX-21225]

i wish they knew what they were doing.

they knew that the problem was rez on E but didn’t do anything with it.
instead they go for the easy route that made her underpowered and not worth playing at all

[Hollypop-21276]

And I wouldn’t be surprised if we find some common ground in that aspect, but they seem to be concerned about addressing other areas of her kit (cough, rez…) so I don’t blame them for trying other ideas first since they seem so hesitant.

She’s such a polarising hero too, I can’t blame Blizzard for being to cautious when the ‘hot topic’ nature of balance receives such extreme levels of responses.

They didn’t make it clear that they meant it was successful for every support so it leaves us wondering are they thinking of changing mercy, are they saying she’s fine but being really vague about it

From my perspective, I think they’re playing the waiting game. Her changes have only lasted a month, and it takes time to analyse data and see more reliable trends unfold. Heck, Torb and other heroes are being changed so it’s possible they want to see how the supports (Mercy included) work with those changes involved on the PTR and on live.

[TheMadHatter-11445]

The support class is more diverse than it’s been in at least six months, so yes the support class is more balanced. By your own definition, the changes were a success.

[Arceus-2238]

I have this weird feeling that they’re preparing for mass resses return since these most recent changes all make mass ress less unfun to play against. That is after all one of the reasons why she was reworked.

[GTIX-21225]

i would be surprised if they did that.

[Arceus-2238]

Yeah, no. More diverse is not more balanced. Again, 3 heroes are left in the dust. It’s basically the same as before. Before it was Mercy, Zen and Moira. Now it’s Ana, Zen, and Lúcio.

Nothing has changed in terms of balance. The only think I’d consider changed in terms of balance is the severity of how unbalanced Mercy was.

The changes were a flop. All they did was swap the roles for each support.

Mercy and Ana have been swapped.

Zen has stayed the same.

Lúcio and Moira have been swapped.

[InTheDesert-1983]

Honestly i really don’t want mass rez back😂 I like bringing more value mid fight

[Arceus-2238]

Well, we won’t be getting that with current res any time soon. Current res is only ever used before or after a fight because of its limitations.

[InTheDesert-1983]

Agreed, I’d like to see a rez replacement but idk

[Titanium-1686]

and how many have public profiles? maybe more dps have public than tanks or healers?

This could be a valid argument, except we’ve seen Mercy at a 14.5% pickrate after private profiles were introduced. The maximum possible pickrate a hero can have is 16.67%. Mercy’s pickrate didn’t drop once private profiles were introduced and set as default; it spiked by 3%.

Moreover, looking at those statistics, the top six slots in pickrate are occupied by the following classes:

1. Tank
2. Support
3. Support
4. Tank
5. Support
6. Tank

Which doesn’t really point to more support/tank players with private profiles.

[Justin-17714]

This could be a valid argument, except we’ve seen Mercy at a 14.5% pickrate after private profiles were introduced.

you realize this does not support you at all? this could just be because it cant track the rest of the mercy players. moreover a rising pickrate goes against your entire argument that “she is picked less and bad”

[Titanium-1686]

you realize this does not support you at all? this could just be because it cant track the rest of the mercy players.

If it can’t track the rest of the Mercy players, that’s fewer data points from mercy players, and thus, her pickrate would appear lower than it actually is.

But…

Mercy’s pickrate didn’t drop once private profiles were introduced and set as default; it spiked by 3%.

Which means that if anything, more non-Mercy players left their profiles private than Mercy players.

It supports my argument

moreover a rising pickrate goes against your entire argument that “she is picked less and bad”

This spike was prior to her latest nerf.

[OooShiny-11855]

Someones a little angry at my post. I never mentioned streamers in my post, those are your assumptions.

[Evan-13804]

Youre naive if you think that they take anything you say into consideration.