Mercy is being ignored?

Justin-17714 · September 22, 2018, 5:17pm

they don’t. her pickrate has stayed the same, her winrate has gone down. this means people are playing them in every situation as before like a must pick, rather than playing her in her niche. You don’t understand statistics if you really believe what you said.

Dastil-2946 · September 22, 2018, 5:18pm

She is not in a bad spot in terms of viable really. However she is in my opinion more boring while being on the same spot pre rework, which isn’t good.

Arceus-2238 · September 22, 2018, 5:21pm

The reasons don’t state that “Hide n Res” was a good strategy and again, it literally would’ve died on its own had the SR system been fixed earlier. As for the pros, yeah, I could care less about what they say. They don’t even say why it’s a good strategy most of to the time. Just that, “ITs baD”.

Absolutely, definitely not LOL. First clip, you need to find me a time stamp. Second clip, it’s literally xQc complaining about a boosted Mercy player. Funny really, because you got even more boosted by playing reworked Mercy.

This isn’t even explicitly stated in any link you provided and is basically false since the subject of these three videos isn’t “Hide n Res”. If you want to prove your point better, put time stamps on the videos.

THEN DONT CLAIM SOMETHING THAT YOU CANT PROVE?? Hello? This isn’t rocket science.

According to the last video you linked, apparently because Res mechanic doesn’t belong in a FPS Game. Yeah, these guys don’t have a problem with “Hide n Res”. Just that Resurrect and to an extent, Mercy shouldn’t exist in their game.

Titanium-1686 · September 22, 2018, 5:22pm

What rock are you living under?

Her pickrate has literally dropped below half of where it was prior to the rework, and she was D-tier at that time.

Justin-17714 · September 22, 2018, 5:30pm

and where is your proof of that? if you say ovebuff youre done. overbuff is unreliable at best, with the private profiles.

Arceus-2238 · September 22, 2018, 5:37pm

No it’s not.

On Mercy and data

This post was conceived as a companion piece for evaluating my previous post (A month of Mercy nerf in numbers). Other than that, it’s not really going to be about Mercy (though I use her to work out examples). It’s about everyone’s other favorite subject - math. More specifically, I want to address the question: how much trust should we place in Overbuff statistics?

I’ve seen a lot of people claim that Overbuff’s numbers are meaningless due to private profiles (or due to the fact that even before private profiles Overbuff only received statistics from a subset of the player base). This claim is false. You absolutely can make very good claims about a large data set by sampling a large piece of it at random.

Here’s an example. A very big bag is full of blue tokens and red tokens. The bag is shaken thoroughly to mix up the tokens. Then you spill 100,000 token on the floor. Of these, 60,000 are blue and 40,000 are red. The remaining tokens stay in the bag (in the Overwatch context, these are the “private” tokens). What percent of the tokens in the bag are blue?

You can’t know for sure without looking at every single token. What you can do is make claims, and then give probabilities that they’re true. The bag should contain about 60% blue tokens and 40% red tokens. If you claim that the bag has exactly 60% blue tokens, then you are probably wrong. If you claim that the bag has somewhere between 59%-61% blue tokens then you are almost surely right (with a probability that can be calculated and shown to be very very close to 100%). The more tokens you draw from the bag, the closer the percentage should be to the actual percentage in the whole bag. In math, we call this the law of large numbers.

You end up replacing a certain claim (exactly 60%) with a more nuanced one, together with a measure of how sure you are about it. This shouldn’t worry you. Your entire life is lived like this. For instance, your brain processes information from your eyes at a certain frame rate, and your eyes themselves have many missing pixels (so there are “private” pixels and “private” moments in time). Everything you see at all times is based off of probability. That’s the way almost all of human experience works.

The situation in Overwatch is almost identical, except we’re not sampling entirely at random. For instance, we’re only sampling people with public profiles. The good news is that there’s excellent reason (supported by data analysis) to believe that private profiles aren’t correlated with, say, skill at playing Roadhog. So the conclusions still hold.

So, now on to the main question that I wanted to discuss. How much should you trust Overbuff data? Like in the bag of tokens example, it really depends on how many games Overbuff records. If you drew 2 tokens out of the bag, your conclusions would be off. If you drew a trillion, they would be really good. So, how many games does Overbuff record?

We can’t know exactly without someone from Overbuff chiming in, but we can get a very good estimate. The thing that comes into play here, and will be important for the discussion later on, is a beautiful theorem called the central limit theorem. I won’t state the theorem itself, since it’s a bit too technical for a forum post, but it deals exactly with the kind of situation we have here - repeating a random event over and over again and trying to understand its averages. The central limit theorem is the source of the bell curves that you might have seen pop up in all kinds of aspects of your life.

Suppose you record N games in which Mercy was played (I told you there would be a little bit of Mercy here). Suppose that Mercy is balanced such that she should win about b% of the time (and b% is pretty close to 50% so the standard deviation for one Mercy game is close to 1/2), but that in those games she won about a%. How far apart can we reasonably expect a% and b% to be? One thing that the central limit theorem (and its companion, the Berry-Esseen theorem) says is that you should expect about 66% percent chance for a% and b% to be within about 100/2 sqrt(N)% of each other (i.e within one standard deviation).

So I looked at the time period between late June (when private profiles were introduced) and early August (when the support balance patch came in) and found several characters whose pick rates were stable (Mercy was one of them). For each day, I found out how far the win rates for that day were from the average win rate during that time period. Then I found the percentage that 66% of the results were smaller than and solved for N.

Now, this is only an estimation but the upshot of it is that Overbuff
records around 80,000 games a day. Using Blizzard’s statistics about player distribution (yes, I know that player distribution isn’t the same as the distribution of the number of games per day, but they’re close) we get about: 6,400 games in Bronze, 16,800 in silver, 25,600 in gold, 20,000 in platinum, 8,000 in diamond, 2,400 in masters, and 800 in gm every day.

Now back to the central limit theorem. Say we are trying to understand Mercy’s win rates in all ranks. Mercy’s pick rate in all ranks is currently 7.32% so she appears in 7.32% * 6 * 80,000 = 35,136 games. The central limit theorem tells us that we have about a 66% confidence that her average is within one standard deviation of the actual average. This means that we have a 66% level of confidence that the average in overbuff is at most 0.26% away from the actual average. It also gives about a 90% confidence that the average is at most 0.52% away. That’s about the level of trust we should have for a daily win rate in all ranks in Overbuff.

In diamond for instance with a pick rate of about 4%, we get a 66% chance that her daily averages are within 1.14% of the correct average and a 90% confidence that they’re within 2.28%. That’s not amazing. But, we can improve it by quite a bit. The first thing we can do is to limit the direction - we have a 84% confidence that the correct average is not MORE than 1.14% above the overbuff average for that day (in this case I don’t really care about it being less) and a 98% chance that the correct average is not more than 2.28% from the Overwatch average.

Better, but still not great. So, what do we do? We take more games. We calculate the average win rate over a whole week (if you want to be accurate, you should weight each day by the pick rate). This ends up being 7 times as many games so the accuracy improves. When you do this, you get a 84% confidence that the actual average is not more than about 0.43% from the Overbuff weekly average in diamond, and a 98% confidence that it’s not more than 0.86% more than it. And if this accuracy isn’t enough, you can average over a whole month. This will give you an 84% confidence that you’re not more than 0.2% too low, and a 98% confidence that you’re not more than 0.4% too low. Not bad. In all ranks, the monthly average has about a 84% chance of not being more than 0.048% too low, and a 98% of not being more than 0.096% too low.

In Masters and gm, you really shouldn’t be paying attention at all to daily numbers - only to weekly and monthly averages. Especially for characters with low pick rates. Mercy’s pick rate in gm is about 2%. That means it only measures about 96 Mercy games a day. That means that you only have a 66% confidence to be within 5.1% of the correct number (or a 84% chance to not be more than 5.1% too low). That’s really bad. However, if you average over a week then you are 84% confidence you’re not more than 1.9% too low, and if you average over a month then you’re 84% confident you’re not more than 0.93% too low. Similarly, in Masters if you average over a month you’re 84% confident that you’re not more than 0.53% too low. Note: you have to be careful when averaging over a month. The days with higher pick rates should count more than the days with lower pick rates because they represent more games. And if this level of confidence isn’t enough? Wait another month and average. This will divide your 84% and 98% confidence numbers by about 2.

The numbers given here are just guidelines. They’re based off of estimates and approximations. But the math behind them is very real. They’re good estimates and guidelines. There are other places to go from here that can increase the confidence levels even further. For instance, if the character you’re interested in spends more day below a certain number than above it, that increases your confidence that the number is low. Or, If the character you’re interested in spends most of their days very close to a certain number, that increases the likelihood that this number is close to the correct average.

I hope this helps people interpret data in future discussions (and not only Mercy based ones).

Note: Special thanks to Helel for pointing out some calculational errors. The numbers are now corrected.

Titanium-1686 · September 22, 2018, 5:45pm

We aren’t talking about when Mercy first got her invulnerability buff, are we?

We’re talking about season 5 Mercy.

“The statistics don’t agree with me, therefore they are wrong”.

Hilarious.

Justin-17714 · September 22, 2018, 5:48pm

everything in this persons claims is wrong. I literally worked analyzing data for a long time. You need to know what base of the population you are taking from otherwise the data is irrelevant.

Justin-17714 · September 22, 2018, 5:58pm

the fact that she was a must pick for 8 seasons because of it?

no. you not understanding, and me not wanting to explain it to a person who lacks the intelligence to, is not grounds to disregard. That’s like saying “you wont explain to me how water exist so it doesn’t”

Justin-17714 · September 22, 2018, 5:58pm

look up 2 comments.1

Otamega-2461 · September 22, 2018, 5:59pm

More i see Overlord, more i think Valkyrie should be like it.

Arceus-2238 · September 22, 2018, 6:03pm

Mass Ress Mercy was not a must pick for 8 seasons. I’m not talking about reworked Mercy.

Horrible comparison. I’ve watched all 3 videos and literally can’t find anything even remotely talk about “Hide n Res”. Since you’re the one who linked the video, I expect you to give me time stamps on where they even talk about “Hide n Res”. I mean, you literally linked a 25 minute video. If you refuse to do so and just say that I’m not smart enough, that’s you refusing to explain your evidence. Therefore, I can disregard it since I can’t find anything that remotely supports your argument in said videos. I merely gave you a chance to make your evidence look better. You refused. That’s on you.

Titanium-1686 · September 22, 2018, 6:03pm

We have a sample of the playerbase.

A modifier is introduced to the playerbase, reducing the data available slightly, but doing so randomly.

The sample is still a sample of the playerbase. Smaller perhaps, but reducing the size of the sample by the margin for which private profiles did in contrast to the existing sample size tells us that the change is too insignificant for a random loss of data to be meaningful.

QuizzyBunny-2512 · September 22, 2018, 6:12pm

The private profiles were implemented well before the last Mercy nerf, didn’t stop people from using them to complain about Mercy anyway. It only became “unusable” when her pick-rates and win-rates started dropping like a stone.

Skink-11839 · September 22, 2018, 6:21pm

what we can do is keep talking about it till they make changes. We are the ones giving them money for the game, not the other way around.

Justin-17714 · September 22, 2018, 6:21pm

you do not have a definitive answer to how much of the playerbase overbuff shows. therefor the data is unreliable at best.

Loba-11371 · September 22, 2018, 6:23pm

Just let them focus on Torb and getting him in a spot they’ll be comfortable with and then we will see if they do anything with Mercy =)

Leewo-1537 · September 22, 2018, 6:28pm

She was not. Are you sure you played this game back then? Or are you sure you follow the pro scene? Because anyone who followed the pro scene back then knew that Mercy was not a must pick in pro play. She was used with Pharah, that’s it. Zen/Lucio were the go to picks for dive and Lucio/Ana for tanks or even dive sometimes.
Just watch Apex S4. It was after Mercy got her invul buff, it was when she had mass res, it was when dive came into the meta. And yet she is not a must pick. Please check your facts before saying things.

What lead to her rework wasn’t that she was a “must pick” it was ladder players complaining about how they don’t like mass ress, and that includes pro players because pro players on ladder =/= pro players in pro scene. Seagull is the best example of what I am trying to say, he was in the pro scene he knew mercy isn’t OP in pro play but he didn’t like playing against her on ladder. He was complaining about mass res on ladder not pro play.

Although a thing to note, on ladder she wasn’t a must pick either, if you picked lucio/ana or zen/lucio no one immediately thought you were throwing by not picking mercy which btw is what happened when Ana was meta in S3, what happened with mercy with her rework, and what is soon going to happen with Ana again.

InTheDesert-1983 · September 22, 2018, 6:56pm

Instead of trying to claim that people can’t say mercy’s being ignored since she’s been nerfed or changed 18373728 times, why don’t you just say what you really think which is, “it annoys me that mercy got so many changes before but now people are claiming she’s being ignored because the devs didn’t mention her in this patch.” Like, we all know that you understand what we mean when we say mercy’s being ignored so stop pretending you don’t and hiding behind logical arguments to try to make us look dumb and hypocritical, if you think we’re dumb and hypocritical just say it flat out lol

Desu-1820 · September 22, 2018, 7:09pm

Just too bad there’s so few supports to over twice the amount of DPS. There isn’t a lot of choice at all and it makes it unfun if you want to play support, you’ll have 1 or 2 viable heroes per patch, which is a couple not a few. And you get to have fun with your skill being invalidated as they’re nerfed to keep the meta fresh.