I would say no, since the sample you pull is highly biased.
Very enfranchised player who care for statistics is not your standard Overwatch player.
The problem with its reliability is that accounts are private by default and certain players, onetricks smurfs etc will keep it locked while very few players make it public, mostly who are just flexing so they won’t get called out for it.
1 Like
Statistics states that a sample size of 10% is enough to accurately predict the population. That’s not me, that’s math speaking.
So long as overbuff has 10% of the player base, it is accurate.
1 Like
Trust me, it is. My friend was the #9 genji at one point in the beginning of a season. (He is not a very good genji)
I would agree with your assessment. The sample size is well beyond what it would need to be to be considered statistically valid and there is no reason to believe that it is biased or skewed.
As far as i know the hero rank only applys to your rank. When you are the number one ana, that means for me fir example the best in diamond, in europe of the once who signed to overbuff. So basically it means you do a good job but not nessesarly. i have been on of the best mercys for example for a long time because i killed on average 12, of course that means not much. I remeber killing a lot of pharas for in stance because no one did in gold at least.
I wrote a post about this a while back with some back of the envelope calculations:
The basic upshot of it is - it depends on what you’re looking at. If you’re looking at statistics that are not strongly correlated with private profiles (such as relative pick rates between heroes) then you can be confident in its reliability as long as you use it carefully.
For instance, if you want to look at GM data, you need to look at averages over several weeks before you have reasonably reliable numbers. If you’re looking at all ranks, you don’t need averages over large time periods. The post I linked has some numerical guidelines for these time periods.
On the other hand, if you’re looking at statistics that are probably correlated to public profiles (such as on fire percentage) then the reliability decreases.
Overbuff hero stats are a good way of seeing an overview of what is and isn’t popular, or a heroes overall performance.
The issue is not that many people actually use overbuff, which would include all of the players with a private profile.
I think its still pretty good, but you have to consider more things when looking at the stats. Like say torb players, maybe most of them have a private profile, and his winrate being like 63% is because of the really good torbs that keep their profiles public.
Personally, I dont find Overbuff to be accurate at all, not even close
But why? I agree with those saying that it needs to be used in a broad sense, minute details will be off for sure, but why do you say it’s not even close?
In my experience, the data found there is highly inaccurate and incomplete
I think it’s accurate for the general information. But the individual information isn’t as accurate, like your skill percentage and some stats are broken. For ana it always says my avg unscoped accuracy is 16%, it won’t change no matter what I do, even though my in game stats say it’s 47% (which still isn’t great but it’s a lot better than 16% lmao)
So if they only take the top 10% of profiles its accurate? Lol no
the data on overbuff comes from those players who look up there profile on overbuff and also have a public profile, thats an incredibly biased sample, it probably contains far more people that listen to youtubers about what the meta is than the general population for example, its probably got a lot more aim based heros than the general population to
Statistics states no such thing. It says the larger the sample size, the smaller the error. Statistics isn’t perfectly accurate until you have a 100% sample size, and isn’t perfectly inaccurate until you have a 0% sample size. (If you have a 100% sample size it’s really just ‘measuring’ so there’s no probability anymore.)
Statistics also is looking for a random sample of the population. Which overbuff doesn’t have. It has a sample that is 100% people with open profiles, and mostly people with enough interest to track their own stats, although people can also get added to overbuff because someone else looked them up.
That’s not random, so we shouldn’t take the data at face value. There are ways to control for non-random data but maybe not here. It’s likely that people tracking their own stats take the game more ‘seriously’ or are ‘more competitive’… but how do you control for that?
In any case, whether you could or not, Overbuff isn’t controlling for anything, they’re simply assembling the raw, biased data as it comes in. That means you should very much take its global results with a big pinch of salt.
For tracking your own personal statistics, of course, it’s nearly perfectly accurate. It should have a 100% sample size of your activity, after all. Where it places you in ‘percentile’ compared to the rest of the population though is just as questionable as any other global data.
1 Like
It’s more reliable than not, but only if you understand how to read the stats you’re looking at properly, meaning, that’s these stats are only comparable to players of roughly your own SR and are subject to changes depending on if you’re climbing or dropping.
Basically, if you have a 3/1 K/D on say Mcree in Bronze, that doesn’t mean you’ll have that same K/D if you were thrown into a Platinum game.
Overbuffs stats don’t take rank into account. You could be the #1 Genji but be in Bronze if your stats are good enough. That doesn’t mean that if that bronze Genji was thrown into a Gold game they’d have the same stats as they do when playing against bronze players.
It isn’t. It only mesures those who visit it I think and also a lot of people have their profile set on private
I don’t even registered in the overbuffed. Can’t find my stats.
The last official stats I recall was March in 2018.
Overbuff last time was generally comparable but, it was also wrong some of the time if the interval that Jeff posted on was similar to a week.
What I had written down from back then was that at the top end Overbuff overestimated Winston’s pickrate and underestimated Mercy’s pickrate. At the bottom it overestimated Zen’s pickrate and underestimated Ana’s pickrate. (Winston and Zen at that time may have had higher pickrates with try hard players that are more likely to look themselves up and Ana and Mercy may have appealed more to non try hard players (Ana was lousy at this point in time).
Overbuff scrapes users/gets names from people looking it up and as such is not complete and can have a biased sample. The addition of users being able to have private profiles likely makes this worse.
I personally assume that the trends are generally accurate, the winrate is within a point or so, and that the pickrate might be off by two or so.
You’ll find naysayers to everything Overbuff has to say because it can threaten their personal position on topics.