I don’t think there is a reasonable justification for excluding the qualifiers. If you did exclude the qualifiers, you’d have to exclude most of the entire Dreamhack tournaments in order to be logically consistent. Korean qualifiers have a higher skill representation than the finals of Dreamhack regional events. By the time you do that, your sample is too small to say anything about balance given how wildly win-rates can vary in small samples. If balance can be measured, it can only be measured in a large sample, which requires the inclusion of Dreamhack and thus the qualifiers to be logically consistent.
Fair enough. I will just agree to disagree on this one. Sorry for my confusion on your earlier post that already posted data with the qualifiers removed. I will admit, you are a smart dude.
Also this is using the forum’s metric for balance. I wouldn’t call it definitive because it doesn’t account for skill representation. So if there is a 50/50 win-rate between Protoss and Terran, is it because they have equal skill representation and the game is balanced; or is it because Protoss is favored in balance by +5% but terran has +5% better skill representation? You can’t say with a win-rate because it’s incomplete information. There are better systems to measure balance, but according to the forum whiners’ logic the game is absolutely balanced and the whiners need to put a cork in it.
The challenge of measuring true balance is being able to objectively measure a player’s skill level, and if you did you’d see protoss far overstep their skill level. Protoss players do far better in asymmetrical matchups than in their symmetrical one. Symmetrical matchups are nothing but skill. Asymmetrical matchups are a product of skill and balance. If balance is equal, a player with X performance in their symmetrical matchup should get X performance in their asymmetrical matchups, but it’s not true for protoss. Alphastar did literally the same exact thing. For a given APM alphastar did best with Protoss.
The Grandmaster leagues find the same:
https://i.imgur.com/tUVdzPE.png
Lmao. 43% protoss.
I don’t know what tenet is.
Here is the actual quote :
Before you review these numbers, we'd like to prevent some common misconceptions. First, these numbers shift fairly rapidly as newly discovered strategies spread through the community and they're changing all the time. Also, due to the way the numbers shake out, we expect a variance of +/- 5% in these results; win/loss ratios that are +/- 5% suggest balance between those races. So, if a win/loss ratio is approximately 55%:45%, then that's still within acceptable boundaries. By contrast, win/loss ratios exceeding 60%:40% could indicate that a small imbalance might exist and merit further investigation.
You are incredibly wrong, and trias is actually right (for once), blizzard specifically pointed out that the numbers shift fairly rapidly due to newly discovered strategies, this clearly means that we are looking at winrates day to day and not over time.
If blizzard was talking about long term winrate then we wouldn’t expect dominance due to a new strategy to actually change the winrate in a measurable way.
Don’t worry, this is a very easy mistake to do if you don’t understand statistics.
1 Like
Are you seriously arguing that a sample of 2 games over a 50 year period meets the criteria to define balance through a win-rate?
Reality check: the time spans aren’t very relevant - it’s the sample sizes that matter. Blizzard’s sample size was much larger than the sample we’re talking about so their logic applies. If a +/- 5% deviation is OK in a larger sample, it’s OK in a smaller one. In fact, using a 5% deviation in a smaller sample is far too generous, since smaller samples have variance added through the sampling process, and a larger deviation therefore would be acceptable.
Are you seriously arguing that the mean of a sample doesn’t shift simply because the time period is large? Do you not understand how the sampling process works? Have you never heard of the sampling mean, which can vary from the population mean? If we only take 2 samples of a random city’s temperature each year, the average is going to vary WILDLY from year to year and city to city, and the fact that the data goes back 100 years is not relevant.
Thats the point.
CLEARLY because they have such a large margin of +/- 5% then we know that the sample size is relatively small, which means that they are not keeping a long term average and sampling from a specific timeframe.
If your having trouble understanding this then i’d suggest you look up how we form confidence intervals and how the sample size affects this.
I see you once again fail to understand the central limit theorem.
As the sample size gets larger and larger, then the sampling distribution’s variance shrink. In simple words, if you take a sample of size 10,000 and then a different sample of size 10,000 then you expect the means to be very close together. In comparison if you took a samples of 10 then the means would vary widely from sample to sample.
if your having trouble understanding this then you can see the 2nd graphics on this website https://seeing-statistics.netlify.app/issue1/.
Also this place is a goldmine for bad statistics memes.
1 Like
Their sample is roughly 250k. This sample is 17k. Their sample is literally 15x larger. The fact that this sample is so small and spread over a large time period is a BIGGER problem for the variance of the sample, given that over large time periods events like balance changes and map pool changes and meta changes are more likely to have occurred, meaning higher variance aka a larger tolerance than +/- 5%.
That’s exactly what I just said. Blizzard has a 5% tolerance for the mean to shift over a sample of roughly 15x larger than the sample we are dealing with. You just admitted that a smaller series of samples will have higher variance, which is exactly what I just said:
Try to keep up, kiddo.
source for this?
Do you not realize that you are arguing against your own pervious statements?
1 Like
You CLEARLY do not understand my arguments IN THE SLIGHTEST.
Ahh yes, pardon me but I have trouble understanding statements of yours such as “the sampling distribution is the distribution of the sample”. I’m sure you know why now.
And again
Source? for this please? Because I can assure you that theres no way they have a tolerance for +/- 5% if their sample size is this big for high league games.
1 Like
https://stats.libretexts.org/Bookshelves/Introductory_Statistics/Book%3A_Introductory_Statistics_(Shafer_and_Zhang)/06%3A_Sampling_Distributions/6.02%3A_The_Sampling_Distribution_of_the_Sample_Mean
Titled: “The Sampling Distribution of the Sample Mean”
What you claim is confusing: “the sampling distribution is the distribution of the (sample) mean”
Nothing I said was confusing. It’s basic statistics jargon.
Sorry it was a typo, i was making a reference to your previous comment about the sampling distribution being the distribution of the Sample, do you want me to go quote it?
1 Like
and once again for the love of god, can you please provide me a source for the 250k sample size for their estimate of the win/loss ratio? or do you just want to admit that it was made up?
1 Like
Feel free to. I’ll be glad to show how you misread back then the same as you just misread in this thread. You do this literally every time we have a conversation. You ignore something I say. Then you say I made a mistake. The reality is that you didn’t understand what I said and assumed I made a mistake. And then I wreck you in the debate, like what just happened.
https://www.rankedftw.com/stats/population/1v1/#v=2&r=-2&sy=c&sx=a
If blizzard is basing their balance decision on a single day’s worth of ladder games, then there are approximately 250k games in their sample. Note that this is a drastic under-estimation, since the game’s popularity has gone down substantially since when they made the post we are discussing.
https://us.forums.blizzard.com/en/sc2/t/visualizing-balance-in-the-simplest-way-possible/13308/123?u=cheezecake-1895
Do you know what the margin of error for a proportion with a sample of size 250,000 is?
Let me rephrase that so its easier for you to understand.
Do you know what the standard deviation for a sampling distribution of a proportion with a sample of size 250,000 is? (hint: its a lot smaller than the 5% that blizzard gave that they were willing to tolerate)
1 Like
I love this. It’s a perfect example of how you don’t bother to read. Let’s quote it here for easy reference:
The sampling distribution is the distribution of a given sample. That distribution can vary from the population distribution which is what the sample intends to measure
The sampling distribution is the distribution of a sample of samples. This statement is correct, but because it lacks context you thought I was referring to a singular sample of a random distribution, when I was specifically referring to a sample of a sampling distribution which is crystal clear in the previous posts:
The CLT states that as the sample size goes to infinity the sample converges on normality. You can’t get an infinite sample so a better question is what size of sample is needed for a high confidence of being normal, or is it possible to test if a sample is normal with a high degree of confidence?