The effect of skill, quantified. DR278

The gap between win rates of different decks at diamond and legend? That’s the gap you pointed out.

I haven’t looked at the underlying spreadsheet, but keeping things simple he’s looking at the value of the change in win rates at different parts of the ladder.

The entire point of the original post in this thread was the op trying to impress people with their statistical expertise… that they don’t actually have… but, please, go ahead and point the finger at the people calling it out as the bunk it is.

It’s not what you were talking about though.

Looking at a binary match up in the way the VS records games played only shows what happens when the recording player’s opponent is playing the given deck. But, as you noted, someone is losing every match that is recorded.

If you switch the narrative and record the player rather than the opponent, you see the exact opposite phenomena, which is what you posited.

It’s not explained, at all, by the skill test put forth in the op. Definitionally for the skill of one side of a match up to improve, the other side would have to show “less” skill if that was the true source of variance.

3 Likes

This is where you fall down, it’s personal for you. The OP is someone doing some analysis and posting their results. You turn it into some other thing in your head and let your ego take over.

We will have to agree to disagree.

What’s going on here, I think, is that you’re tunnel visioning so hard on Priest’s good matchups that you’ve coming to the polar opposite of the correct conclusion.

At Diamond 4-1, Control Priest has a 47.69% winrate. If we kept Diamond 4-1 matchup winrates but substituted in top 1000 Legend deck popularity, Control Priest would go to a 46.87% winrate. That’s a LOSS of 0.82%. Overall the top Legend meta is LESS favorable to Control Priest than the Diamond 4-1 meta.

The biggest thing that you’re missing is that Enrage Warrior and Mech Rogue are not even the biggest explosions in popularity at top Legend. Yes, Mech Rogue gains about 5% and Enrage Warrior gains about 8.5%. But Rainbow Mage gains more popularity than Mech Rogue and Enrage Warrior COMBINED, and Rainbow Mage is a bad matchup for Control Priest. It feels like you didn’t factor Control Priest’s bad matchups into your analysis at all.

Control Priest does well at top Legend DESPITE the meta turning against it because the matchups themselves improve with top Legend skill. The Rainbow Mage matchup is still bad, but it’s 0.5% less bad, and a 0.5% increase against a deck that’s 23% of the meta goes a long way. But the good matchups improve tremendously. Control Priest vs Mech Rogue is 56-44 for Priest at Diamond 4-1 and 60-40 at top Legend. Control Priest vs Enrage Warrior is 53-47 for Priest at Diamond 4-1 and 59-41 at top Legend. Those are huge gains.

So no, the 2% winrate IS NOT explained by the fact that it has a favorable matchup against “the 2 decks” that explode in popularity at that level. It’s explained by the matchup winrates themselves improving, in both the good matchups and to a lesser extent the bad matchups.

Also, in D4-1 Control Priest has a 47.69% winrate. After -0.82% due to meta shifts and +2.34% due to the effect of skill, it ends up at 49.21%. That’s going from bad to mediocre. I put it at the top of the list because it gains more from skill than any other deck, but I don’t mean to imply that it’s the “top dog of top Legend” or any other endorsement to actually play the deck. At least not if your primary concern is winrate.

No, that’s not the entire point. If I’m being honest it probably is a sub point, but it’s secondary at best. The main point, if I’m being totally honest, is to show empirically that you’re objectively wrong. You spread mistruths on this topic and people should not believe you. I don’t want anyone to believe you, or at least the you that you’ve been, regarding the subject of skill ever again. I have a powerful animosity of you that’s borderline empowering in a Sith kind of way, and I am trying to channel that energy in a constructive rather than a destructive manner.

But I would have published the numbers even if they hadn’t have gone my way. I can’t possibly know before I start, the calculations are too complex to see where they’re going before they’re mostly done. And if I can see that I’m wrong I’ll admit it.

Correct. This is pretty much lacking in this entire analysis.

It probably makes sense to talk about perfect play rather than “skill”. Perfect play between both sides should approach an equilibrium win rate when averaged over all possible starting hands and Mulligans.

Given enough compute resources, it should be possible to compute all possible games in this matchup and determine what is considered perfect play and what is not. As a player approaches perfect play we can then label them as skilled, and quantify that skill by measuring how often they misplay.

Obviously this is a bit idealized, there doesn’t exist enough compute in the world to even come close to compute this, so instead some form of stronger than human AI can be used to approximate perfect play (as is the case already in other games such as chess.)

That said, I think scrotie set out to prove a tautology, namely that better players, on average, will pilot decks better. I have no doubt that there exist many indicators in aggregate statistics that may actually be caused by this, but as anyone learns in reading their first statistics book, stats can never prove a causal relationship, only a correlation. At best you can find evidence of possible causality, sometimes weaker, sometimes stronger. I’d argue as it is, this analysis falls on the weaker side.

I mean yeah. Im usually not one to lurk around. But a correlation between winrate gain and experience with your deck is well known. Especially for decks that may have matchup specific power plays or vice aversa.

Now i wont argue playing decks gives anyone brain damage, or else we’d all be critically endangered lmao. But i mean there definitely are decks that plateau early.

Some decks like miracle rogue or 20 card a turn combo decks always tend to have something going on. A combo deck piloted by someone who only plays cards by random chance is akin to the world’s best sniper rifle assigned to a person who’s aim is so bad. A shotgun would be more “accurate” for them.

But then ofc some people craft by budget or just whatever they want to play.

I’d agree with retrospect a lot of sw pirate warrior probably was a plateau deck vs 7 mana Alterac miracle scabbs rogue. The one that ran 0 mana gnolls and could edwin2 + Mr smite a 10/10 + 6/5 to face quite regularly.

Pirate warrior hit it’s peak quickly. If you could play a pirate and control a random rng random pirate spawn. And random cannons. The deck mostly played itself. Discover rng seems like it can still show up slightly in decision making.

But the 2x2 dmg cannons that occasionally made or lost games on crucial pings definitely had little pilot benefit to speak of. It climbed to d5 on early lethals then it got taunt druided and countered by people who finally learned to play their decks.

7 mana alterac scabbs maestra rogue just didn’t get countered by much though. It took a medium time to learn. Playing edwin2 without a deck tracker sucked for mana counting/odds. But it could bounce, 0 mana gnoll, cloak, stealth. Charge 10/10+6/5. It had hand damage. Maybe a well played garrote rogue could screw it over but honestly my memory is too foggy.

I do remember top 100 standard players would routinely take every turn to count aggro, aoe. They would like count every potential play your deck could make and calculate the odds of each one.

They made aggro matches last 40 minutes long. I don’t know the exact margin but the tournament atendees (they gave 4 free packs to attend), often played 3-5 decks well at a mastery level with 1 ban and loved decks that always had a good matchup.

The winrate margin being +1-7% in their favor seems wildly believable to me. They couldn’t rng their way out of bad rng but they usually played each matchup well, never missed lethals, thought full and well about outs. And sometimes knew the most obscure tricks for the most niche situations.

The one thing I will generally say is that evidence pretty continuously points to arbitrary meta changes (edit:) NOT being the primary cause of deck performance differences between diamond (where we have lots and lots of data) and high legend (which is far more sparse, but not useless).

It’s really common to see decks that gain or lose win rates between those points on the ladder, but not have the changes align with what you’d expect the deck distribution to cause.

Skill and piloting choices is a very large umbrella of things, and we obviously can’t pinpoint which aspect of play is driving the disconnect, but we can pretty well say that it isn’t just “the meta” causing decks in high legend to perform differently, and that how these decks are piloted is a major factor.

Attributing it to higher skill is somewhat reasonable given the fact that the players up there have the best overall play records out of anyone in the region.

It’s frustrating how many times I repeat myself here.

I am NOT quantifying skill. There title of the thread is NOT “skill, quantified.” The title of the thread is “the effect of skill quantified.”

There is some skill difference that exists between the average Diamond 4-1 player, and the average top Legend player. I take this to be self-evident, but I’m not measuring that. I don’t even know what units I’d use. But whatever that skill difference is, it causes a measurable effect on winrates and we can factor out the difference in the two metagames. We can know how much winrate change that skill difference creates.

The entire point of the opening post is that what you are saying here is a myth. Arbitrary meta changes only account for somewhere between 31% and 40% of the deck performance differences. The majority, 60-69%, is due to matchup winrate changes.

I was reluctant to create a weighted average like this in the opening post because there’s no fair way to do it. The distance between Diamond 4-1 and top 1000 Legend is not of either, it is something between them. So a weighted average according to Diamond 4-1 popularity isn’t quite right and a weighted average according to top Legend popularity isn’t any better. But I calculated the average according to both weightings here, and I presume that the truth is somewhere in the middle.

The reason I made the opening post on the first place is that the data continuously points AWAY FROM arbitrary meta changes being the PRIMARY cause of deck performance differences between diamond and high legend. I’ve done this type of calculation once before and the weighted average was approximately 73-27.

1 Like

Because people exagerate a lot on the subject.

They keep discussing what the primary cause is because it distracts from the fact that the 2/3% they did lose by just not playing as good is enough to climb.

Sorry but the theocracy here is people putting everything at RNG fault.
And it’s a really big one.

1 Like

Yeah, I meant to have the word NOT in there… which makes the entire paragraph wrong -_-

Edited it so that I actually make the point I was trying to make.

1 Like

Wow, you guys spend a lot of time trying to justify the time and money you waste on playing a luck based digital card game.

1 Like

Lol I don’t think that’s quite right.

We (the forum regulars) are all probably crackpots, or quacks, if you will, that like discussing pseudo science and pseudo stats about a digital card game that most of us don’t like playing much anymore.

… Yup, that about sums it up.

1 Like

Do you know how in math classes, very often one or more people ask, “When are we ever going to need this?”

For me, that moment came in my Algebraic Structures class, when I had to prove that 0xA = 0 = Ax0. Or that 1xA = A = Ax1. I hated that class. None of the content matters.

That’s how this thread makes me feel. You don’t need to prove things that are axiomatic.

Was this a college/university level class? I’m genuinely interested, because typically you don’t care to go into that level of detail unless you’re on track for some kind of fairly proofy math program.

1 Like

Good discussion,

I wanted to take a moment and sincerely thank Scrotie for the thought, dedication, and hard work, that he put into compiling his thoughts, data, and results. I don’t know much about statistics and I imagine this was a lot of work. So, thanks for doing that.
I think one takeaway that folks are forgetting here, is that Scrots didn’t have to do this. He chose to do this for the benefit of the community. More than I’ve ever done that’s for sure.

4 Likes

Yes, it was required for a math degree. That class was the biggest waste of time in my entire academic career.

I bring it up because I feel like this thread is the bell curve meme incarnate.

2 Likes

That’s the underlying issue though, right? It doesn’t matter what you do mathematically - the math can be perfect - this data can’t be used to make inferences of this type, and that’s something you learn in stats class before you ever do a single calculation.

This idea that match ups are binary and the win rates regress towards the mean (this is evidenced by the decrease in variance around the 50% winrates for all decks at higher ranks) is a better indicator of opponent skill than anything in the op, but that’s a separate issue.

I don’t disagree that top 1k is more skilled players, but I vehemently disagree with the absolute sophistry of the op’s statistical analysis.

It’s really galling that they had the audacity to accuse someone of suffering from Dunning-Kruger in another thread while this one is out here for everyone to see.

I’m honestly glad I put them back on ignore so I don’t have to deal with this.

You’re not wrong.

You certainly don’t need to misuse statistics to give weight to it, either.

No, he chose to do it for selfagradizement. I’d wager he would admit as much if pressed.

Nothing about this is altruistic in any real sense.

1 Like

You can’t even describe it accurately. You don’t even know what the analysis is.

I do feel like it’s fair to note that for legend matchmaking. Mmr itself might balance even.

In addition top 1k legend, especially standard’s top 100-1k due to past tournamant entry prizes, ( i think like the top 1-10# of standard ladder could either play like 2-200 tournaments or run for top standard), could get entered into worlds.

Wild definitely had some extemely powerful decks and brewers but half the appeal was you used to find unseen power cards. I think these days most stones are trod though. But you can definitely see extremes sometimes.

Sometimes when a deck is autopilot it may not have much d5-legend potential since anyone can play it to max. Other times you see absurdities like a pilot with a 70-92% wr in a mirror match over 40++ games that just flat out statistically probably shouldn’t exist on paper.

But a lot of d5-1 players will often be good enough to climb and play their deck. Maybe it’s like chess where a skilled player will know the best play for 1-2 turns. A adept might know three. And a Hikaru level grandmaster will have calculated all your potential 20 moves in advance while premoving a guarenteed checkmate 14 turn away you haven’t seen yet. And then a Magnus will spend 2 hours staring at the pieces and making a Hikaru lose.

There can be a absurd level of differences and time investments but it is kinda true. Pilots in skill intensive decks can make absurd differences or even if discounted, long term trends for mirror winrate might be good too.

But theres definitely a balance between a game you do after work. And a game you professionally play AS work.

Magnus isn’t going to checkmate you if you play a game you have 30 queens and him 20 pawns. When even the greatest piloting can’t save a lost cause, sure they might lose that game and everyone can brick, even top 100.

Maybe it’s just that 60%-80% of the time they might be able to clobber most average joes.

Im honestly far from a defender. Just saying honestly that win margin and high legend having different winrates as players improve/master decks or pocket metas is known to a lot of people peeking at the high end.

A skill testing deck is a lot like a sniper rifle in fps. If you can’t aim, it’s the most useless weapon in bronze.

If you aim like apm jesus, someone may be the most skilled sniper in the game they know of. But if someone comes in biggerx instasnipes you and camps a lobby. Sometimes unchecked dominance is a idea people are more in love with the idea of than practice.

Chess is a prime example. The best player always wins 75-97%+ of the time by elo. Yet everyone thinks they would win out the gate and yet it’s often sometimes difficult to attract new players to it in real life.

Moat people play at 1 skill level unless they hold back. And if people’s experience starting a game is losing every single one, many people might just leave.

Rng is seen as a neccesary evil perhaps so maybe one there’s still a skill curve to hand them a 52-70%+ chance to win. But the other person gets a chance as well.

I think maybe it’s akin to surfing. You can’t control every wave, but if someone consistently falls into the ocean 60%+ of the time, while the other flies but occasionally gets tossed by the worst tsunami 10% of thr time. I don’t think it means sailing is skilless if there’s rng.

Just long term performance is often a more valuable metric for measuring if something is working or not. And maybe i’d argue if you have more pilot edge, you can afford to screw around more than if one who need every edge to survive.

But d5 vs legend is still a single pack, a single 1$ worth. If it takes one say 3-20 hrs to get. It might not technically the ‘best’ use of time. But i mean technically that’s like saying any hobby, free time, or recreation is a waste of time.

I enjoy freemium as a challenge but if you were paying 80-200$ for a deck to play it never and just buy more packs… “Whats the point of that?” is what my knee jerk reaction would say. But maybe for someone else they just play with a hundred 100$ toys one time. Or see it as visible consumption.

Lots of people will spend 1000$-10000$ on jewelry, not because they need it. But i bet you could find a lot of unhappy relationships if someone took out 300$ less on a pearl necklace to buy cards lmao.

Hearthstone is far from “technically rewarding” at like 10 cents of xp a hr so if you optimized the fun out of it. You’d close the game. Hook up blizzard to your bank account and wire money you’d never see doing overtime haha.

I just like collecting cards to like feeling “like a raid boss” part. Fremium is semi limited at the “afford the deck that can” part. XD.

If you leave me alone to my own devices i’d start drifting away from winrate to try and create greed lord decks with 26 attack, 80 armor and 20 mana and 3 otks and it’d be like some greedlord deck getting hate dms when it works. Fall over like a top heavy tumbler when it doesn’t lol.

Games are games. It’s fun to win, but if winning ain’t fun, who will stop ypur opponent from becoming a unchecked megazord? :joy::stuck_out_tongue: