Are nerfs and buffs completely off base?

I doubt that any of you are capable of actually understanding this, but I try anyway.

The number of games recorded is NOT directly representative of the play rate of a deck.

For example, let’s imagine a very simple meta with 4 decks. The actual play rate of the decks is 40% A, 30% B, 20% C and 10% D. However, there is a smaller subset of players who have the deck tracker software installed, that records the games. Let’s say that the deck preferences of the tracker software players is 35% A, 35% B, 25% C and 5% D. For every 200 games played, you’re going to record, on average, games according to the chart below.

↓tracker opponent→ A B C D
A 28 21 14 7
B 28 21 14 7
C 20 15 10 5
D 4 3 2 1

Then for the matchup winrates, they’re going to add the games where tracker A played opponent B, to the games where tracker B played opponent A. So in the matchup winrate chart published by VS, you’re going to see something like this for games played:

A B C D
A 56 49 34 11
B 49 42 29 10
C 34 29 20 7
D 11 10 7 2
total 150 130 90 30
% 37.5% 32.5% 22.5% 7.5%

If you actually ever bother to count the games in a VS matchup winrate table, you’ll notice that, like this chart, the overall sum is about double the total number of games recorded. Slightly less, because not every deck fits the archetype definitions. This is because most matchups are presented twice, and mirror matches are doubled (always an even number).

So when you take a deck’s games played as a percentage of total games played, you are not getting the popularity of the deck. You are getting the average of the actual popularity, and the tracker player popularity. For D in this example, 7.5% is the average of 10% and 5%.

In VS Report #302, Handbuff Paladin was in a D type situation. The deck was between 4% and 5% popularity with the overall playerbase, and BELOW those numbers in popularity for players who had the tracker software installed — for example, in Legend overall, Handbuff Paladin had 4.09% overall popularity but only 2.91% tracker player popularity, therefore coming in at 3.5% of overall games played within the sample.

The tracker side of data is marred by selection bias. Best practice is to only look at opponent side data for overall deck popularity. You don’t accomplish this by counting games played. Games played is essentially a 50/50 mix of good opponent side data and contaminated tracker side data.

1 Like