Sure dps mattered, but to say it wasn’t a test of tanking skill is just completely and totally incorrect.
You have an NPC to protect, you have to position yourself and the boss so you don’t get thrown off/NPC doesn’t get hit, you have stacking debuffs to manage, you have points of burst damage to counter, and you have a stacking tank buster mechanic to rotate CDs on.
What exactly else would you like to make this a “tank” challenge?
Fact of the matter is, tank DPS mattering has been a thing since MoP. Sure in Normal/Heroic it doesn’t matter, but in Mythic and upper level M+ it matters… So it only makes sense that you should have to maximize your dps in a challenge meant to push you to your limits.
I can guarantee you that if DPS didn’t matter at all in that challenge (and they had an NPC doing all the DPS for you), it’d be ranked the worst challenge out of the lot and considered by far the easiest of the challenges. Instead of last gasp smacking the boss with everything you got because the next tank buster is going to kill you… You’d have tanks getting threat and just running away…