When you are playing WC3, or even more traditional shooters like CS:GO each, players can be judged on generally the same criteria since all characters/races are very similar (this also allows W/L to carry more meaningful weight as ranking metric). In OW though, the heroes are very asymmetrical by design so how you judge Rein play vs how you judge Widow play vs how you judge Mercy play are three very different sets of criteria.
It’s sorta like judging NBA players vs judging NFL players. In the NBA stats like field goal %, free throw %, assists, blocked shots, etc., are, to varying degrees, applicable to all players regardless of position. In the NFL though QBs, RBs, receivers, offensive lineman, kickers, corner backs, etc., are all judged on very different criteria because, like in OW, each position has very different responsibilities.
Some things I think could improve the rankings/comp experience:
6-stacks should get their own queue in comp, though you’d probably have to differentiate between LFG 6 stacks (which could be 6 randos) and pre-made 6-stack groups.
Each role should get a different MMR (but that will probably necessitate role locks so someone can’t get match-made in a gold role and the switch to a bronze level role or a diamond level role during a match). Maybe you could switch roles in-game as long as the MMR difference wasn’t too big?
Having guilds could help foster team-focused play so players are less likely to chase individual stats to the detriment of the rest of their team. Maybe there is an XP bonus for playing with guild mates like there is for playing in a group?
BUT the problem with trying to get too specific and creating a lot of subdivisions during match making is you’ll run the risk of longer and longer queue times. Ex. I think Jeff said that in their role queue tests the queue time for DPS players could be as high as 20-30min. They have to weigh finding a ‘close enough’ match in a shorter amount of time verse finding a ‘perfect match’ in a much longer amount of time.
We are talking about how to rank individual players though, not how the playoff system works.
Outside of QB Rank I don’t think the NFL has any official formula to rank individual player performance. And even QBR is limited which is why other companies like ESPN and Football Outsiders have developed their own player ranking methodologies. A lot of it comes down to context which is why relying on software only is so hard. For example, if a pass is intercepted who gets the blame? Is it the QBs fault because he threw a bad pass, or is it the receivers fault for running a bad route or tipping a catchable ball into the air? But Blizz has no choice but to rely on software because it’s not realistic to have humans watch and evaluate every player in every comp game.
How pro scouts grade individual players when it comes time for the draft and for trades is a more apt comparison, IMO. Ex. Trent Dilfer has a Super Bowl ring but by most accounts he’s a ‘meh’ QB that was hard-carried by one of the best defenses in the history of the NFL (which is why he’s the only starting NFL QB to be released from their team after winning a Super Bowl). If you were building a team from scratch I’d rather have a guy like Dan Marino or Warren Moon as QB even though Dilfer has the ultimate symbol of NFL success, a Super Bowl ring, and they don’t.
Can OWL ranking be better? Of course. Do I blame Blizz for trying to create a more comprehensive, if currently flawed, MM? No.