AI could help somewhat. AI is a misnomer – it is neither artificial, nor intelligent. It is purely a matter of analyzing existing data, identifying patterns and trends, and projecting them into the future. In the old days it was called “predictive analysis.” The only thing that has changed in this field in the past two decades is computing and storage power – the ability to collect and analyze a larger volume of data.
Email spam filters are AI. They analyze millions of known spam emails, and based on that data they can detect new spam messages with a high degree of accuracy.
The whole house of AI cards collapses under two conditions: bad data, and new data. Spam emails still get by now and then when they don’t match enough of the old patterns. Ironically a spammer could use AI to generate unique messages from various servers, and stand a good chance of defeating most of the filters.
Back to Overwatch. Blizzard could identify a reasonable number of matches where both teams, one team, or even one player had a perfect game by reasonable human standards (no inhuman stats like 100% crit accuracy). Not just stats, but also map pathing, group composition, ult usage, team play, countering, and resource management. The best way to do this would be to take OWL replays from the team that wins the championship each year, and eliminate low-performing outliers. The AI analysis of those replays will represent the best of the best – people who should be in the top 50. Other playoff-caliber OWL team replays could fill out the next tier, and so on for the lower-ranked OWL teams, then Contenders. Then for the bottom 500, Blizzard could easily find replays of known-new players, or create a private focus-group-like project to find 100 completely new-to-FPS players, then use 20 of their replays to train the “bad player” data model.
From that, AI could identify players that perform more toward the top or the bottom of the ability spectrum. Here’s what’s crazy: Blizzard probably already does / did exactly what I just described. They certainly have patents for it, and it’s not difficult to implement. But like I said: this all is sabotaged by bad data and new data. You can’t make “balance changes” without rebuilding all the AI models, which is very expensive, so I doubt it’s been done in a long time. You also can’t use cheaters or unqualified new accounts as reference points – new accounts are not new players, and cheaters are not good players (and Blizzard is one of the worst game developers for cheat detection). The whole system is also thrown off by private groups (stacks of any size), which will always outperform each group member’s individual solo-queue stats (aka “boosting”).
