Thanks for the well written post on your matchmaking experiences, I’ll try to explain some of what’s going on.
The system does in fact try to place equal sized groups on opposite teams whenever possible. Your report for game 2 where both 3 player groups were on the same team definitely seems like something that shouldn’t happen based on the rules we’ve setup, so I’ll look into it further.
We do need to do a better job of not placing players into “unwinnable” matches. When the matchmaker creates a match, it determines the % chance for each team to win based on the match it made. The VAST majority of matches are usually near to 50% (especially if you’re a player closer to median skill rating and you’re not in a group), but I’ve definitely seen logs of matches where that’s really not the case and my eyebrows raise.
The unfortunate truth is that there is not always a “perfect” match for you, especially at very high (and very low!) skill ratings where there’s fewer players of similar skill. Then you throw in the desire to match groups vs. groups, with everybody having low latency, and doing ALL of this as fast as possible even though it’s the 3AM offpeak… it can get tough. We’ve tried different tunings with regards to wait times, and the improvements were unfortunately modest as we increased the time to wait. Still, this is an area we’re always looking to improve and tune better.
Fortunately, when we do put you in a match that we know isn’t a 50/50, we adjust your SR gain or loss based on your calculated change of winning. So if you did get placed into a match with only a 20% chance to win and then you lose, you shouldn’t lose much SR.
For matchmaking groups, there’s actually two separate issues that we try to solve. The first issue is “How do we handle groups formed of players with different MMRs?”. With season 2 we prevented players of REALLY disparate Skill Rating from grouping, but there’s still some variance we need to handle. Over time we’ve tested different models to try and see what’s best and are now using what tested most accurately. (Hint: it’s not simply averaging the MMRs)
The other issue is how do we model the synergistic effects of players being together in a group. As you noted, they have access to voice chat. Now here’s where things get interesting. This “massive” advantage actually differs based upon the skill rating of the group members. Based upon the data we’ve seen groups of low to mid SR players don’t see that much improvement to their win %. Higher SR players do see more notable improvements, but it’s not as huge as you might think. Still, we do take this into account when we predict the win% for each team. Regardless of how the data looks, we do know there’s a perception of a large advantage for groups. That’s one of the reasons why we explicitly try to match similar sized groups together.
So then why do points for losses and wins seem so random? Well, the amount of MMR (and SR) you go up or down isn’t simply a matter of whether you won or lost, and what was your predicted chance of winning. There’s a couple of other things at work. One is the matchmaker’s confidence in what your MMR should be. Play a lot of games, it gets more certain. Don’t play Overwatch for a while, it gets less certain. You go on a large win or loss streak, it gets less certain. The more certain the matchmaker is about your MMR, the less your MMR will change in either direction based on a win or loss.
As a minor factor, we also do evaluate how well you played the heroes you used in a match. The comparison is largely based on historical data of people playing a specific hero (not medals, not pure damage done), and we’ve done a lot of work to this system based on the community’s feedback. In fact, I’ve seen some people indicate that they don’t think we’re doing this anymore. We still are. While it’s a minor factor compared to wins/losses (The best way to increase your SR is still to play together and win as a team!), doing so does help us determine your skill more accurately and faster.
So take all that into account, the SR gain/loss after any single match can be a bit more “noisy” that it seems it should, but we’re asking it to look at a lot of different factors to do the best job it can creating fair matches for you.
As an aside to all of this…
“Fair” matches doesn’t always mean that every Ilios match goes 3-2 and 100-99 on the final point, or each team gets the payload to the end in overtime on Dorado, etc. Sometimes when two evenly matches teams play, the result can be one-sided. It just means that at that single moment in time the enemy team played better. It’s not always the matchmaker’s, your’s, or your team’s(!!!) fault that you got stomped.