I’d like to add something about the difference between accuracy and precision, if I may. The common usage mixes the terms but to discuss what I want to discuss I need to separate the ideas.
If I were to tell you that my location was California, that would be accurate, but not very precise. Similarly, to say that your skill is between 2200 and 3400 also is likely accurate, but not very precise.
You are talking more about precision. The question then becomes one of the interface between you and the measuring system (SR). If you were to try to find me, I could likely give you my address and that would be sufficiently precise to accomplish your task. What I couldn’t do is to just give you my zip-code. You’d never find me that way. That wouldn’t be precise enough. On the other hand, I could also give you a 12 digit latitude/longitude coordinate, there are literally billions of those coordinates that would accurately describe my position and any one of those would work as long as I didn’t move.
If I was walking around my house, or even doing yardwork, by the time you got to the 12 digit coordinate it would likely be wrong. You’d be better served by the address.
Kaa’s experiment in reference (27) assumes that the precision is 4 digits and thus discusses accuracy in terms of the fact that the two accounts were 476 SR different. He’s not wrong (I mean, SR is 4 digits, it’s most reasonable to conclude that that precision is meaningful) but he would be able to say that the system gave the same (accurate) results if both accounts ended up in Gold tier. If we consider that SR is likely over-precise then the question of accuracy become a bit different. How different is 476 SR really? The answer probably depends on where that 476 range is on the overall scale.
This is where your hero and playstyle come in (and perhaps the other factors Kaa gives). If you, personally, aren’t that consistent or you play in a way that isn’t that consistent, or you play a hero that relies on factors beyond your control, then what you are doing is very much like moving around your house while someone is looking for you with a 12 digit coordinate. You keep updating the number, but a few games later it’s wrong.
Measurement systems should never be more precise than they are accurate. You wouldn’t want to use the 12 digit coordinate to find me in my house. It would only be right by sheer dumb luck. If you look at coordinate x,y but you only find empty space, you are simply inaccurate. Wrong. You of course want them to be meaningful, using a zip code wouldn’t work, but generally it’s important to match your precision to your desired accuracy.
The SR system fails dramatically in this respect. I’ve never heard someone that claimed they could tell a difference in skill level below a 200 SR difference. There are 5000 SR levels currently, so we could just take 5000, divide it by 200, and get essentially 25 meaningful tiers of game play in the MOST PRECISE version. According to Kaawumba’s experiment, we should take that 5000 and divide it by 500 to 1000. Consider if it is meaningful to have an SR difference of 1. If not, there probably shouldn’t BE the possibility of an SR difference of 1. It means absolutely nothing.
In a sense, we already have these tiers. If you forget the over precise 4 digit number and understand your rank only in terms of tiers, I think you would find your results to be a bit more accurate but THEN you have to consider not just YOUR consistency, but what “2500” or “Platinum Tier” even MEANS. The 3 possibilities Kaa just posted are all variations on the same theme, which is that the skill level that “2500” corresponds to may potentially change throughout the season. It’s a ranking system. A RELATIVE number. You’re not a Platinum player, you’re a player higher than the people in Gold but not quite as good as those in Diamond. It may seem like a pedantic difference, but it will help you to understand.
So even it the system was perfectly accurate and precise to 4 digits and you played perfectly consistently, it’s still possible for your SR to go up and down through the season. Not that it necessarily would so dramatically, but it could certainly be a factor.
Hope that helps you to understand it.