Chromaggus Drop Rates

Gameplay Dungeons, Raids and Scenarios
31

Data has been updated through this week’s run…still have not seen the same 7 items. I also reran my Chi-Square test except instead of assuming the drop rates followed WowHead’s, I assumed that each of the 7 I am missing has a 0.5% drop rate (and then I distributed the remaining probability between the other items accordingly). Here are the results of that test:

Item Actual Expected Chi Contribution
Bloodfang Spaulders 22 29 1.76
Dragonstalker’s Spaulders 23 28 0.88
Pauldrons of Transcendence 31 28 0.39
Chromatic Boots 25 27 0.21
Epaulets of Ten Storms 37 27 3.40
Stormrage Pauldrons 21 27 1.50
Nemesis Spaulders 20 26 1.42
Netherwind Mantle 21 26 0.98
Taut Dragonhide Shoulderpads 27 26 0.07
Empowered Leggings 30 24 1.32
Judgment Spaulders 29 24 0.97
Pauldrons of Wrath 22 24 0.13
Angelista’s Grasp 26 23 0.48
Chromatically Tempered Sword 23 18 1.43
Elementium Reinforced Bulwark 31 17 12.32
Taut Dragonhide Gloves 0 2 1.94
Elementium Threaded Cloak 0 2 1.94
Girdle of the Fallen Crusader 0 2 1.94
Primalist’s Linked Waistguard 0 2 1.94
Shimmering Geta 0 2 1.94
Ashjre’thul, Crossbow of Smiting 0 2 1.94
Claw of Chromaggus 0 2 1.94

Once again, even with expected drop rates that low, realistically one would have expected to have seen at least one of these items (and then I once again question why the drop rate is that low). The results of the chi-square test in this case is a value of 40.84, which equates to a p-value of approximately 0.586%, so quite low.

34

Well considering they are 1% , I believe the fact you haven’t seen it yet.

35

On my mage in vanilla, my guild killed Nefarian every week for 6 months before he ever dropped a Netherwind Robe

1 Like
36

same with the rogue hood I believe, some people never saw it -ever- for all of vanilla.

37

Two comments:

  1. the analysis above shows that even if they each have less than a 1% drop chance (like I said I used .5% for all 7 items) the likelihood of me seeing the results I am seeing is extremely, extremely unlikely.

  2. 97 runs is the equivalent of doing this every week for almost 2 years. Add on the undocumented runs and you have the equivalent of a guild running this every week for almost 4 years. For 7 items to never drop in those 4 years? That doesn’t seem a little odd?

38

I’ll admit it’s been a really, really long time since I’ve done this type of math, but I don’t think the analysis shows what you think it shows. If I’m not mistaken, the chi square test is to measure correlation between observation and expectation of that observation. Your tables all show a pretty good correlation. The items you haven’t seen are all substantially rarer in expectation than the ones you have, using both the wowhead values and the forced 0.5% you chose for the second table.

The expected outcomes of 2 in 71 for those rare items doesn’t mean that everyone who creates a set of 71 will all see 2. It means that with a statistically significant group of sets of 71, the average of all of those sets will be 2 per group. Some sets of 71 will see more than 2 and some will see fewer. Your sample size of one set of 71 is not significant, so you can’t really draw any conclusion from it.

Not necessarily, depending on the drop rate. For each individual item, using your 0.5% rate, seeing the item somewhere in the first 97 attempts would happen only 40% of the time. That rises to over 60% if the rate is 1%, but that still means that almost 40% of the time it doesn’t happen within 97 attempts. Even in 200 iterations, 13% of the time the item still won’t appear.

On the non-theoretical side, while I’m still waiting for a crossbow and the leather gloves, I have personally seen the Claw of Chromaggus and the Girdle of the Fallen Crusader, so they do exist. The other 3 I can’t comment on. I have the transmogs for them, but they share with a number of other items, so I don’t know if I got them from Chromaggus or elsewhere.

39

Yeah, I am no statistician, but I think the chi-square test is used to test the appropriateness of an expected distribution based on a sample. By evaluating the chi-square statistic, you can determine the likelihood of seeing your results if the expected distribution holds. My results show that you if you assume the expected distribution follows either that of WoWhead or the made up distinction where each of the 7 items has a small drop rate, that my samples indicate that both of those distributions are unlikely to be the true distributions and that you can say that with a fair amount of certainty.

Regarding your second point, that may be true for a single item, but now throw in the fact that we are not seeing 7 items drop, and things look even more bleak. I also need to revisit how to handle the fact that he drops multiple items. So each time the item has a .5% drop chance, for example, and in my runs the roll wouldn’t happen 97 times, it would be happening 388 times! Not sure if this is right thinking.

I understand they drop, I know there are stories of them dropping, but the question really is that they each of these items has an extremely extremely low chance of dropping- like on the order of legendary drop rates. So the big question is why? Why are these items so rare? Is it on purpose? Did someone at Blizzard make a math mistake and move a decimal place one place too far?

40

Right, but when you’re determining likelihood, what you’re doing is setting up a statistical situation for your single set of results. How likely is that set of results to occur? Unless the answer to that question is a hard 0%, then there is a chance. That means that your set is one iteration, and while perhaps unlikely, is not enough information to assess whether or not something is wrong with the loot or if your set is simply an outlier.

It doesn’t work this way. Drop rate is the rate of seeing an item per loot event, regardless of how many items are part of that event. So if the final drop rate is 1%, that is 1 out of every 100 times the boss is looted, whether there are 2 items dropped or 22.

I can chime in on the math, but these questions are over my pay grade =)

41

I don’t think you’ll ever get a “hard 0”, so you set a threshold of likelihood that you’re willing to accept and then reject the hypothesis if the result is under this threshold. So in my case, my null assumption is that it follows the likelihood from Wowhead (or the hypothetical one I made up with very small drop rates). Given that the value comes out extremely small, the test says that you can, with a high degree of confidence, reject your hypothesis. All this does is say that the drop rates are very unlikely to follow the distribution from my assumptions - it does not tell you what the expected drop rates are or what the issue with the assumption is (for example I could theoretically have the “right” drop rates for a few items but the “wrong” drop rates of others).

I think, for the second, you need to define the event. If I flip 4 coins 12 times, that should have a distribution that follows flipping 1 coin 48 times. What is the likelihood that in 1 “event” I see a heads? It should be 94%, right? But that’s not what would be published as the “drop rate” for heads.

Like I said, though, math aside (right or wrong or whatever), I think everyone would agree that the drop rates on these items are very small (look at comments on WoWHead for some of the items) and my question/concern is why? And then, also of interest to me, is why it appears as if the drop rates on WoWhead don’t appear to be close to the actual drop rates? I think what appears there represents a fairly random sample of something like 60K or more kills, so one would expect to get fairly close to the right drop rates. Have they been changed drastically?

All in all, I really just want to stop doing this stupid dungeon every week, but my neuroticism refuses to do so until I get those items, so dear RNJesus, just let them drop for me…

42

Drop rates are created from reports of loot from boss kills. Whatever the math is behind the % per item isn’t relevant to published drop rates because the published drop rates are based on how many times an item was seen on boss kills. If an item has a 5% drop rate and the boss drops 4 items, that 5% is 1 in 20 boss kills. You don’t multiply out the 5% for each item because that has already been incorporated into the drop rate. The looting event is defined by looting the boss, not by each item the boss generates.

No doubt. I’m in the same grind =)

Good luck!

43

I’m likewise not a statistician, but i’m confident this test shows exactly what turoken is claiming. The chi square goodness of fit test is testing exactly what we want and showing exactly what we want to demonstrate the items aren’t dropping anywhere near what wowhead approximates.

Substituting the .5% drop rate in i think introduces problems - we lose validity when the number of expected items drops below 5. But i don’t actually think the intended drop rates of those items are <1% - I think the intended drop rates are ~4-10% as shown on wowhead, and the chi test is perfectly valid for that.

44

Ah great point, I completely missed the requirement that the expected number of items be greater than some threshold. As you say, though, the initial test still stands - the WoWHead drop rates do not appear to be accurate, which just has me scratching my head.

One other thing, just to clear up confusion regarding the per kill versus per drop numbers. I was not using the actually “drop rates” that WoWHead publishes, I was backing into them from the number of drops. For example, if you look at the page for the Claw of Chromaggus, it lists a 6% drop rate which is calculated from 4552 times it was seen / 77451 times Chromaggus was killed. If instead, you go to Chromaggus’s page and calculate the drop rate for this item as the 4552 times it was seen / the total of all the items dropped**, which is around 287K, you get a drop rate of approximately 1.6% - this is the number I generally used and the number that I was replacing with a 0.5%.

One other thing I’ve been thinking through. Let’s forget about chi-square testing for a moment and go back to Probability 101. Let’s assume that each of the 7 items has a drop rate of x% per kill, then the probability of not seeing those 7 items in a single kill is (1-7x)%, right? And the probability of not seeing it in 2 kills is (1-7x)*(1-7x) [don’t see it on the first kill and then also don’t see if on the second kill). Thus, the probability of NOT seeing all the items in y kills is (1-7x)^y, right?

So I have, at this point (though I have not updated the stats above) killed (recorded) Chromaggus 102 times. That means the probability of me NOT seeing any of the 7 items under different drop rate assumptions is:

5% drop rate: approximately 1 in 12 QUINTILLION (essentially 0%)
1% drop rate: approximately 1 in 1640 (0.6%)
0.5% drop rate: approximately 1 in 38 (2.6%)

Now in reality, they do not each have the same drop rate, but I think the conclusion is clear - under any assumption of the drop rate per kill, the probability that I haven’t seen any of these items is extremely small.

And if the drop rate on each of these items really is something as small as 0.5% per kill…why? Why such a small drop rate for these items - even the legendary bindings off of Geddon/Garr appear to have higher drop rates than that.

And that is why I just feel like there is a bug somewhere.

** I exclude the Whistle of the Chromatic Bone from my calculations as it appears this is an independent drop. That is, Chromaggus either drops 4 items and none of them is the whistle OR he drops 5 items and the fifth is always the whistle.

45

killing something on a 1-5% drop rate at 100 times isn’t 1 in 12 quintillion

46

Yes, you’re right, however that’s not what I said. If 7 items each had a 5% drop rate per kill, that means that the odds of not seeing any of those items on one kill is 65%. And in 100 kills, the odds of never seeing any of those items is .65^100, which is where the “1 in 12 quintillion” came from.

One edit though as I sit on the train going to work. I think the math checks out above if the items each have a 5% drop chance and only 1 item drops per kill. Since the 5% I am using here is a per kill metric and he drops multiple items, it is not correct to simply take 1 - 7 x 5% as the odds of none of the 7 items dropping on a single kill. Need to account for some overlaps between the various outcomes, sorry about that.

So the probabilities I site above are likely too low in this environment, but I don’t think they will be significantly different if I fixed it (I may, if I have time today, try to finish the math and do it correctly) - I believe my point will still stand.

47

I checked with a statistics professor and confirmed that the chi square test establishes only establishes a probability, not a certainty. With only Turoken’s data, we can’t confirm that his sample isn’t an outlier. Like any probability, we would need multiple samples all showing similar results to confirm that there is an actual issue. If you think about doing set of 97 die rolls, and 50 of them came up 6, you couldn’t conclude that the die is loaded without doing more trials. All you would have is a set of 97 that indicates that either the die is loaded OR you got an extremely unlikely outcome on the one sample.

You’re right that this is a more complex problem, and I’m actually working on setting this up to find a definitive answer, but it will take some time. I’m also revisiting the notion that the drop rate is based on looting and not for each individual drop. I’ve checked a number of bosses on wowhead, and I’m having trouble getting their numbers to work in either situation, so I’m diving deeper into that to see what’s happening.

I’m really intrigued by how complex this is becoming and I’m looking forward to finding a concrete mathematical solution. Hopefully I won’t be the only person not bored by all of this by the time we reach it.

48

i mean, yes probabilities are probabilities not certainties. but we’ve already accounted for this by the p value - a chi sqaured test provides you with the probability of the outcome matching - that is the entire point of the test.

the chi contribution is 93.28, there are 21 degrees of freedom. typically, a p value of say .05 is considered adequate to say the distributions do not match. our p value is 4.878e-11. the odds of the drop rate being what wowhead says is 1 in 20 billion. it is far, far more likely that the odds are not what wowhead lists, than that OP is just on a 1 in 20 billion bad luck streak.

the problem turoken is trying to do in his latest post basically IS a chi squared analysis. we can’t just multiply the odds of each item having not dropped together because they are dependent variables. given their low relative drop rates, simply multiplying probably isn’t that awful of an approximation. but a chi squared test is a more sophisticated way of doing this without approximating.

49

Yes, I never meant to imply a certainty with my analysis - however, the p-value established shows that there is an extremely low probability that I am seeing the results that I am seeing if the drop rates from WoWHead are to believed. Thus, we can effectively throw out the hypothesis that those are the “correct” drop rates. Again, the test doesn’t do much more than throw out that null hypothesis. It doesn’t tell us why those are not the “right answers” and it does not tell us what the “right answers” actually are.

At this point, there are only a few scenarios that I can consider:

  1. there is a bug that is preventing these items from dropping (for just me? For some subset of the WoW population?)

  2. each of these items has an extremely low drop rate. What I was trying to demonstrate above was the “extremeness” of this situation. They would all have to be really, really low (even a 1% per kill for each doesn’t seem to align with my results). Again if this is true, my big question is generally “why?” Why do 7 items off a classic-era raid appear to have drop rates lower than even some legendary items?

And the only way to know for sure is if we have a Blizzard rep comment and say that they investigated the code underlying his drop rates and all was in line with expectations…and I don’t think that’s going to happen…

50

Plenty of vanilla epic items seem to “never” drop
why? who knows, but I can think of plenty of items that are 1 in 10,000 drop rate or lower.

51

Any examples off the top of your head? 1 in 10,000 seems awfully low for pretty much anything.

54

Papal fez
you could do 1000 uldaman runs and won’t see a papal fez. (probably)
doesn’t mean its bugged.