D2R-Axe Mastery Looses Damage?!

Adding a second or third level to Axe Mastery (Barb) decreased my overall Axe damage (Noticed while viewing stat on character screen)