This sounds like it is working as intended, unless you’re saying that your ilvl 480 pants are being downgraded to ilvl 400-450.
Season 4 is a “victory lap” season, where the player base voted for the best DF look and bonuses for each class spec, resulting in a mismatch of appearances and bonuses for many classes.
Similar to any other tier, set bonuses are not compatible with previous tiers (despite them potentially being the same bonuses for this season).