The armory can’t seem to tell the difference between one Soul Shard and another of the same type and level.
When switching between sets, the armory looks like it just picks any Soul Shard of the right type and level and pops it into the helm or weapon, despite not being the one that was socketed at the time of saving the build. As each of these Soul Shards have unique properties, this is obviously problematic.