You can target farm them from regular bosses (excluding Vidian). They have much higher drop rates for the soul shards.
Acts 1/2/5 bosses seem to have a guaranteed shard drop. Act 1 = Anguish, Act 2 = Lies, Act 5 = hatred.
Acts 3/4 bosses drop two different specific shards, but they didn’t drop them 100% of the time (maybe compensating for the fact that they drop more than the other bosses?). The drop chance was still high enough that you could target farm them off bosses if you want. Act 3 = Sin + Destruction, Act 4 = Terror + Pain.
It looked like each boss also had a small chance to drop a completely random shard independent of their guaranteed drop.
I posted my data (10 full runs of all bosses) here: Soul Shard boss drop mechanics - my findings
The embers definitely seem like they will be the limiting factor.