It would appear that this issue is APFS related. Or more accurately, a combination of APFS and the game’s own filesystem, CASC.
The issue with CASC is that over time as patches are downloaded, the data and index files become fragmented, which slows performance down, sometimes noticeably. And because CASC files are patched in many different spots, every single change is accompanied by a new b-tree extents node under APFS, which causes severe performance degredation due to having to enumerate every last node for each file that is read from on the drive. It gets even worse if you are using a platter based hard drive instead of an SSD (SSDs mitigate APFS’ enumeration issues to an extent, but not completely as you’ve seen here in this thread).
One way to fix this for an APFS system, at least in the short term, is to uninstall and reinstall Diablo 3. This is actually easier than it sounds. To do so, you simply need to go into your /Diablo III folder and delete the /Data folder (while the game is not running of course!), then emptying the trash. When you start up the Battle.net app, instead of “Play”, you’ll see the big blue button for Diablo 3 reads as “Update”. Click it and let the game reinstall. You’ll have a fresh set of files that are as contiguous as possible, have no edits or changes (patches) to them, and will run markedly faster under APFS than a game that has been patched multiple times.
Note: If you have the Diablo 3 PTR installed, you need to delete that first, otherwise the Battle.net app will attempt to copy existing data over from the PTR folder and then patch after that has completed. You don’t want that because that just trades one set of fragmented files for another. With the PTR folder deleted, you’ll have a roughly 17 GB download. Go make a snack or three unless you have 300 Mbit/sec or better internet, because 17 GB takes a while to download.
This should restore some, if not all of your performance. The icons are loading slowly because they’re being read very slowly from the drive, and it’s the combo of APFS and CASC causing it. It doesn’t matter how fast your internet is because the icons aren’t downloaded, they’re read from the game data on the drive that Diablo 3 is installed onto.
Also, it should go without saying, but do not click the Play button until the installation has completed. Doing so will almost always corrupt your game’s files. CASC is finnicky like that. Always let the installation or patching finish before trying to play.