Thanks for the list. I am familiar with most of these games, but I do not think many of them meet the criteria I was looking for.
Many of these (Warframe, PoE, D3, D4, Monster Hunter, DRG, Vermintide) are solo games that can be played co-op (I’m not sure about Sea of Thieves or Dying Light) which is not the same as a game designed around teamwork PvE, like WoW’s raids and dungeons.
(Some don’t even have challenging content. Over the past year, the Warframe devs basically admitted that, given the game’s core design, it was impossible for them to try to make truly challenging content and so they were no longer going to even try.)
I didn’t specifically ask for lobby-based PvE games, but I’m surprised you did not mention Wayfinder, which seems to be trying to make a mostly lobby-based PvE game (though, again, there doesn’t seem to be any group-only content). 
They’re catering to these players because M+ is popular. I think a lot of people forget that before M+, dungeons became easier as the expansion progressed and stopped providing players with any challenge, leading to tedium and burnout and the “gogogo” mentality.
If anything, M+ saved dungeons by keeping them relevant throughout the expansion, both in terms of challenge and rewards.
I’m 99.9% sure that the reason Challenge Modes existed in MoP and WoD is because Blizzard fundamentally misunderstood why players kept trying to speed run Heroic dungeons.
It’s either that or the obvious solution, scaling dungeon difficulty, was not something they wanted to spend dev resources implementing and thought they could get away with lower effort CMs.
Class balance and homogenization is not a consequence of M+ but of 10-man raids, and its roots go all the way back to Wrath of the Lich King. Remember the “bring the player, not the class” design philosophy of that era? It had nothing to do even with dungeons in that expack, as the dungeons were seen as being generally low-challenge mechanics-wise.
The classes were designed, among other reasons, so that 10-man raiding with friends could be viable.
Players who hate M+ keep saying this, but I’ve never seen any real evidence for this.
If Blizzard truly wanted M+ to be an e-sport, where’s the e-sports league, a la OWL, for M+? Just because once a season (which translates to about twice a year) Blizzard hosts the MDI does not mean M+ was designed to be an e-sport. If it was, it’s been a complete and total failure on that front, as there’s no viable way for any gamer to make a living playing M+.
The other thing that almost every M+ hater seems to forget is that the MDI is different from live M+ runs.
In the MDI, all the character item levels are normalized, and instead of trying to beat the timer, each team is trying to beat another in real time. The timer could be removed entirely and the tournament would play out exactly the same way.
On live, you’re only trying to beat the timer; there’s no competition against any other team. And as long as you max chest it, it doesn’t matter whether you beat the timer by 5 minutes or 5 seconds.
And finally, consider that the Mythic raid race to world first (RWF) exists. Does that mean raids were designed to be e-sports? 
It’s not that we don’t realize, it’s that we honestly don’t care.
Lots of players like Classic. Great! Have fun!
I’ll be on retail having fun running M+ and raids.
