Yes, other races and cultures did wage war on the various trolls to conquer their land. That is obvious, it’s how the high elves, night elves and humans came to live in their current lands. However, to my knowledge, there is no evidence that the troll empires lived in total peace until the other races started fighting them.
The Gurubashi trolls practically destroyed themselves, waging constant war among themselves until they were all weaker for it. They bullied and cannibalized smaller tribes, and basically forced the Darkspear to flee from their homelands. The Amani continued to fight the humans and elves for centuries, despite knowing that they were much weaker.
Honestly, the only troll empires that did well for themselves were the Zandalari, who were completely unopposed in their lands but still fell to an internal corruption (See: Zul’s rebellion and the scheming of Rakastan’s advisors), and the Drakkari. The Drakkari were a powerful empire in Northrend, tied in power to the Nerubians as superpowers in Northrend, and they did VERY well for themselves until Arthas turned his attention to them.
The thing that is important to remember is that the Amani (who are the main tribe in constant warfare with other races) are still around. The humans and the elves defeated them, and then saw it fit to basically say “You stay away from our stuff and we’ll stay away from your stuff”. There are still troll villages scattered around Lordaeron, Zul’Aman is still a standing city. The troll, faced with a MUCH stronger foe, could have chosen to accept the terms and rebuild. But they didn’t, they decided to constantly raid the countryside and hold onto the desire to reclaim ALL their land one day. And where has that led them? They live in ruins, most of their fighting population is set off to fight for this baseless dream of one day winning back territory they haven’t controlled in centuries, and their culture has stagnated.
Regardless of how you look at it, the trolls would be better off to find a way to live in peace with their neighbours, even if those neighbours are massive pricks. The Zandalari have good relations with the Vulpera and Tortollen, they trade with various cultures across Azeroth, and they are stronger for it. The Amani and Gurubashi, they cling to the memories of being grand empires and became spiteful and xenophobic as a result.
I am not saying that the trolls deserve what they got; they are clearly victims here, but they don’t need to remain that way. They could have picked themselves back up, fortified the lands they did control, and found some way to live in this new world. They choose not to, and that has meant the doom of their race.