It depends on the mitigation and the activity. You don’t generally want 100% uptime on Shield Block in raids if you are only going to be taking hits ~50% of an encounter, so you can spend talents that give you more uptime elsewhere. Ironfur is similarly not worth keeping up if you aren’t going to be getting hit.
In M+, you generally want your mitigation up at all times in combat unless you’re somehow not taking damage (kiting like mad, immune, happy hour)