No, it is not that simple. Double-edged sword.
While it remains in question if you need to progress the campaign with player housing, Garrisons required the player to have one to progress. “Forced Introvert”. It SUUUUUUUUCKED.
Also, to my beloved GD users who flagged my thread, despite it having relevancy:
Disclaimer: My opinion is mine and mine alone. Some may have the same view as mine, some may not. Regardless, if the community of GD decides to pull out their torches and pitchforks and decides to flag in an unjustified manner, you are part of the reason why the devs will not listen to us.