I just found out that once you build the second and third production building permanently increases unit production cool down (even if they are destroyed).
I thought once I have the necessary structures/upgrades I would just have to destroy extra production buildings so I can benefit short cooldown again, but no.
Once the cooldown is lengthened by second or third production building it stays even if those are destroyed.
Why this is such a problem is you really need an Armory or Barrack so you can get necessary upgrades (For Mechanical/Starport units) and defence (Railguns for trains) but you need to build other production buildings to unlock them.
My proposal is while enabling tech trees (spending more time and resources is a penalty on its own) but remove permanent longer cooldown if the second and third buildings are destroyed.