This is the problem though. What factory speed are you talking about?
Most of the motherboards “factory” options (defaults) are in fact still overclocking the chip.
Many have BIOS updates, but even with the update not all of them are still using “defaults” to an Intel specified default.
So yeah, you can literally be using the default settings provided by the motherboard manufacturer and still be overclocking the chip!