I was redirected to post this over here from a GM (not sure how it ended up there, possibly my fault).
The gist is that there needs to be an option to disable SMS protect while retaining the phone number on an account. Sim-jacking/Sim-swapping is a very real threat to security, and when the system allows text messaging to bypass the authenticator on login, and allow it to also be used to remove the authenticator, all on its own, well that’s not good. I know this because I was allowed to do just this after I had left it on an old phone, and not once was I forced to use the auth backup codes. I have since removed my phone number from my account to close up this security hole.
In short, it should never be assumed that a person has control of a phone number, as it is not difficult to steal it if someone wants to.