Correct. Just ignore the number. It does not update in real time. It will go from 165 to 88 to IN without a count down.
They mention it in the patch notes
We should note that the higher your queue number appears, the slower the number will refresh in the prompt. The number is still refreshing in the background, so we do not recommend leaving queue as this will create further delays to you entering the game during these high traffic windows.