Eventually these things usually time out after an hour or so. Otherwise the work around used to be (not sure if still works) to alt+F4 at the “rejoin” prompt after restarting the client. When the client is next started it should (in the past) not try to rejoin.