Unable to port after exiting a Overcharged Delve

I do delves in a 2 man group of mage and warlock.

After exiting an overcharged delve the mage will cast a portal, but when I try to use it, I get the sound and animation yet I cannot port.
Using the hearthstone also refuses to work.

The only way to fix it for me is to exit WoW completely and then restart from scratch.