When an Elder Rift that you put crests into it fails, i.e. you either don’t get zoned into it, or you don’t complete it, the crests will be returned to your hero via the in-game email system.
This can take quite a while, but it will happen. You have not lost the crests.