Recently the 10.0 UI revamp has made Legion artifacts’ “gem socketing” window very buggy, usually not opening at all. I’ve done some research and found a temporary work-around.
Equip the artifact first, and then shift-right-click the artifact in your character (C) window. This should open the “gem socketing” window for now, and allows you to insert relics. 