Dropped from party for skipping cutscene for Woodsman's House

There is a quest very early in Act 1 where you’re in a Woodsman’s House. (Sorry I don’t remember the exact quest name - you open a secret back wall behind his bookcase, then he comes in with a deer carcass over his shoulder and invites you to supper.) On two separate characters, while I was in a party during this quest, if I skipped the cutscene I got dropped from the party and sent back to the beginning of the map.