When the Coward Debuff elaspes, perhaps a new debuff could replace it called Stain of Cowardice that reduces all reputation gains by 20%, but otherwise is just a flavor Debuff.
Also, the only way to remove it is to challenge the original person you were facing to another duel to the death.