In a series of excellent, recent papers, Deborah Hellman expounds the intuitively appealing idea that we have a duty not to compound injustice. Roughly, one compounds injustice when facts that obtain as a result of prior injustice form part of one’s reason for imposing further disadvantages on the victims of this prior injustice. This article identifies several complexities and problems motivating various amendments to Hellman’s formulation of the duty not to compound injustice. Critically, it argues that the intuitions she and others have sought to explain in terms of the duty not to compound injustice are better explained by the duty not to cause additional harm to people who are already unjustly worse off – perhaps in conjunction with a duty not to implicate ourselves in injustice.