Most studies on the ethics of privacy focus on what others ought to do to accommodate our interest in privacy. I focus on a related but distinct question that has attracted less attention in the literature: When, if ever, does morality require us to safeguard our own privacy? While we often have prudential reasons for safeguarding our privacy, we are also, at least sometimes, morally required to do so. I argue that we, sometimes, ought to safeguard our privacy for the sake of the possible wrongdoer, in order to prevent the possible wrongdoer from committing a morally defiling act that renders them worse off. I illustrate the variety of cases in which this duty binds us and contrast my view to some other claims in the literature which also purport to show that we sometimes have a duty to safeguard our own privacy.