We argue that existing methods for the treatment of missing observations in time-varying parameter observation-driven models lead to inconsistent inference. We provide a formal proof of this inconsistency for a Gaussian model with time-varying mean. A Monte Carlo simulation study supports this theoretical result and illustrates how the inconsistency problem extends to score-driven and, more generally, to observation-driven models, which include well-known models for conditional volatility. To overcome the problem of inconsistent inference, we propose a novel estimation procedure based on indirect inference. This easy-to-implement method delivers consistent inference. The asymptotic properties of the new method are formally derived. Our proposed estimation procedure shows a promising performance in a Monte Carlo simulation exercise as well as in an empirical study concerning the measurement of conditional volatility from financial returns data.