In this paper we suggest two continuous-time models which exhibit an autoregressive structure. We obtain existence and uniqueness results and study the structure of the solution processes. One of the models, which corresponds to general stochastic delay differential equations, will be given particular attention. We use the obtained results to link the introduced processes to both discrete-time and continuous-time ARMA processes.