In this study we show how to represent a continuous time autoregressive moving average (CARMA) as a higher order stochastic delay differential equation, which may be thought of as a CAR(∞) representation. Furthermore, we show how the CAR(∞) representation gives rise to a prediction formula for CARMA processes. To be used in the above mentioned results we develop a general theory for multivariate stochastic delay differential equations, which will be of independent interest, and which will have particular focus on existence, uniqueness and representations.