In this talk we will demonstrate how solutions to Lévy-driven stochastic delay differential equations (SDDEs) may be viewed as a continuous-time counterpart of the discrete-time AR($\infty$) processes. In relation to this we discuss how the two classes share several properties: (i) As an analogue to the result that invertible ARMA processes form a subclass of the AR($\infty$) processes, we show that invertible CARMA processes are solutions to suitable SDDEs. This result is useful for prediction and estimation. (ii) As an analogue to the result that certain AR($\infty$) processes (such as ARFIMA processes) possess long memory in the sense that their auto-correlation functions are hyperbolically decaying, we show that certain SDDEs produce solutions with a similar long memory behavior. This result gives rise to a modeling framework producing moving average processes, which are both semimartingales and possess long memory.