4.8.6 Properties

4.8.6 Properties

As generally implemented, the models of this section are stationary and conditionally homoskedastic, but nothing in our definitions requires this. A white noise can be conditionally heteroskedastic. Any nontrivial MA, AR, or ARMA model based upon such a white noise will also be conditionally heteroskedastic.

MA processes are necessarily stationary, although some choices of coefficient matrices βk will cause components to oscillate. AR and ARMA processes need not be stationary. Depending upon coefficient matrices bk, components can increase or decrease without bound. They may also oscillate wildly. If an AR or ARMA process does not exhibit such behaviors, it is generally stationary. See Hamilton (1994) for necessary and sufficient conditions for covariance stationarity.

4.8.7 Estimation

Stochastic processes are estimated from time series using techniques of statistical estimation as appropriate. In the case of a Gaussian white noise W, terms are IID. We may treat a segment {–αW, … , –1W, 0W} of the stochastic processes as a sample, and employ sample estimators to estimate the constant covariance matrix of terms tW. Estimation for other processes may be more difficult. ML estimators are widely used for this purpose. Consider an example.

Exhibit 4.15 indicates a time series, which is graphed in Exhibit 4.16.

Exhibit 4.15: Example time series.
Exhibit 4.16: Graph of the time series of Exhibit 4.14.

We treat the time series as a realization of a segment of a univariate stationary AR(1) process:


where W is Gaussian white noise, tW ~ N(0, σ2). Parameters that need to be estimated are a, b1, and σ. Let θ = (a, b1, σ). Let tϕ(tx | θ) be the PDF of tX conditional only on θ, and let t | t –1ϕ(tx | θ, t –1x) be the PDF of tX conditional on both θ and the previous value t –1x. With tW normal, it can be shown that tX is both conditionally and unconditionally normal. It has unconditional mean


By [3.32], it has unconditional variance






Similarly, we conclude


Since terms tX are dependent, the likelihood function employs conditional probability densities t | t –1f(tx | θ, t –1x) for all but the first term:


The log-likelihood function is


Substituting in time series values from Exhibit 4.15, this becomes


We take the gradient of the log-likelihood function and set it equal to 0. Applying Newton’s method, we estimate θ = (a, b1, σ) as (0.82, 0.31, 1.50).