Postscript version of these notes

STAT 804

Lecture 22 Notes

Properties of the Periodogram

The discrete Fourier transform

$${\hat X}(\omega) = \frac{1}{\sqrt{T}}\sum_{t=0}^{T-1} X_t \exp(2\pi \omega t i)$$

is periodic with period 1 because all the exponentials have period 1. Moreover,

$${\hat X}(1-\omega) = \frac{1}{\sqrt{T}}\sum_{t=0}^{T-1} X_t \exp(-2\pi \omega t i) \exp(2\pi t i) = {\overline{{\hat X}(\omega)}}$$

so that the periodogram satisfies

$$\vert{\hat X}(1-\omega)\vert^2 = \vert{\hat X}(\omega)\vert^2 \,$$

Thus the periodogram is symmetric around $\omega=1/2$ which is called the Nyquist or folding frequency. (The value is always 1/2 in cycles per point but usually it would be converted to cycles per time unit like year or day or whatever.)

Similarly the power spectral density $f_X$ given by

$$f_X(\omega) = \sum_{-\infty}^\infty C_X(h) \exp(2 \pi h \omega i)$$

is periodic with period 1 and satisfies

$$f_X(-\omega) = f_X(\omega)$$

which is equivalent to

$$f_X(1-\omega) = f_X(\omega) \, .$$

Spectra of Some Basic Processes

Here I compute the spectra of a few basic processes directly and then indirectly by a more powerful technique.

Direct from the definition

White Noise:

Since $C_\epsilon(k)=0$ for all non-zero $k$ we have

$$f_\epsilon(\omega) \equiv C_\epsilon(0) = \sigma^2_\epsilon \, .$$

MA(1):

For $X_t = \epsilon_t - b \epsilon_{t-1}$ we have $C_X(0)= \sigma^2(1+b^2)$ and $C_X(\pm 1) = -b\sigma^2$ so that

$$f_X(\omega) = \sigma^2(1+b^2)-b\sigma^2 (\exp(2\pi\omega i) +\exp(-2\pi\omega i))$$

$$= \sigma^2(1+b^2)-2b\sigma^2 \cos(2\pi\omega)$$

AR(1):

We have $C_X(k) = \rho^{\vert k\vert} C_X(0)$ and

$$f_X(\omega) = C_X(0)( 1+\sum_{k>0} \rho^k(\exp(2\pi\omega ki) + (\exp(-2\pi\omega ki) )$$

$$= C_X(0)( 1 + \rho ( \frac{e^{2\pi\omega i}}{1-\rho\exp(2\pi\omega i)} + \frac{e^{-2\pi\omega i}}{1-\rho\exp(-2\pi\omega i)}))$$

$$= C_X(0)( 1 + \rho \frac{\exp(2\pi\omega i)-\rho+ \exp(-2\pi\omega i)-\rho}{(1-\rho\exp(2\pi\omega i))(1-\rho\exp(-2\pi\omega i))}$$

$$= C_X(0)( 1 + \rho \frac{2(\cos(2\pi\omega)-\rho))}{1+\rho^2 - 2\rho\cos(2\pi\omega)}$$

$$= C_X(0) \frac{1-\rho^2}{1+\rho^2 -2\rho\cos(2\pi\omega)}$$

$$= \frac{\sigma_\epsilon^2}{1+\rho^2 -2\rho\cos(2\pi\omega)}$$

Using filters

Any mean 0 ARMA($p,q$) process can be rewritten in MA form as

$$X_t = \sum_{s=0}^\infty a_s \epsilon_{t-s}$$

and then the covariance of $X$ is

$$C_X(h) = \sum_{r=0}^\infty \sum_{s=0}^\infty a_r a_s Cov(\epsilon_{t+h-r}, \epsilon_{t-s})$$

Although the covariance simplifies for white noise, let us simply write $C(t+h-r -(t-s)) = C(h+s-r)$ for the covariance in this double sum so that the calculation will apply to any stationary $\epsilon$. Then plug this double sum into the definition of $f_X$ to get

$$f_X(\omega) = \sum_{h=-\infty}^\infty \exp(2\pi\omega h i) \sum_{r=0}^\infty \sum_{s=0}^\infty a_r a_s C(h+s-r) \\$$

Now write the $h$ in the complex exponential in the form $(h+s-r) +r-s$ and bring the sum over $h$ to the inside to get

$$f_X(\omega) = \sum_{r=0}^\infty a_r e^{2\pi\omega ri} \sum_{s=0}^... ..._se^{-2\pi\omega si} \sum_{h=-\infty}^\infty \exp(2\pi\omega(h+s-r)i) C(h+s-r)$$

Finally make the substitution $k=h+s-r$ in the inside sum and define

$$A(\omega) = \sum_{r=0}^\infty a_r e^{2\pi\omega ri}$$

to see that

$$f_X(\omega) = A(\omega) {\overline{A(\omega)}} f_\epsilon(\omega)$$

or

$$f_X(\omega) = \vert A(\omega)\vert^2 f_\epsilon(\omega) \, .$$

The function $A$ (or $\bar A$) is called the frequency response function and $\vert A\vert^2$ the power transfer function. The jargon gain is sometimes used for $\vert A\vert$.

The Spectrum of an ARMA($p,q$)

An ARMA($p,q$) process $X$ satisfies

$$\sum_{s=0}^p a_s X_{t-s} = \sum_{r=0}^q b_r \epsilon_{t-r}$$

so that if $Y$ is the process $\sum_{s=0}^p a_s X_{t-s}$ then

$$f_Y(\omega) = \vert A(\omega)\vert^2 f_X(\omega)$$

where $A(\omega) = \sum_{s=0}^p a_s \exp(2\pi\omega si)$. At the same time $Y_t = \sum_{r=0}^q b_r \epsilon_{t-r}$ so

$$f_Y(\omega) = \vert B(\omega)\vert^2 f_\epsilon(\omega)$$

where $B(\omega) = \sum_{s=0}^q b_s \exp(2\pi\omega si)$. Hence

$$f_X(\omega) = \frac{\vert B(\omega)\vert^2}{\vert A(\omega)\vert^2}\, .$$

For example, in the ARMA(1,1) case $X_t-aX_{t-1} = \epsilon_t-b\epsilon_{t-1}$ we find (referring to our MA(1) calculation above that $\vert B(\omega)\vert^2 = 1+b^2-2b \cos(2\pi\omega)$ and $\vert A(\omega)\vert^2 = 1+a^2-2a \cos(2\pi\omega)$ so that

$$f_X(\omega) = \frac{1+b^2-2b \cos(2\pi\omega)}{1+a^2-2a \cos(2\pi\omega} \, .$$