STAT 802: Multivariate Analysis

Course outline:

• Multivariate Distributions.

• The Multivariate Normal Distribution.

• The 1 sample problem.

• Paired comparisons.

• Repeated measures: 1 sample.

• One way MANOVA.

• Two way MANOVA.

• Profile Analysis.

• Multivariate Multiple Regression.

• Discriminant Analysis.

• Clustering.

• Principal Components.

• Factor analysis.

• Canonical Correlations.

Basic structure of typical multivariate data set:

Case by variables: data in matrix. Each row is a case, each column is a variable.

Example: Fisher's iris data: 5 rows of 150 by 5 matrix:

 Case Sepal Sepal Petal Petal # Variety Length Width Length Width 1 Setosa 5.1 3.5 1.4 0.2 2 Setosa 4.9 3.0 1.4 0.2 &vellip#vdots; &vellip#vdots; &vellip#vdots; &vellip#vdots; &vellip#vdots; &vellip#vdots; 51 Versicolor 7.0 3.2 4.7 1.4 &vellip#vdots; &vellip#vdots; &vellip#vdots; &vellip#vdots; &vellip#vdots; &vellip#vdots;
Usual model: rows of data matrix are independent random variables.

Vector valued random variable: function such that, writing ,

defined for any const's .

Cumulative Distribution Function (CDF) of : function on defined by

Defn: Distribution of rv is absolutely continuous if there is a function such that

 (1)

for any (Borel) set . This is a dimensional integral in general. Equivalently

Defn: Any satisfying () is a density of .

For most is differentiable at and

Building Multivariate Models

Basic tactic: specify density of

Tools: marginal densities, conditional densities, independence, transformation.

Marginalization: Simplest multivariate problem

(or in general is any ).

Theorem 1   If has density and then has density

is the marginal density of and the joint density of but they are both just densities. Marginal'' just to distinguish from the joint density of .

Independence, conditional distributions

Def'n: Events and are independent if

(Notation: is the event that both and happen, also written .)

Def'n: , are independent if

for any .

Def'n: and are independent if

for all and .

Def'n: Rvs independent:

for any .

Theorem:

1. If and are independent with joint density then and have densities and , and

2. If and independent with marginal densities and then has joint density

3. If has density and there exist and st for (almost) all then and are independent with densities given by

Theorem: If are independent and then are independent. Moreover, and are independent.

Conditional densities

Conditional density of given :

in words conditional = joint/marginal''.

Change of Variables

Suppose with having density . Assume is a one to one (injective") map, i.e., if and only if . Find :

Step 1: Solve for in terms of : .

Step 2: Use basic equation:

and rewrite it in the form

Interpretation of derivative when :

which is the so called Jacobian.

Equivalent formula inverts the matrix:

This notation means

but with replaced by the corresponding value of , that is, replace by .

Example: The density

is the standard bivariate normal density. Let where and is angle from the positive axis to the ray from the origin to the point . I.e., is in polar co-ordinates.

Solve for in terms of :

so that
 argument

It follows that

Next: marginal densities of , ?

Factor as where

and

Then

so marginal density of is a multiple of . Multiplier makes but in this case

so that

(Special Weibull or Rayleigh distribution.) Similarly

which is the Uniform density. Exercise: has standard exponential distribution. Recall: by definition has a distribution on 2 degrees of freedom. Exercise: find density.

Remark: easy to check .

Thus: have proved original bivariate normal density integrates to 1.

Put . Get

So .

Richard Lockhart
2002-09-25