Defn: iff
Defn: if and only if with the independent and each .
In this case according to our theorem
Defn: has a multivariate normal distribution if it has the same distribution as for some , some matrix of constants and .
, singular: does not have a density.
invertible: derive multivariate normal density by change of variables:
For which , is this a density?
Any but if then
Conversely, if is a positive definite symmetric matrix then there is a square invertible matrix such that so that there is a distribution. ( can be found via the Cholesky decomposition, e.g.)
When is singular will not have a density: such that ; is confined to a hyperplane.
Still true: distribution of depends only on : if then and have the same distribution.
Defn: If has density then
FACT: if for a smooth (mapping )
Linearity: for real and .
Defn: The moment (about the origin) of a real rv is (provided it exists). We generally use for .
Defn: The central moment is
Defn: For an valued random vector
Fact: same idea used for random matrices.
Defn: The ( ) variance covariance matrix of is
Example moments: If then
If now , that is, , then and
Similarly for we have with and
Theorem: If are independent and each is integrable then is integrable and
Defn: The moment generating function of a real valued is
Defn: The moment generating function of is
Example: If then
Theorem: () If is finite for all in a neighbourhood of
then
Note: means has continuous derivatives of all orders. Analytic
means has convergent power series expansion in neighbourhood of each
.
The proof, and many other facts about mgfs, rely on techniques of complex variables.
Theorem: Suppose and are valued random vectors such that
The proof relies on techniques of complex variables.
If are independent and then mgf of is product mgfs of individual :
Example: If are independent then
Conclusion: If then
Example: If then and
Theorem: Suppose
and
where
and
. Then
and have the same distribution if and only iff the
following two conditions hold:
Alternatively: if , each MVN
then
and
imply that and have
the same distribution.
Proof: If 1 and 2 hold the mgf of is
Thus mgf is determined by and .
Theorem: If then there is a matrix such that has same distribution as for .
We may assume that is symmetric and non-negative definite, or that is upper triangular, or that is lower triangular.
Proof: Pick any such that such as from the spectral decomposition. Then .
From the symmetric square root can produce an upper triangular square root by the Gram Schmidt process: if has rows then let be . Choose proportional to where so that has unit length. Continue in this way; you automatically get if . If has columns then is orthogonal and is an upper triangular square root of .
Defn: The covariance between and is
Properties:
Properties of the distribution
1: All margins are multivariate normal: if
2: : affine transformation of MVN is normal.
3: If
4: All conditionals are normal: the conditional distribution of given is Proof of ( 1): If then
So
Compute mean and variance to check rest.
Proof of ( 2): If then
Proof of ( 3): If
Proof of ( 4): first case: assume has an inverse.
Define
Now joint density of and factors
Specialization to bivariate case:
Write
Then
This simplifies to
More generally: any and :
0 | ||
Defn: Multiple correlation between and
Thus: maximize
Note
Summary: maximum squared correlation is
Notice: since is squared correlation between two scalars ( and ) we have
Correlation matrices, partial correlations:
Correlation between two scalars and is
If has variance then the correlation matrix of is with entries
If are MVN with the usual partitioned variance covariance matrix then the conditional variance of given is
From this define partial correlation matrix
Note: these are used even when are NOT MVN