Parameterization of Mixed Models

The MIXED Procedure

Parameterization of Mixed Models

Recall that a mixed model is of the form

$y= X{\beta}+ Z{\gamma}+ {\epsilon}$

where y represents univariate data, ${\beta}$ is an unknown vector of fixed effects with known model matrix X, ${\gamma}$ is an unknown vector of random effects with known model matrix Z, and ${\epsilon}$ is an unknown random error vector.

PROC MIXED constructs a mixed model according to the specifications in the MODEL, RANDOM, and REPEATED statements. Each effect in the MODEL statement generates one or more columns in the model matrix X, and each effect in the RANDOM statement generates one or more columns in the model matrix Z. Effects in the REPEATED statement do not generate model matrices; they serve only to index observations within subjects. This section shows precisely how PROC MIXED builds X and Z.

Intercept

By default, all models automatically include a column of 1s in X to estimate a fixed-effect intercept parameter $\mu$ . You can use the NOINT option in the MODEL statement to suppress this intercept. The NOINT option is useful when you are specifying a classification effect in the MODEL statement and you want the parameter estimate to be in terms of the mean response for each level of that effect, rather than in terms of a deviation from an overall mean.

By contrast, the intercept is not included by default in Z. To obtain a column of 1s in Z, you must specify in the RANDOM statement either the INTERCEPT effect or some effect that has only one level.

Regression Effects

Numeric variables, or polynomial terms involving them, may be included in the model as regression effects (covariates). The actual values of such terms are included as columns of the model matrices X and Z. You can use the bar operator with a regression effect to generate polynomial effects. For instance, X|X|X expands to X X*X X*X*X, a cubic model.

Main Effects

If a class variable has m levels, PROC MIXED generates m columns in the model matrix for its main effect. Each column is an indicator variable for a given level. The order of the columns is the sort order of the values of their levels and can be controlled with the ORDER= option in the PROC MIXED statement. The following table is an example.

Data I A B

A B $\mu$ A1 A2 B1 B2 B3

1 1 1 1 0 1 0 0

1 2 1 1 0 0 1 0

1 3 1 1 0 0 0 1

2 1 1 0 1 1 0 0

2 2 1 0 1 0 1 0

2 3 1 0 1 0 0 1

Typically, there are more columns for these effects than there are degrees of freedom for them. In other words, PROC MIXED uses an over-parameterized model.

Interaction Effects

Often a model includes interaction (crossed) effects. With an interaction, PROC MIXED first reorders the terms to correspond to the order of the variables in the CLASS statement. Thus, B*A becomes A*B if A precedes B in the CLASS statement. Then, PROC MIXED generates columns for all combinations of levels that occur in the data. The order of the columns is such that the rightmost variables in the cross index faster than the leftmost variables. Empty columns (that would contain all 0s) are not generated for X, but they are for Z.

Data I A B A*B

A B $\mu$ A1 A2 B1 B2 B3 A1B1 A1B2 A1B3 A2B1 A2B2 A2B3

1 1 1 1 0 1 0 0 1 0 0 0 0 0

1 2 1 1 0 0 1 0 0 1 0 0 0 0

1 3 1 1 0 0 0 1 0 0 1 0 0 0

2 1 1 0 1 1 0 0 0 0 0 1 0 0

2 2 1 0 1 0 1 0 0 0 0 0 1 0

2 3 1 0 1 0 0 1 0 0 0 0 0 1

In the preceding matrix, main-effects columns are not linearly independent of crossed-effect columns; in fact, the column space for the crossed effects contains the space of the main effect.

When your model contains many interaction effects, you may be able to code them more parsimoniously using the bar operator ( | ). The bar operator generates all possible interaction effects. For example, A|B|C expands to A B A*B C A*C B*C A*B*C. To eliminate higher-order interaction effects, use the at sign ( @ ) in conjunction with the bar operator. For instance, A|B|C|D@2 expands to A B A*B C A*C B*C D A*D B*D C*D.

Nested Effects

Nested effects are generated in the same manner as crossed effects. Hence, the design columns generated by the following two statements are the same (but the ordering of the columns is different):

   model Y=A B(A);

   model Y=A A*B;

The nesting operator in PROC MIXED is more a notational convenience than an operation distinct from crossing. Nested effects are typically characterized by the property that the nested variables never appear as main effects. The order of the variables within nesting parentheses is made to correspond to the order of these variables in the CLASS statement. The order of the columns is such that variables outside the parentheses index faster than those inside the parentheses, and the rightmost nested variables index faster than the leftmost variables.

Data I A B(A)

A B $\mu$ A1 A2 B1A1 B2A1 B3A1 B1A2 B2A2 B3A2

1 1 1 1 0 1 0 0 0 0 0

1 2 1 1 0 0 1 0 0 0 0

1 3 1 1 0 0 0 1 0 0 0

2 1 1 0 1 0 0 0 1 0 0

2 2 1 0 1 0 0 0 0 1 0

2 3 1 0 1 0 0 0 0 0 1

Note that nested effects are often distinguished from interaction effects by the implied randomization structure of the design. That is, they usually indicate random effects within a fixed-effects framework. The fact that random effects can be modeled directly in the RANDOM statement may make the specification of nested effects in the MODEL statement unnecessary.

Continuous-Nesting-Class Effects

When a continuous variable nests with a class variable, the design columns are constructed by multiplying the continuous values into the design columns for the class effect.

Data I A X(A)

X A $\mu$ A1 A2 X(A1) X(A2)

21 1 1 1 0 21 0

24 1 1 1 0 24 0

22 1 1 1 0 22 0

28 2 1 0 1 0 28

19 2 1 0 1 0 19

23 2 1 0 1 0 23

This model estimates a separate slope for X within each level of A.

Continuous-by-Class Effects

Continuous-by-class effects generate the same design columns as continuous-nesting-class effects. The two models are made different by the presence of the continuous variable as a regressor by itself, as well as a contributor to a compound effect.

Data I X A X*A

X A $\mu$ X A1 A2 X*A1 X*A2

21 1 1 21 1 0 21 0

24 1 1 24 1 0 24 0

22 1 1 22 1 0 22 0

28 2 1 28 0 1 0 28

19 2 1 19 0 1 0 19

23 2 1 23 0 1 0 23

You can use continuous-by-class effects to test for homogeneity of slopes.

General Effects

An example that combines all the effects is X1*X2*A*B*C(D E). The continuous list comes first, followed by the crossed list, followed by the nested list in parentheses. You should be aware of the sequencing of parameters when you use the CONTRAST or ESTIMATE statements to compute some function of the parameter estimates.

Effects may be renamed by PROC MIXED to correspond to ordering rules. For example, B*A(E D) may be renamed A*B(D E) to satisfy the following:

Class variables that occur outside parentheses (crossed effects) are sorted in the order in which they appear in the CLASS statement.
Variables within parentheses (nested effects) are sorted in the order in which they appear in the CLASS statement.

The sequencing of the parameters generated by an effect can be described by which variables have their levels indexed faster:

Variables in the crossed list index faster than variables in the nested list.
Within a crossed or nested list, variables to the right index faster than variables to the left.

For example, suppose a model includes four effects - A, B, C, and D -each having two levels, 1 and 2. If the CLASS statement is

   class A B C D;

then the order of the parameters for the effect B*A(C D), which is renamed A*B(C D), is

$A_1 B_1 C_1 D_1 arrow & A_1 B_2 C_1 D_1 arrow & A_2 B_1 C_1 D_1 arrow & A_2 B... ...1 B_2 C_2 D_2 arrow & A_2 B_1 C_2 D_2 arrow & A_2 B_2 C_2 D_2 \hspace*{.2in}$

Note that first the crossed effects B and A are sorted in the order in which they appear in the CLASS statement so that A precedes B in the parameter list. Then, for each combination of the nested effects in turn, combinations of A and B appear. The B effect moves fastest because it is rightmost in the cross list. Then A moves next fastest, and D moves next fastest. The C effect is the slowest since it is leftmost in the nested list.

When numeric levels are used, levels are sorted by their character format, which may not correspond to their numeric sort sequence (for example, noninteger levels). Therefore, it is advisable to include a desired format for numeric levels or to use the ORDER=INTERNAL option in the PROC MIXED statement to ensure that levels are sorted by their internal values.

Implications of the Nonfull-Rank Parameterization

For models with fixed-effects involving class variables, there are more design columns in X constructed than there are degrees of freedom for the effect. Thus, there are linear dependencies among the columns of X. In this event, all of the parameters are not estimable; there is an infinite number of solutions to the mixed model equations. PROC MIXED uses a generalized (g2) inverse to obtain values for the estimates (Searle 1971). The solution values are not displayed unless you specify the SOLUTION option in the MODEL statement. The solution has the characteristic that estimates are 0 whenever the design column for that parameter is a linear combination of previous columns. With this parameterization, hypothesis tests are constructed to test linear functions of the parameters that are estimable.

Some procedures (such as the CATMOD procedure) reparameterize models to full rank using restrictions on the parameters. PROC GLM and PROC MIXED do not reparameterize, making the hypotheses that are commonly tested more understandable. Refer to Goodnight (1978) for additional reasons for not reparameterizing.

Missing Level Combinations

PROC MIXED handles missing level combinations of classification variables similarly to the way PROC GLM does. Both procedures delete fixed-effects parameters corresponding to missing levels in order to preserve estimability. However, PROC MIXED does not delete missing level combinations for random-effects parameters because linear combinations of the random-effects parameters are always estimable. These conventions can affect the way you specify your CONTRAST and ESTIMATE coefficients.

Chapter Contents
Previous
Next
Top

Data		I	A		B
A	B	$\mu$	A1	A2	B1	B2	B3
1	1	1	1	0	1	0	0
1	2	1	1	0	0	1	0
1	3	1	1	0	0	0	1
2	1	1	0	1	1	0	0
2	2	1	0	1	0	1	0
2	3	1	0	1	0	0	1

Data		I	A		B			*AB**
A	B	$\mu$	A1	A2	B1	B2	B3	A1B1	A1B2	A1B3	A2B1	A2B2	A2B3
1	1	1	1	0	1	0	0	1	0	0	0	0	0
1	2	1	1	0	0	1	0	0	1	0	0	0	0
1	3	1	1	0	0	0	1	0	0	1	0	0	0
2	1	1	0	1	1	0	0	0	0	0	1	0	0
2	2	1	0	1	0	1	0	0	0	0	0	1	0
2	3	1	0	1	0	0	1	0	0	0	0	0	1

Data		I	A		B(A)
A	B	$\mu$	A1	A2	B1A1	B2A1	B3A1	B1A2	B2A2	B3A2
1	1	1	1	0	1	0	0	0	0	0
1	2	1	1	0	0	1	0	0	0	0
1	3	1	1	0	0	0	1	0	0	0
2	1	1	0	1	0	0	0	1	0	0
2	2	1	0	1	0	0	0	0	1	0
2	3	1	0	1	0	0	0	0	0	1

Data		I	A		X(A)
X	A	$\mu$	A1	A2	X(A1)	X(A2)
21	1	1	1	0	21	0
24	1	1	1	0	24	0
22	1	1	1	0	22	0
28	2	1	0	1	0	28
19	2	1	0	1	0	19
23	2	1	0	1	0	23

Data		I	X	A		*XA**
X	A	$\mu$	X	A1	A2	X*A1	X*A2
21	1	1	21	1	0	21	0
24	1	1	24	1	0	24	0
22	1	1	22	1	0	22	0
28	2	1	28	0	1	0	28
19	2	1	19	0	1	0	19
23	2	1	23	0	1	0	23