STAT 350: Lecture 14

Reparametrization

Suppose X is the design matrix of a linear model and that is the design matrix of the linear model we get by imposing some linear restrictions on the model using X. A good example is on assignment 3 but here is another. Consider the one way layout, also called the K sample problem. We have K samples from K populations with means for . Suppose the ith sample size is n+i. This is a linear model, provided we are able to assume that all the population variances are the same.

The resulting design matrix is

where there are copies of the first row, and then copies of and so on. Under the restriction there is just one parameter, say , and the design matrix is just a column of 1s. Notice that is not a submatrix of the original design matrix X. However, if A is the matrix with all entries equal to 1 we have so that the column space of is a subspace (of dimension 1) of the K dimensional column space of X. The extra sum of squares principle can thus be used to test the null hypothesis by fitting the two models and computing, using the notation of ANOVA from STAT 330:

The numerator of this sum simplifies algebraically to

so that this extra Sum of Squares F test is the usual F test in this problem; this is universal.

It is actually quite common to reparametrize the full model in such a way that the null hypothesis of interest is of the form . For the 1 way ANOVA there are two such reparametrizations in common use.

The first of these defines a grand mean parameter and individual ``effects'' . This new model has K+1 parameters apparently and the corresponding design matrix, , would not have full rank; its rank would be K although it would have K+1 columns. As such the matrix would be singular and we could not find unique least squares estimates. The problem is that we have defined the parameters in such a way that there is a linear restriction on them, namely, . We get around this problem by dropping and remembering in our model equations that .

If you now write out the model equations with and the as parameters you get the design matrix

Students will have seen this matrix in 330 in the case where all the are the same and the fractions in the last rows of are all equal to -1. Notice that the hypothesis is the same as .

The other reparametrization is ``corner-point coding'' where we define new parameters by and . For this parameterization the null hypothesis of interest is . The design matrix is