STAT 801

Problems: Assignment 5

Postscript version of these questions

1. Suppose $\{X_{ij}; j=1, \ldots ,n_i ; i=1, \ldots ,p\}$ are independent $N( \mu_i , \sigma^2)$ random variables. (This is the usual set-up for the one-way layout.)
   1. Find the MLE's for $\mu_i$ and $\sigma$ .

   2. Find the expectations and variances of these estimators.

2. Let $T_i$ be the error sum of squares in the $i$th cell in the previous question.
   1. Find the joint density of the $T_i$.

   2. Find the best estimate of $\sigma^2$ of the form $\sum_1^p a_i T_i$ in the sense of mean squared error.

   3. Do the same under the condition that the estimator must be unbiased.

   4. If only $T_1, \ldots T_p$ are observed what is the MLE of $\sigma$?

   5. Find the UMVUE of $\sigma^2$ for the usual one-way layout model, that is, the model of the last two questions.

3. In question 1 take $n_i=2$ for all $i$ and let $p\to\infty$ . What happens to the MLE of $\sigma$?

4. Suppose that $Y_1,\ldots,Y_n$ are independent random variables and that $x_1,\ldots,x_n$ are the corresponding values of some covariate. Suppose that the density of $Y_i$ is
   
   $$f(y_i)=\exp\left(-y_i\exp(-\alpha - \beta x_i)- \alpha - \beta x_i\right)1(y_i > 0 )$$
   
   
   where $\alpha$, and $\beta$ are unknown parameters.
   1. Find the log-likelihood, the score function and the Fisher information.

   2. For the data set in

      ~lockhart/teaching/courses/801/Data/Extreme_Value_Data

      fit the model and produce a contour plot of the log-likelihood surface, the profile likelihood for $\beta$ and an approximate 95% confidence interval for $\beta$.

5. Consider the random effects one way layout. You have data $X_{ij}; i=1, \ldots ,p;j=1, \ldots ,n$ and a model $X_{ij} = \mu+ \alpha_i + \epsilon_{ij}$ where the $\alpha$'s are iid $N(0, \tau^2)$ and the $\epsilon$'s are iid $N(0, \sigma^2)$. The $\alpha$s are independent of the $\epsilon$s.
   1. Compute the mean and variance covariance matrix of the vector you get by writing out all the $X_{ij}$ as a vector.

   2. Suppose that $M$ is a matrix of the form $aI+b11^t$ where $I$ is a $p\times p$ identity and $1$ denotes a column vector of $p$ ones. Show that $M^{-1}$ is of the form $cI+d11^t$ and find $c$ and $d$. In what follows you may use the fact that the determinant of $M$ is $a^{p-1}(a+pb)$.

   3. Write down the likelihood.

   4. Find minimal sufficient statistics.

   5. Are they complete?

   6. Data sets like this are usually analyzed based on the fixed effects ANOVA table. Use the formulas for expected mean squares in this table to develop ``method of moments'' estimates of the three parameters. (Because the data are not iid this is not going to be exactly the same technique as the examples in class.)

   7. Can you find the MLE's?

6. For each of the doses $d_1, \ldots , d_p$ a number of animals $n_1, \ldots , n_p$ are treated with the corresponding dose of some drug. The number dying at dose $d$ is Binomial with parameter $h(d)$. A common model for $h(d)$ is $\log(h/(1-h))= \alpha + \beta d.$
   1. Find the likelihood equations for estimating $\alpha$ and $\beta$.

   2. Find the Fisher information matrix.

   3. Define the parameter LD50 as the value of $d$ for which $h(d)= 1/2$; express LD50 as a function of $\alpha$ and $\beta$.

   4. Use a Taylor expansion to find large sample confidence limits for LD50.

   5. At each of the doses -3.204, -2.903, -2.602, -2.301 and -2.000 a sample of 40 mice were exposed to antipneumonococcus serum. The numbers surviving were 7, 18, 32, 35, and 38 respectively. Get numerical values for the theory above. You can use glm or get preliminary estimates based on linear regression of the MLE of $h(d_i)$ against dose.

7. Suppose $X_1,\ldots,X_n$ are a sample of size $n$ from the density
   
   $$f_{\alpha,\beta}(x)=\frac{1}{\beta\Gamma(\alpha)}\left(\frac{x}{\beta} \right)^{\alpha-1}\exp(-x/\beta)\,1(x>0).$$
   
   
   In the following question the digamma function $\psi$ is defined by $\psi(\alpha)=\frac{d}{d\alpha}\log(\Gamma(\alpha))$ and the trigamma function $\psi^\prime$ is the derivative of the digamma function. From the identity $\Gamma(\alpha+1)=\alpha\Gamma(\alpha)$ you can deduce recurrence relations for the digamma and trigamma functions.
   1. For $\alpha=\alpha_o$ known find the mle for $\beta$.

   2. When both $\alpha$ and $\beta$ are unknown what equation must be solved to find $\hat\alpha$, the mle of $\alpha$?

   3. Evaluate the Fisher information matrix.

   4. A sample of size 20 is in the file

      ~lockhart/teaching/courses/801/Data/Gamma_Data.

      Use this data in the following questions. First take $\alpha=1$ and find the mle of $\beta$ subject to this restriction.

   5. Now use ${\rm E}(X)=\alpha\beta$ and ${\rm Var}(X)=\alpha\beta^2$ to get method of moments estimates $\tilde\alpha$ and $\tilde\beta$ for the parameters. (This was done in class so I just mean get numbers.)

   6. Do two steps of Newton Raphson to get MLEs.

   7. Use Fisher's scoring idea, which is to replace the second derivative in Newton Raphson with the Fisher information (and then not change it as you run the iteration), to redo the previous question.

   8. Compute standard errors for the MLEs and compare the difference between the estimates in the previous 2 questions to the SEs.

   9. Do a likelihood ratio test of $H_o: \alpha=1$.