Robust Regression Examples

Examples Combining Robust Residuals and Robust Distances

This section is based entirely on Rousseeuw and Van Zomeren (1990). Observations x_i, which are far away from most of the other observations, are called leverage points. One classical method inspects the Mahalanobis distances MD_i to find outliers x_i:

$MD_i = \sqrt{(x_i - \mu) C^{-1}(x_i - \mu)^T}$

where C is the classical sample covariance matrix. Note that the MVE subroutine prints the classical Mahalanobis distances MD_i together with the robust distances RD_i. In classical linear regression, the diagonal elements h_ii of the hat matrix

H = X(X^TX)^-1X^T

are used to identify leverage points. Rousseeuw and Van Zomeren (1990) report the following monotone relationship between the h_ii and MD_i

h_ii = [((MD_i)²)/(N-1)] + [1/n]

and point out that neither the MD_i nor the h_ii are entirely safe for detecting leverage points reliably. Multiple outliers do not necessarily have large MD_i values because of the masking effect.

The definition of a leverage point is, therefore, based entirely on the outlyingness of x_i and is not related to the response value y_i. By including the y_i value in the definition, Rousseeuw and Van Zomeren (1990) distinguish between the following:

Good leverage points are points (x_i,y_i) that are close to the regression plane; that is, good leverage points improve the precision of the regression coefficients.
Bad leverage points are points (x_i,y_i) that are far from the regression plane; that is, bad leverage points reduce the precision of the regression coefficients.

Rousseeuw and Van Zomeren (1990) propose to plot the standardized residuals of robust regression (LMS or LTS) versus the robust distances RD_i obtained from MVE. Two horizontal lines corresponding to residual values of +2.5 and -2.5 are useful to distinguish between small and large residuals, and one vertical line corresponding to the $\sqrt{\chi^2_{n,.975}}$ is used to distinguish between small and large distances.

Example 9.6: Hawkins-Bradu-Kass Data

Example 9.7: Stackloss Data

Chapter Contents
Previous
Next
Top