Hessian Scaling

The NLMIXED Procedure

Hessian Scaling

The rows and columns of the Hessian matrix can be scaled when you use the trust region, Newton-Raphson, and double dogleg optimization techniques. Each element H_i,j, i,j = 1, ... ,n is divided by the scaling factor d_i d_j, where the scaling vector d = (d₁, ... ,d_n) is iteratively updated in a way specified by the HESCAL=i option, as follows.

i = 0:

No scaling is done (equivalent to d_i=1).

$i \neq 0:$

First iteration and each restart iteration sets:

$d_i^{(0)} = \sqrt{\max(| H^{(0)}_{i,i}|,\epsilon)}$

i = 1:

Refer to Mor $\acute{e}$ (1978):

$d_i^{(k+1)} = \max [ d_i^{(k)},\sqrt{\max(| H^{(k)}_{i,i}|, \epsilon)} ]$

i = 2:

Refer to Dennis, Gay, and Welsch (1981):

$d_i^{(k+1)} = {\rm max} [ .6 d_i^{(k)}, \sqrt{\max(| H^{(k)}_{i,i}|,\epsilon)} ]$

i = 3:

d_i is reset in each iteration:

$d_i^{(k+1)} = \sqrt{\max(| H^{(k)}_{i,i}|,\epsilon)}$

In the preceding equations, $\epsilon$ is the relative machine precision or, equivalently, the largest double precision value that, when added to 1, results in 1.

Chapter Contents
Previous
Next
Top