|
A underlying premise of stochastic models
is that exact outcomes cannot be predicted with certainty.
Though we are often tempted to express predictions made by a good
model as a point estimate of a parameter value, on average, our
predictions will be wrong by one standard error (SE).
Thus stochastic models require an expression
of this uncertainty, usually as a SE of prediction or estimation.
Useful expressions of uncertainty are not necessarily those with
the smallest SE, but those that you can state with some specified
confidence that they include the 'true' value, e.g., 19 times out
of 20 (alpha=0.05).
Undesirably small SEs can result from badly fit models whose increased
estimation bias results in those
SEs not encompassing the 'true' value of a parameter.
An important goal of goodness-of-fit
testing is to detect such poor and potentially misleading models.
A model's SEs are generally calculated and expressed by a covariance
matrix that assumes asymptotic conditions, i.e., a very large
sample size.
As such, SEs are assumed to apply to a Gaussian (normal) distribution
of uncertainty around a point estimate, in accordance with the central
limit theorem, and have minimal bias.
If such conditions are not met, SEs can be biased and the distribution
of uncertainty can be highly skewed (non-normal).
A covariance matrix can be calculated
by inverting a Hessian matrix calculated using analytical
second partial derivatives of the likelihood function with respect
to the parameters.
Under asymptotic conditions the values of these second partial
derivatives remain constant for all values of a particular parameter,
especially near the maximum-likelihood point parameter estimates.
However, for an analytically calculated Hessian matrix
there is no easy way to assess that the second partial derivatives
are constant throughout that restricted parameter space, and thus
conformance with the central limit theorem cannot be ascertained.
This limitation arises from the analytical derivatives
generally being calculated only for the point values of the maximum-likelihood
estimates.
However, when the covariance matrix
is determined by inverting a Hessian matrix calculated using numerical
second partial derivatives of the likelihood function with respect
to the parameters, it is possible to assess the non-constancy of
the second partial derivatives.
This is achieved by systematically taking small incremental steps
away from the point parameter estimates then approximating the second
partial derivatives using a finite difference calculation.
Differences in values for the second partial derivatives at different
distances from the point parameter values are diagnostic of the degree
of conformance with the asymptotic conditions of the central limit
theorem.
|