Example 26.3: Maximum-Likelihood Factor Analysis
This example uses maximum-likelihood factor
analyses for one, two, and three factors.
It is already apparent from the principal factor analysis that
the best number of common factors is almost certainly two.
The one- and three-factor ML solutions reinforce this conclusion
and illustrate some of the numerical problems that can occur.
The following statements produce Output 26.3.1:
proc factor data=SocioEconomics method=ml heywood n=1;
title3 'Maximum-Likelihood Factor Analysis with One Factor';
run;
proc factor data=SocioEconomics method=ml heywood n=2;
title3 'Maximum-Likelihood Factor Analysis with Two Factors';
run;
proc factor data=SocioEconomics method=ml heywood n=3;
title3 'Maximum-Likelihood Factor Analysis with Three Factors';
run;
Output 26.3.1: Maximum-Likelihood Factor Analysis
|
| Maximum-Likelihood Factor Analysis with One Factor |
| The FACTOR Procedure |
| Initial Factor Method: Maximum Likelihood |
| Prior Communality Estimates: SMC |
| Population |
School |
Employment |
Services |
HouseValue |
| 0.96859160 |
0.82228514 |
0.96918082 |
0.78572440 |
0.84701921 |
Preliminary Eigenvalues: Total = 76.1165859 Average = 15.2233172 |
| |
Eigenvalue |
Difference |
Proportion |
Cumulative |
| 1 |
63.7010086 |
50.6462895 |
0.8369 |
0.8369 |
| 2 |
13.0547191 |
12.7270798 |
0.1715 |
1.0084 |
| 3 |
0.3276393 |
0.6749199 |
0.0043 |
1.0127 |
| 4 |
-0.3472805 |
0.2722202 |
-0.0046 |
1.0081 |
| 5 |
-0.6195007 |
|
-0.0081 |
1.0000 |
| 1 factor will be retained by the NFACTOR criterion. |
| Iteration |
Criterion |
Ridge |
Change |
Communalities |
| 1 |
6.5429218 |
0.0000 |
0.1033 |
0.93828 |
0.72227 |
1.00000 |
0.71940 |
0.74371 |
| 2 |
3.1232699 |
0.0000 |
0.7288 |
0.94566 |
0.02380 |
1.00000 |
0.26493 |
0.01487 |
| Convergence criterion satisfied. |
|
|
| Maximum-Likelihood Factor Analysis with One Factor |
| The FACTOR Procedure |
| Initial Factor Method: Maximum Likelihood |
| Significance Tests Based on 12 Observations |
| Test |
DF |
Chi-Square |
Pr > ChiSq |
| H0: No common factors |
10 |
54.2517 |
<.0001 |
| HA: At least one common factor |
|
|
|
| H0: 1 Factor is sufficient |
5 |
24.4656 |
0.0002 |
| HA: More factors are needed |
|
|
|
| Chi-Square without Bartlett's Correction |
34.355969 |
| Akaike's Information Criterion |
24.355969 |
| Schwarz's Bayesian Criterion |
21.931436 |
| Tucker and Lewis's Reliability Coefficient |
0.120231 |
Squared Canonical Correlations |
| Factor1 |
| 1.0000000 |
Eigenvalues of the Weighted Reduced Correlation Matrix: Total = 0 Average = 0 |
| |
Eigenvalue |
Difference |
| 1 |
Infty |
Infty |
| 2 |
1.92716032 |
2.15547340 |
| 3 |
-.22831308 |
0.56464322 |
| 4 |
-.79295630 |
0.11293464 |
| 5 |
-.90589094 |
|
|
|
| Maximum-Likelihood Factor Analysis with One Factor |
| The FACTOR Procedure |
| Initial Factor Method: Maximum Likelihood |
| Factor Pattern |
| |
Factor1 |
| Population |
0.97245 |
| School |
0.15428 |
| Employment |
1.00000 |
| Services |
0.51472 |
| HouseValue |
0.12193 |
| Variance Explained by Each Factor |
| Factor |
Weighted |
Unweighted |
| Factor1 |
17.8010629 |
2.24926004 |
Final Communality Estimates and Variable Weights |
Total Communality: Weighted = 17.801063 Unweighted = 2.249260 |
| Variable |
Communality |
Weight |
| Population |
0.94565561 |
18.4011648 |
| School |
0.02380349 |
1.0243839 |
| Employment |
1.00000000 |
Infty |
| Services |
0.26493499 |
1.3604239 |
| HouseValue |
0.01486595 |
1.0150903 |
|
Output 26.3.1 displays the results of the analysis with one
factor.
The solution on the second iteration
is so close to the optimum that PROC FACTOR cannot find
a better solution, hence you receive this message:
Convergence criterion satisfied.
When this message appears, you should try rerunning
PROC FACTOR with different prior communality estimates
to make sure that the solution is correct.
In this case, other prior estimates lead to the same
solution or possibly to worse local optima, as indicated
by the information criteria or the Chi-square values.
The variable Employment has a communality of 1.0 and,
therefore, an infinite weight that is displayed next to
the final communality estimate as a missing/infinite value.
The first eigenvalue is also infinite.
Infinite values are ignored in computing the total
of the eigenvalues and the total final communality.
Output 26.3.2: Maximum-Likelihood Factor Analysis: Two Factors
|
| Maximum-Likelihood Factor Analysis with Two Factors |
| The FACTOR Procedure |
| Initial Factor Method: Maximum Likelihood |
| Prior Communality Estimates: SMC |
| Population |
School |
Employment |
Services |
HouseValue |
| 0.96859160 |
0.82228514 |
0.96918082 |
0.78572440 |
0.84701921 |
Preliminary Eigenvalues: Total = 76.1165859 Average = 15.2233172 |
| |
Eigenvalue |
Difference |
Proportion |
Cumulative |
| 1 |
63.7010086 |
50.6462895 |
0.8369 |
0.8369 |
| 2 |
13.0547191 |
12.7270798 |
0.1715 |
1.0084 |
| 3 |
0.3276393 |
0.6749199 |
0.0043 |
1.0127 |
| 4 |
-0.3472805 |
0.2722202 |
-0.0046 |
1.0081 |
| 5 |
-0.6195007 |
|
-0.0081 |
1.0000 |
| 2 factors will be retained by the NFACTOR criterion. |
| Iteration |
Criterion |
Ridge |
Change |
Communalities |
| 1 |
0.3431221 |
0.0000 |
0.0471 |
1.00000 |
0.80672 |
0.95058 |
0.79348 |
0.89412 |
| 2 |
0.3072178 |
0.0000 |
0.0307 |
1.00000 |
0.80821 |
0.96023 |
0.81048 |
0.92480 |
| 3 |
0.3067860 |
0.0000 |
0.0063 |
1.00000 |
0.81149 |
0.95948 |
0.81677 |
0.92023 |
| 4 |
0.3067373 |
0.0000 |
0.0022 |
1.00000 |
0.80985 |
0.95963 |
0.81498 |
0.92241 |
| 5 |
0.3067321 |
0.0000 |
0.0007 |
1.00000 |
0.81019 |
0.95955 |
0.81569 |
0.92187 |
| Convergence criterion satisfied. |
|
|
| Maximum-Likelihood Factor Analysis with Two Factors |
| The FACTOR Procedure |
| Initial Factor Method: Maximum Likelihood |
| Significance Tests Based on 12 Observations |
| Test |
DF |
Chi-Square |
Pr > ChiSq |
| H0: No common factors |
10 |
54.2517 |
<.0001 |
| HA: At least one common factor |
|
|
|
| H0: 2 Factors are sufficient |
1 |
2.1982 |
0.1382 |
| HA: More factors are needed |
|
|
|
| Chi-Square without Bartlett's Correction |
3.3740530 |
| Akaike's Information Criterion |
1.3740530 |
| Schwarz's Bayesian Criterion |
0.8891463 |
| Tucker and Lewis's Reliability Coefficient |
0.7292200 |
| Squared Canonical Correlations |
| Factor1 |
Factor2 |
| 1.0000000 |
0.9518891 |
Eigenvalues of the Weighted Reduced Correlation Matrix: Total = 19.7853157 Average = 4.94632893 |
| |
Eigenvalue |
Difference |
Proportion |
Cumulative |
| 1 |
Infty |
Infty |
|
|
| 2 |
19.7853143 |
19.2421292 |
1.0000 |
1.0000 |
| 3 |
0.5431851 |
0.5829564 |
0.0275 |
1.0275 |
| 4 |
-0.0397713 |
0.4636411 |
-0.0020 |
1.0254 |
| 5 |
-0.5034124 |
|
-0.0254 |
1.0000 |
|
|
| Maximum-Likelihood Factor Analysis with Two Factors |
| The FACTOR Procedure |
| Initial Factor Method: Maximum Likelihood |
| Factor Pattern |
| |
Factor1 |
Factor2 |
| Population |
1.00000 |
0.00000 |
| School |
0.00975 |
0.90003 |
| Employment |
0.97245 |
0.11797 |
| Services |
0.43887 |
0.78930 |
| HouseValue |
0.02241 |
0.95989 |
| Variance Explained by Each Factor |
| Factor |
Weighted |
Unweighted |
| Factor1 |
24.4329707 |
2.13886057 |
| Factor2 |
19.7853143 |
2.36835294 |
Final Communality Estimates and Variable Weights |
Total Communality: Weighted = 44.218285 Unweighted = 4.507214 |
| Variable |
Communality |
Weight |
| Population |
1.00000000 |
Infty |
| School |
0.81014489 |
5.2682940 |
| Employment |
0.95957142 |
24.7246669 |
| Services |
0.81560348 |
5.4256462 |
| HouseValue |
0.92189372 |
12.7996793 |
|
Output 26.3.2 displays the results of the analysis using two factors.
The analysis converges without incident.
This time, however, the Population variable is a Heywood case.
Output 26.3.3: Maximum-Likelihood Factor Analysis: Three Factors
|
| Maximum-Likelihood Factor Analysis with Three Factors |
| The FACTOR Procedure |
| Initial Factor Method: Maximum Likelihood |
| Prior Communality Estimates: SMC |
| Population |
School |
Employment |
Services |
HouseValue |
| 0.96859160 |
0.82228514 |
0.96918082 |
0.78572440 |
0.84701921 |
Preliminary Eigenvalues: Total = 76.1165859 Average = 15.2233172 |
| |
Eigenvalue |
Difference |
Proportion |
Cumulative |
| 1 |
63.7010086 |
50.6462895 |
0.8369 |
0.8369 |
| 2 |
13.0547191 |
12.7270798 |
0.1715 |
1.0084 |
| 3 |
0.3276393 |
0.6749199 |
0.0043 |
1.0127 |
| 4 |
-0.3472805 |
0.2722202 |
-0.0046 |
1.0081 |
| 5 |
-0.6195007 |
|
-0.0081 |
1.0000 |
| 3 factors will be retained by the NFACTOR criterion. |
| WARNING: |
Too many factors for a unique solution. |
|
| Iteration |
Criterion |
Ridge |
Change |
Communalities |
| 1 |
0.1798029 |
0.0313 |
0.0501 |
0.96081 |
0.84184 |
1.00000 |
0.80175 |
0.89716 |
| 2 |
0.0016405 |
0.0313 |
0.0678 |
0.98081 |
0.88713 |
1.00000 |
0.79559 |
0.96500 |
| 3 |
0.0000041 |
0.0313 |
0.0094 |
0.98195 |
0.88603 |
1.00000 |
0.80498 |
0.96751 |
| 4 |
0.0000000 |
0.0313 |
0.0006 |
0.98202 |
0.88585 |
1.00000 |
0.80561 |
0.96735 |
| ERROR: Converged, but not to a proper optimum. |
| Try a different 'PRIORS' statement. |
|
|
| Maximum-Likelihood Factor Analysis with Three Factors |
| The FACTOR Procedure |
| Initial Factor Method: Maximum Likelihood |
| Significance Tests Based on 12 Observations |
| Test |
DF |
Chi-Square |
Pr > ChiSq |
| H0: No common factors |
10 |
54.2517 |
<.0001 |
| HA: At least one common factor |
|
|
|
| H0: 3 Factors are sufficient |
-2 |
0.0000 |
. |
| HA: More factors are needed |
|
|
|
| Chi-Square without Bartlett's Correction |
0.0000003 |
| Akaike's Information Criterion |
4.0000003 |
| Schwarz's Bayesian Criterion |
4.9698136 |
| Tucker and Lewis's Reliability Coefficient |
0.0000000 |
| Squared Canonical Correlations |
| Factor1 |
Factor2 |
Factor3 |
| 1.0000000 |
0.9751895 |
0.6894465 |
Eigenvalues of the Weighted Reduced Correlation Matrix: Total = 41.5254193 Average = 10.3813548 |
| |
Eigenvalue |
Difference |
Proportion |
Cumulative |
| 1 |
Infty |
Infty |
|
|
| 2 |
39.3054826 |
37.0854258 |
0.9465 |
0.9465 |
| 3 |
2.2200568 |
2.2199693 |
0.0535 |
1.0000 |
| 4 |
0.0000875 |
0.0002949 |
0.0000 |
1.0000 |
| 5 |
-0.0002075 |
|
-0.0000 |
1.0000 |
|
|
| Maximum-Likelihood Factor Analysis with Three Factors |
| The FACTOR Procedure |
| Initial Factor Method: Maximum Likelihood |
| Factor Pattern |
| |
Factor1 |
Factor2 |
Factor3 |
| Population |
0.97245 |
-0.11233 |
-0.15409 |
| School |
0.15428 |
0.89108 |
0.26083 |
| Employment |
1.00000 |
0.00000 |
0.00000 |
| Services |
0.51472 |
0.72416 |
-0.12766 |
| HouseValue |
0.12193 |
0.97227 |
-0.08473 |
| Variance Explained by Each Factor |
| Factor |
Weighted |
Unweighted |
| Factor1 |
54.6115241 |
2.24926004 |
| Factor2 |
39.3054826 |
2.27634375 |
| Factor3 |
2.2200568 |
0.11525433 |
Final Communality Estimates and Variable Weights |
Total Communality: Weighted = 96.137063 Unweighted = 4.640858 |
| Variable |
Communality |
Weight |
| Population |
0.98201660 |
55.6066901 |
| School |
0.88585165 |
8.7607194 |
| Employment |
1.00000000 |
Infty |
| Services |
0.80564301 |
5.1444261 |
| HouseValue |
0.96734687 |
30.6251078 |
|
The three-factor analysis displayed in Output 26.3.3 generates this message:
WARNING: Too many factors for a unique solution.
The number of parameters in the model exceeds the
number of elements in the correlation matrix from
which they can be estimated, so an infinite number
of different perfect solutions can be obtained.
The Criterion approaches zero at an improper
optimum, as indicated by this message:
Converged, but not to a proper optimum.
The degrees of freedom for the chi-square test are -2,
so a probability level cannot be computed for three factors.
Note also that the variable Employment is a Heywood case again.
The probability levels for the chi-square test are 0.0001
for the hypothesis of no common factors, 0.0002 for
one common factor, and 0.1382 for two common factors.
Therefore, the two-factor model seems
to be an adequate representation.
Akaike's information criterion and Schwarz's Bayesian criterion
attain their minimum values at two common factors, so there is
little doubt that two factors are appropriate for these data.
Copyright © 1999 by SAS Institute Inc., Cary, NC, USA. All rights reserved.