Factor Analysis example
The data for this example are in Table 9.12 in Johnson and Wichern. They consist of 3 measurements on the sales performance of 50 salespeople for a large firm and 4 test scores.
The data begin:
I used SAS to carry out Factor Analysis of these variables several different ways.
SAS code for first run, requesting principal components factor analysis, no rotation, all output printed and allowing SAS to select m the number of factors:
data sales; infile "T9-12.DAT"; input growth profit new create mech abst math; proc factor method=prin rotate=none all; run;The (edited) output is
Initial Factor Method: Principal Components
Inverse Correlation Matrix
GROWTH PROFIT NEW CREATE MECH ABST MATH
GROWTH 35.1412 -8.2675 15.7402 -13.9069 -1.5588 -13.8694 -23.7101
PROFIT -8.2675 31.6239 -2.7059 1.4128 -8.1820 6.5890 -19.5028
NEW 15.7402 -2.7059 21.6934 -13.2468 -0.1384 -10.4680 -19.0616
CREATE -13.9069 1.4128 -13.2468 10.5387 -0.1251 7.6430 14.2502
MECH -1.5588 -8.1820 -0.1384 -0.1251 4.5610 -0.8843 7.2216
ABST -13.8694 6.5890 -10.4680 7.6430 -0.8843 8.6865 7.9984
MATH -23.7101 -19.5028 -19.0616 14.2502 7.2216 7.9984 43.0948
Partial Correlations Controlling all other Variables
GROWTH PROFIT NEW CREATE MECH ABST MATH
GROWTH 1.00000 0.24800 -0.57008 0.72265 0.12312 0.79383 0.60927
PROFIT 0.24800 1.00000 0.10331 -0.07739 0.68127 -0.39755 0.52829
NEW -0.57008 0.10331 1.00000 0.87610 0.01392 0.76257 0.62342
CREATE 0.72265 -0.07739 0.87610 1.00000 0.01805 -0.79882 -0.66868
MECH 0.12312 0.68127 0.01392 0.01805 1.00000 0.14049 -0.51510
ABST 0.79383 -0.39755 0.76257 -0.79882 0.14049 1.00000 -0.41340
MATH 0.60927 0.52829 0.62342 -0.66868 -0.51510 -0.41340 1.00000
Kaiser's Measure of Sampling Adequacy: Over-all MSA = 0.61609232
GROWTH PROFIT NEW CREATE MECH ABST MATH
0.662688 0.782579 0.629694 0.409556 0.749798 0.417191 0.631877
Prior Communality Estimates: ONE
Eigenvalues of the Correlation Matrix: Total = 7 Average = 1
1 2 3 4 5 6 7
Eigenvalue 5.0346 0.9335 0.4979 0.4212 0.0810 0.0203 0.0113
Difference 4.1011 0.4356 0.0767 0.3402 0.0607 0.0090
Proportion 0.7192 0.1334 0.0711 0.0602 0.0116 0.0029 0.0016
Cumulative 0.7192 0.8526 0.9237 0.9839 0.9955 0.9984 1.0000
1 factors will be retained by the MINEIGEN criterion.
Eigenvectors
GROWTH 0.43367
PROFIT 0.42021
NEW 0.42105
CREATE 0.29429
MECH 0.34909
ABST 0.28917
MATH 0.40740
Factor Pattern
FACTOR1
GROWTH 0.97307
PROFIT 0.94287
NEW 0.94475
CREATE 0.66032
MECH 0.78329
ABST 0.64883
MATH 0.91413
Variance explained by each factor
FACTOR1
5.034598
Final Communality Estimates: Total = 5.034598
GROWTH PROFIT NEW CREATE MECH ABST MATH
0.946864 0.889006 0.892553 0.436018 0.613542 0.420981 0.835633
Scoring Coefficients Estimated by Regression
Squared Multiple Correlations of the Variables with each Factor
FACTOR1
1.000000
Standardized Scoring Coefficients
FACTOR1
GROWTH 0.19328
PROFIT 0.18728
NEW 0.18765
CREATE 0.13116
MECH 0.15558
ABST 0.12887
MATH 0.18157
Residual Correlations With Uniqueness on the Diagonal
GROWTH PROFIT NEW CREATE MECH ABST MATH
GROWTH 0.05314 0.00860 -0.03531 -0.07050 -0.05412 0.04305 0.03780
PROFIT 0.00860 0.11099 -0.04825 -0.08109 0.00737 -0.14638 0.08239
NEW -0.03531 -0.04825 0.10745 0.07653 -0.10254 0.02811 -0.01106
CREATE -0.07050 -0.08109 0.07653 0.56398 0.07352 -0.28153 -0.19098
MECH -0.05412 0.00737 -0.10254 0.07352 0.38646 -0.12227 -0.14147
ABST 0.04305 -0.14638 0.02811 -0.28153 -0.12227 0.57902 -0.02674
MATH 0.03780 0.08239 -0.01106 -0.19098 -0.14147 -0.02674 0.16437
Root Mean Square Off-diagonal Residuals: Over-all = 0.10322669
GROWTH PROFIT NEW CREATE MECH ABST MATH
0.045646 0.078788 0.058961 0.151951 0.094753 0.140826 0.104516
Partial Correlations Controlling Factors
GROWTH PROFIT NEW CREATE
MECH ABST MATH
GROWTH 1.00000 0.11194 -0.46725 -0.40724 -0.37768 0.24544 0.40447
PROFIT 0.11194 1.00000 -0.44187 -0.32409 0.03558 -0.57739 0.60998
NEW -0.46725 -0.44187 1.00000 0.31088 -0.50322 0.11268 -0.08320
CREATE -0.40724 -0.32409 0.31088 1.00000 0.15747 -0.49265 -0.62725
MECH -0.37768 0.03558 -0.50322 0.15747 1.00000 -0.25848 -0.56133
ABST 0.24544 -0.57739 0.11268 -0.49265 -0.25848 1.00000 -0.08669
MATH 0.40447 0.60998 -0.08320 -0.62725 -0.56133 -0.08669 1.00000
Root Mean Square Off-diagonal Partials: Over-all = 0.38975152
GROWTH PROFIT NEW CREATE MECH ABST MATH
0.356644 0.412217 0.361262 0.414017 0.366023 0.347216 0.458011
The procedure selects only one factor on which all the variables are fairly
highly loaded. The factor is the first principal component of the
correlation matrix.
Now I try insisting on two factors, using varimax rotation and then plotting the factor loadings before and after rotation.
proc factor m=prin nfactor=2 rotate=v preplot plot all;The output
2 factors will be retained by the NFACTOR criterion.
Initial Factor Method: Principal Components
Factor Pattern
FACTOR1 FACTOR2
GROWTH 0.97307 -0.10798
PROFIT 0.94287 0.02830
NEW 0.94475 0.00889
CREATE 0.66032 0.64581
MECH 0.78329 0.28497
ABST 0.64883 -0.62066
MATH 0.91413 -0.19359
Variance explained by each factor
FACTOR1 FACTOR2
5.034598 0.933516
Final Communality Estimates: Total = 5.968114
GROWTH PROFIT NEW CREATE MECH ABST MATH
0.958522 0.889807 0.892632 0.853094 0.694751 0.806195 0.873111
Residual Correlations With Uniqueness on the Diagonal
GROWTH PROFIT NEW CREATE MECH ABST MATH
GROWTH 0.04148 0.01165 -0.03435 -0.00077 -0.02335 -0.02397 0.01690
PROFIT 0.01165 0.11019 -0.04851 -0.09936 -0.00070 -0.12881 0.08787
NEW -0.03435 -0.04851 0.10737 0.07079 -0.10508 0.03362 -0.00934
CREATE -0.00077 -0.09936 0.07079 0.14691 -0.11052 0.11930 -0.06595
MECH -0.02335 -0.00070 -0.10508 -0.11052 0.30525 0.05460 -0.08631
ABST -0.02397 -0.12881 0.03362 0.11930 0.05460 0.19380 -0.14690
MATH 0.01690 0.08787 -0.00934 -0.06595 -0.08631 -0.14690 0.12689
Root Mean Square Off-diagonal Residuals: Over-all = 0.07538345
GROWTH PROFIT NEW CREATE MECH ABST MATH
0.021296 0.078182 0.058881 0.087256 0.075533 0.097545 0.083137
Root Mean Square Off-diagonal Partials: Over-all = 0.50908239
GROWTH PROFIT NEW CREATE MECH ABST MATH
0.277921 0.601054 0.443291 0.567641 0.386234 0.623235 0.565089
Plot of Factor Pattern for FACTOR1 and FACTOR2
FACTOR1
1
A B
G .9
.8 E
.7
F D
.6
.5
.4
.3
.2
F
.1 A
C
-1 -.9-.8-.7-.6-.5-.4-.3-.2-.1 0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0T
O
-.1 R
2
-.2
GROWTH=A PROFIT=B NEW=B CREATE=D MECH=E ABST=F MATH=G
Rotation Method: Varimax
Orthogonal Transformation Matrix
1 2
1 0.73145 0.68189
2 -0.68189 0.73145
Rotated Factor Pattern
FACTOR1 FACTOR2
GROWTH 0.78538 0.58455
PROFIT 0.67037 0.66364
NEW 0.68498 0.65072
CREATE 0.04261 0.92265
MECH 0.37862 0.74256
ABST 0.89781 -0.01155
MATH 0.80065 0.48174
Variance explained by each factor
FACTOR1 FACTOR2
3.127683 2.840431
Final Communality Estimates: Total = 5.968114
GROWTH PROFIT NEW CREATE MECH ABST MATH
0.958522 0.889807 0.892632 0.853094 0.694751 0.806195 0.873111
Scoring Coefficients Estimated by Regression
Squared Multiple Correlations of the Variables with each Factor
FACTOR1 FACTOR2
1.000000 1.000000
Standardized Scoring Coefficients
FACTOR1 FACTOR2
GROWTH 0.22024 0.04719
PROFIT 0.11632 0.14988
NEW 0.13076 0.13492
CREATE -0.37581 0.59546
MECH -0.09436 0.32938
ABST 0.54763 -0.39843
MATH 0.27422 -0.02788
Plot of Factor Pattern for FACTOR1 and FACTOR2
FACTOR1
1
F9
.8 G A
.7 C
B
.6
.5
.4 E
.3
.2
F
.1 A
D C
-1 -.9-.8-.7-.6-.5-.4-.3-.2-.1 0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0T
O
-.1 R
2
GROWTH=A PROFIT=B NEW=C CREATE=D MECH=E ABST=F MATH=G
With principal components factor analysis fitting a second factor does
not change the first factor (before rotation). Creativity and
abstract reasoning are loaded on the second factor with opposite signs
which would appear to represent a difference between people on a dimension of
creativity as opposed to abstract reasoning.
After rotation everything except creativity is loaded on factor 1, though Mechanical reasoning has a rather smaller loading. Abstraction is not loaded on factor 2.
Now I tried iterated principal factor analysis with varimax rotation. The option heywood permits iteration to continue if the estimated uniqueness of a variable drops below 0.
proc factor m=prinit nfactor=2 rotate=v preplot plot all heywood; run;
Initial Factor Method: Iterated Principal Factor Analysis
Prior Communality Estimates: ONE
2 factors will be retained by the NFACTOR criterion.
Iter Change Communalities
1 0.305249 0.95852 0.88981 0.89263 0.85309 0.69475 0.80620 0.87311
2 0.122396 0.96840 0.87423 0.87908 0.80520 0.60002 0.68380 0.85064
3 0.090342 0.98057 0.87429 0.88013 0.79735 0.57498 0.59346 0.85545
4 0.063814 0.98875 0.87617 0.88190 0.80757 0.56804 0.52964 0.86690
5 0.040187 0.99332 0.87790 0.88272 0.82483 0.56497 0.48946 0.87810
6 0.022953 0.99547 0.87917 0.88280 0.84403 0.56254 0.46650 0.88668
7 0.019141 0.99628 0.88000 0.88249 0.86317 0.56024 0.45397 0.89244
8 0.018399 0.99646 0.88051 0.88204 0.88157 0.55808 0.44705 0.89597
9 0.017475 0.99640 0.88084 0.88156 0.89905 0.55609 0.44302 0.89798
10 0.016551 0.99629 0.88107 0.88109 0.91560 0.55429 0.44044 0.89906
11 0.015686 0.99618 0.88126 0.88065 0.93128 0.55265 0.43862 0.89958
12 0.014897 0.99611 0.88142 0.88023 0.94618 0.55115 0.43722 0.89977
13 0.014181 0.99605 0.88158 0.87984 0.96036 0.54978 0.43605 0.89978
14 0.013532 0.99601 0.88172 0.87946 0.97389 0.54852 0.43503 0.89970
15 0.012944 0.99599 0.88186 0.87910 0.98684 0.54736 0.43411 0.89956
16 0.012408 0.99597 0.88200 0.87875 0.99925 0.54628 0.43327 0.89939
17 0.001004 0.99596 0.88213 0.87842 1.00000 0.54528 0.43250 0.89922
18 0.000295 0.99600 0.88224 0.87833 1.00000 0.54499 0.43220 0.89918
Convergence criterion satisfied.
Eigenvalues of the Reduced Correlation Matrix:
Total = 5.63350769 Average = 0.80478681
1 2 3 4 5 6 7
Eigenvalue 4.8786 0.7663 0.2047 0.0838 0.0051 -0.1182 -0.1868
Difference 4.1123 0.5616 0.1209 0.0787 0.1233 0.0686
Proportion 0.8660 0.1360 0.0363 0.0149 0.0009 -0.0210 -0.0332
Cumulative 0.8660 1.0020 1.0384 1.0532 1.0541 1.0332 1.0000
Eigenvectors
1 2
GROWTH 0.44687 -0.16852
PROFIT 0.42379 -0.08882
NEW 0.42404 0.03803
CREATE 0.30550 0.85228
MECH 0.32820 0.15943
ABST 0.26439 -0.34496
MATH 0.41224 -0.30243
Factor Pattern
FACTOR1 FACTOR2
GROWTH 0.98704 -0.14753
PROFIT 0.93605 -0.07775
NEW 0.93660 0.03329
CREATE 0.67478 0.74609
MECH 0.72492 0.13956
ABST 0.58396 -0.30198
MATH 0.91054 -0.26475
Variance explained by each factor
FACTOR1 FACTOR2
4.878597 0.766328
Final Communality Estimates: Total = 5.644925
GROWTH PROFIT NEW CREATE MECH ABST MATH
0.996003 0.882237 0.878330 1.011982 0.544987 0.432204 0.899183
Residual Correlations With Uniqueness on the Diagonal
GROWTH 0.00400 -0.00931 -0.03554 0.01607 0.01314 0.05347 -0.01049
PROFIT -0.00931 0.11776 -0.03159 -0.03212 0.07820 -0.10471 0.07140
NEW -0.03554 -0.03159 0.12167 0.04352 -0.04614 0.10420 0.00857
CREATE 0.01607 -0.03212 0.04352 -0.01198 -0.00255 -0.02184 -0.00425
MECH 0.01314 0.07820 -0.04614 -0.00255 0.45501 0.00477 -0.04857
ABST 0.05347 -0.10471 0.10420 -0.02184 0.00477 0.56780 -0.04530
MATH -0.01049 0.07140 0.00857 -0.00425 -0.04857 -0.04530 0.10082
Root Mean Square Off-diagonal Residuals: Over-all = 0.04822817
GROWTH PROFIT NEW CREATE MECH ABST MATH
0.028135 0.063630 0.053566 0.024783 0.042435 0.067370 0.040229
Partial Correlations Controlling Factors
GROWTH PROFIT NEW CREATE MECH ABST MATH
GROWTH 1.00000 -0.42908 -1.61171 0.00000 0.30811 1.12224 -0.52229
PROFIT -0.42908 1.00000 -0.26395 0.00000 0.33781 -0.40493 0.65524
NEW -1.61171 -0.26395 1.00000 0.00000 -0.19609 0.39645 0.07734
CREATE 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000
MECH 0.30811 0.33781 -0.19609 0.00000 1.00000 0.00938 -0.22677
ABST 1.12224 -0.40493 0.39645 0.00000 0.00938 1.00000 -0.18934
MATH -0.52229 0.65524 0.07734 0.00000 -0.22677 -0.18934 1.00000
Root Mean Square Off-diagonal Partials: Over-all = 0.51059853
GROWTH PROFIT NEW CREATE MECH ABST MATH
0.857210 0.400252 0.691481 0.000000 0.223239 0.519055 0.364095
Plot of Factor Pattern for FACTOR1 and FACTOR2
FACTOR1
A 1
B C
G .9
.8
.7 E
D
F .6
.5
.4
.3
.2
F
.1 A
C
-1 -.9-.8-.7-.6-.5-.4-.3-.2-.1 0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0T
O
-.1 R
2
GROWTH=A PROFIT=B NEW=C CREATE=D MECH=E ABST=F MATH=G
Rotation Method: Varimax
Orthogonal Transformation Matrix
1 2
1 0.84716 0.53133
2 -0.53133 0.84716
Rotated Factor Pattern
FACTOR1 FACTOR2
GROWTH 0.91457 0.39946
PROFIT 0.83430 0.43149
NEW 0.77577 0.52585
CREATE 0.17523 0.99059
MECH 0.53997 0.50340
ABST 0.65516 0.05445
MATH 0.91205 0.25951
Variance explained by each factor
FACTOR1 FACTOR2
3.717650 1.927275
Final Communality Estimates: Total = 5.644925
GROWTH PROFIT NEW CREATE MECH ABST MATH
0.996003 0.882237 0.878330 1.011982 0.544987 0.432204 0.899183
Scoring Coefficients Estimated by Regression
Squared Multiple Correlations of the Variables with each Factor
FACTOR1 FACTOR2
1.298471 1.311894
Standardized Scoring Coefficients
FACTOR1 FACTOR2
GROWTH 3.46196 -2.72182
PROFIT -0.91749 1.49807
NEW 2.32759 -2.18108
CREATE -2.03309 2.57935
MECH 0.08882 -0.22785
ABST -1.46013 1.47280
MATH -1.80143 1.46081
Rotation Method: Varimax
Plot of Factor Pattern for FACTOR1 and FACTOR2
FACTOR1
1
.9 G A
B
.8 C
.7
F
.6
E
.5
.4
.3
.2 D
F
.1 A
C
-1 -.9-.8-.7-.6-.5-.4-.3-.2-.1 0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0T
O
-.1 R
2
GROWTH=A PROFIT=B NEW=C CREATE=D MECH=E ABST=F MATH=G
The rotated factor loadings seem rather similar here. There seems to be
a distinct latent variable which fully explains creativity and is at
play in determining other variables a bit. There also seems to be a
variable on which every variable except creativity is highly loaded.
I am not really sure how to interpret this variable.
Now I tried maximum likelihood.
proc factor m=ml nfactor=2 rotate=v preplot plot all heywood; run;The output is
Initial Factor Method: Maximum Likelihood
Prior Communality Estimates: SMC
GROWTH PROFIT NEW CREATE MECH ABST MATH
0.971543 0.968378 0.953903 0.905112 0.780748 0.884878 0.976795
Preliminary Eigenvalues: Total = 148.339548 Average = 21.191364
1 2 3 4 5 6 7
Eigenvalue 130.3121 9.4410 6.5497 2.5446 0.7601 -0.5331 -0.7349
Difference 120.8711 2.8913 4.0051 1.7845 1.2932 0.2018
Proportion 0.8785 0.0636 0.0442 0.0172 0.0051 -0.0036 -0.0050
Cumulative 0.8785 0.9421 0.9863 1.0034 1.0085 1.0050 1.0000
2 factors will be retained by the NFACTOR criterion.
Iter Criterion Ridge Change Communalities
1 4.18397 0.000 0.09489 0.96827 0.95946 0.92988 1.00000
0.73649 0.79560 0.96985
2 2.64993 0.000 0.37880 0.94441 0.91603 0.87751 1.00000
0.54254 0.41681 0.96161
3 2.63510 0.000 0.01113 0.93584 0.92687 0.87771 1.00000
0.53141 0.41206 0.96421
4 2.63204 0.000 0.01336 0.93342 0.92724 0.87616 1.00000
0.53049 0.39870 0.96884
5 2.63142 0.000 0.00620 0.93205 0.92916 0.87683 1.00000
0.52684 0.39250 0.96942
6 2.63130 0.000 0.00210 0.93149 0.92913 0.87648 1.00000
0.52659 0.39040 0.97035
7 2.63128 0.000 0.00125 0.93123 0.92951 0.87666 1.00000
0.52576 0.38915 0.97042
8 2.63127 0.000 0.00035 0.93112 0.92948 0.87658 1.00000
0.52575 0.38880 0.97061
Convergence criterion satisfied.
Significance tests based on 50 observations:
Test of H0: No common factors.
vs HA: At least one common factor.
Chi-square = 499.661 df = 21 Prob>chi**2 = 0.0001
Test of H0: 2 Factors are sufficient.
vs HA: More factors are needed.
Chi-square = 117.092 df = 8 Prob>chi**2 = 0.0001
Chi-square without Bartlett's correction = 128.93234776
Akaike's Information Criterion = 112.93234776
Schwarz's Bayesian Criterion = 97.636163717
Tucker and Lewis's Reliability Coefficient = 0.4017366379
Squared Canonical Correlations
FACTOR1 FACTOR2
1.000000 0.980050
Eigenvectors
1 2
GROWTH 0.57204 0.78496
PROFIT 0.03116 0.20891
NEW 0.41606 -0.17154
CREATE 0.70619 -0.54400
MECH 0.00000 -0.06982
ABST 0.00000 -0.03499
MATH 0.00000 0.09345
Eigenvalues of the Weighted Reduced Correlation Matrix:
Total = 49.1264918 Average = 8.18774864
1 2 3 4 5 6 7
Eigenvalue . 49.1265 1.1504 0.7153 -0.2018 -0.7704 -0.8935
Difference . 47.9761 0.4351 0.9171 0.5686 0.1231
Proportion . 1.0000 0.0234 0.0146 -0.0041 -0.0157 -0.0182
Cumulative . 1.0000 1.0234 1.0380 1.0339 1.0182 1.0000
Factor Pattern
FACTOR1 FACTOR2
GROWTH 0.57204 0.77709
PROFIT 0.54151 0.79768
NEW 0.70036 0.62138
CREATE 1.00000 -0.00000
MECH 0.59074 0.42024
ABST 0.14691 0.60579
MATH 0.41264 0.89462
Variance explained by each factor
FACTOR1 FACTOR2
Weighted 19.448504 49.126492
Unweighted 2.651787 2.970211
Final Communality Estimates and Variable Weights
Total Communality: Weighted = 68.574996 Unweighted = 5.621998
GROWTH PROFIT NEW CREATE
Communality 0.931096 0.929532 0.876616 1.000000
Weight 14.518860 14.180085 8.102194 .
MECH ABST MATH
Communality 0.525572 0.388566 0.970616
Weight 2.108574 1.636117 34.029166
Residual Correlations With Uniqueness on the Diagonal
GROWTH PROFIT NEW CREATE MECH ABST MATH
GROWTH 0.06890 -0.00356 0.00050 0.00000 0.04359 0.11962 -0.00393
PROFIT -0.00356 0.07047 -0.03239 0.00000 0.09080 -0.09740 0.00722
NEW 0.00050 -0.03239 0.12338 0.00000 -0.03739 0.16178 0.00768
CREATE 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000
MECH 0.04359 0.09080 -0.03739 0.00000 0.47443 0.04459 -0.04516
ABST 0.11962 -0.09740 0.16178 0.00000 0.04459 0.61143 -0.03620
MATH -0.00393 0.00722 0.00768 0.00000 -0.04516 -0.03620 0.02938
Root Mean Square Off-diagonal Residuals: Over-all = 0.05691855
GROWTH PROFIT NEW CREATE MECH ABST MATH
0.052019 0.056043 0.069134 0.000000 0.050941 0.094219 0.024073
Root Mean Square Off-diagonal Partials: Over-all = 0.27892670
GROWTH PROFIT NEW CREATE MECH ABST MATH
0.260769 0.320222 0.290900 0.000000 0.283366 0.405477 0.211487
Plot of Factor Pattern for FACTOR1 and FACTOR2
FACTOR1
D
.9
.8
.7 C
.6 E
A
.5
.4 G
.3
.2
F F
.1 A
C
-1 -.9-.8-.7-.6-.5-.4-.3-.2-.1 0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0T
O
-.1 R
2
GROWTH=A PROFIT=A NEW=C CREATE=D MECH=E ABST=F MATH=G
Rotation Method: Varimax
Orthogonal Transformation Matrix
1 2
1 0.14646 0.98922
2 0.98922 -0.14646
Rotated Factor Pattern
FACTOR1 FACTOR2
GROWTH 0.85249 0.45205
PROFIT 0.86839 0.41884
NEW 0.71725 0.60180
CREATE 0.14646 0.98922
MECH 0.50223 0.52282
ABST 0.62078 0.05660
MATH 0.94541 0.27716
Variance explained by each factor
FACTOR1 FACTOR2
Weighted 56.990420 11.584576
Unweighted 3.548142 2.073856
Final Communality Estimates and Variable Weights
Total Communality: Weighted = 68.574996 Unweighted = 5.621998
GROWTH PROFIT NEW CREATE
Communality 0.931096 0.929532 0.876616 1.000000
Weight 14.518860 14.180085 8.102194 .
MECH ABST MATH
Communality 0.525572 0.388566 0.970616
Weight 2.108574 1.636117 34.029166
Scoring Coefficients Estimated by Regression
Squared Multiple Correlations of the Variables with each Factor
FACTOR1 FACTOR2
0.980478 0.999572
Standardized Scoring Coefficients
FACTOR1 FACTOR2
GROWTH 0.22265 -0.03297
PROFIT 0.22322 -0.03305
NEW 0.09935 -0.01471
CREATE -0.43247 1.07493
MECH 0.01749 -0.00259
ABST 0.01956 -0.00290
MATH 0.60078 -0.08895
Plot of Factor Pattern for FACTOR1 and FACTOR2
FACTOR1
1
G
.9
BA
.8
.7 C
.6F
.5 E
.4
.3
.2
D F
.1 A
C
-1 -.9-.8-.7-.6-.5-.4-.3-.2-.1 0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0T
O
-.1 R
2
GROWTH=A PROFIT=B NEW=C CREATE=D MECH=E ABST=F MATH=G
The pattern is the same as for the other methods. Notice however, that this
procedure factors S not R. This explains the totally different
eigenvalues and so on.
Finally I ran proc glm regressing the sales figures on the psychological test scores.
proc glm ; model growth profit new = create mech abst math; manova h=_all_ /printh printe; run;The output is:
General Linear Models Procedure
Dependent Variable: GROWTH
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 4 2520.49121 630.12280 241.32 0.0001
Error 45 117.50399 2.61120
Corrected Total 49 2637.99520
R-Square C.V. Root MSE GROWTH Mean
0.955457 1.634952 1.61592 98.8360
Source DF Type III SS Mean Square F Value Pr > F
CREATE 1 61.431768 61.431768 23.53 0.0001
MECH 1 25.570007 25.570007 9.79 0.0031
ABST 1 92.585812 92.585812 35.46 0.0001
MATH 1 548.009560 548.009560 209.87 0.0001
T for H0: Pr > |T| Std Error of
Parameter Estimate Parameter=0 Estimate
INTERCEPT 68.95355221 51.00 0.0001 1.35196005
CREATE 0.35938316 4.85 0.0001 0.07409369
MECH 0.30033781 3.13 0.0031 0.09597644
ABST 0.79574727 5.95 0.0001 0.13363585
MATH 0.44315482 14.49 0.0001 0.03059014
Dependent Variable: PROFIT
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 4 4852.96681 1213.24170 321.87 0.0001
Error 45 169.61899 3.76931
Corrected Total 49 5022.58580
R-Square C.V. Root MSE PROFIT Mean
0.966229 1.820892 1.94147 106.622
Source DF Type III SS Mean Square F Value Pr > F
CREATE 1 6.79727 6.79727 1.80 0.1860
MECH 1 213.27427 213.27427 56.58 0.0001
ABST 1 48.81780 48.81780 12.95 0.0008
MATH 1 1764.23024 1764.23024 468.05 0.0001
T for H0: Pr > |T| Std Error of
Parameter Estimate Parameter=0 Estimate
INTERCEPT 75.41979427 46.43 0.0001 1.62433197
CREATE 0.11954420 1.34 0.1860 0.08902094
MECH 0.86738881 7.52 0.0001 0.11531228
ABST -0.57781927 -3.60 0.0008 0.16055873
MATH 0.79513166 21.63 0.0001 0.03675297
Dependent Variable: NEW
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 4 1013.57078 253.39270 153.11 0.0001
Error 45 74.47422 1.65498
Corrected Total 49 1088.04500
R-Square C.V. Root MSE NEW Mean
0.931552 1.251300 1.28646 102.810
Source DF Type III SS Mean Square F Value Pr > F
CREATE 1 153.408261 153.408261 92.69 0.0001
MECH 1 1.846673 1.846673 1.12 0.2965
ABST 1 63.380815 63.380815 38.30 0.0001
MATH 1 150.951693 150.951693 91.21 0.0001
T for H0: Pr > |T| Std Error of
Parameter Estimate Parameter=0 Estimate
INTERCEPT 83.70817574 77.77 0.0001 1.07631783
CREATE 0.56791793 9.63 0.0001 0.05898722
MECH -0.08071229 -1.06 0.2965 0.07640844
ABST 0.65838791 6.19 0.0001 0.10638972
MATH 0.23258431 9.55 0.0001 0.02435332
Multivariate Analysis of Variance
E = Error SS&CP Matrix
GROWTH PROFIT NEW
GROWTH 117.50398618 32.670138508 -52.8579666
PROFIT 32.670138508 169.61898601 -5.37650348
NEW -52.8579666 -5.37650348 74.474217358
Partial Correlation Coefficients from the Error SS&CP Matrix / Prob > |r|
DF = 45 GROWTH PROFIT NEW
GROWTH 1.000000 0.231413 -0.565043
0.0001 0.1218 0.0001
PROFIT 0.231413 1.000000 -0.047837
0.1218 0.0001 0.7522
NEW -0.565043 -0.047837 1.000000
0.0001 0.7522 0.0001
H = Type III SS&CP Matrix for CREATE
GROWTH PROFIT NEW
GROWTH 61.431768316 20.434489514 97.078013797
PROFIT 20.434489514 6.7972707468 32.291755705
NEW 97.078013797 32.291755705 153.40826125
Characteristic Roots and Vectors of: E Inverse * H, where
H = Type III SS&CP Matrix for CREATE E = Error SS&CP Matrix
Characteristic Percent Characteristic Vector V'EV=1
Root
GROWTH PROFIT NEW
5.55606041 100.00 0.09060203 -0.00667035 0.13437927
0.00000000 0.00 -0.02427392 0.07898107 -0.00126443
0.00000000 0.00 0.06725495 0.00372078 -0.04334269
Manova Test Criteria and Exact F Statistics for
the Hypothesis of no Overall CREATE Effect
H = Type III SS&CP Matrix for CREATE E = Error SS&CP Matrix
S=1 M=0.5 N=20.5
Statistic Value F Num DF Den DF Pr > F
Wilks' Lambda 0.15253063 79.6369 3 43 0.0001
Pillai's Trace 0.84746937 79.6369 3 43 0.0001
Hotelling-Lawley Trace 5.55606041 79.6369 3 43 0.0001
Roy's Greatest Root 5.55606041 79.6369 3 43 0.0001
H = Type III SS&CP Matrix for MECH
GROWTH PROFIT NEW
GROWTH 25.570006621 73.847304527 -6.871641257
PROFIT 73.847304527 213.27426569 -19.84560239
NEW -6.871641257 -19.84560239 1.8466734979
Manova Test Criteria and Exact F Statistics for
the Hypothesis of no Overall MECH Effect
S=1 M=0.5 N=20.5
Statistic Value F Num DF Den DF Pr > F
Wilks' Lambda 0.43419910 18.6776 3 43 0.0001
Pillai's Trace 0.56580090 18.6776 3 43 0.0001
Hotelling-Lawley Trace 1.30309091 18.6776 3 43 0.0001
Roy's Greatest Root 1.30309091 18.6776 3 43 0.0001
H = Type III SS&CP Matrix for ABST
GROWTH PROFIT NEW
GROWTH 92.585811859 -67.22972077 76.603944
PROFIT -67.22972077 48.817796856 -55.62474057
NEW 76.603944 -55.62474057 63.380815252
Manova Test Criteria and Exact F Statistics for
the Hypothesis of no Overall ABST Effect
S=1 M=0.5 N=20.5
Statistic Value F Num DF Den DF Pr > F
Wilks' Lambda 0.17661640 66.8218 3 43 0.0001
Pillai's Trace 0.82338360 66.8218 3 43 0.0001
Hotelling-Lawley Trace 4.66198821 66.8218 3 43 0.0001
Roy's Greatest Root 4.66198821 66.8218 3 43 0.0001
H = Type III SS&CP Matrix for MATH
GROWTH PROFIT NEW
GROWTH 548.00955988 983.26753203 287.61601248
PROFIT 983.26753203 1764.230244 516.05575425
NEW 287.61601248 516.05575425 150.95169261
Manova Test Criteria and Exact F Statistics for
the Hypothesis of no Overall MATH Effect
S=1 M=0.5 N=20.5
Statistic Value F Num DF Den DF Pr > F
Wilks' Lambda 0.04524661 302.4492 3 43 0.0001
Pillai's Trace 0.95475339 302.4492 3 43 0.0001
Hotelling-Lawley Trace 21.10110519 302.4492 3 43 0.0001
Roy's Greatest Root 21.10110519 302.4492 3 43 0.0001
All the variables are very significant predictors of the sales indices. The Type 3 Sums of Squares, which adjust for all other variables in the model are the relevant ones and show that from a multivariate point of view no variables can be deleted. However, in the univariate regression for PROFIT we can probability drop Creativity while for predicting NEW sales Mechanical reasoning appears unimportant. All 4 must be retained for prediction of sales growth.
So far as I can see this data set is relatively well understood from this
regression output, though the correlation structure is of some interest
too.