Factor Analysis example

The data for this example are in Table 9.12 in Johnson and Wichern. They consist of 3 measurements on the sales performance of 50 salespeople for a large firm and 4 test scores.

The data begin:

I used SAS to carry out Factor Analysis of these variables several different ways.

SAS code for first run, requesting principal components factor analysis, no rotation, all output printed and allowing SAS to select m the number of factors:

```data sales;
infile "T9-12.DAT";
input growth profit new create mech abst math;
proc factor method=prin rotate=none all;
run;```
The (edited) output is
```Initial Factor Method: Principal Components
Inverse Correlation Matrix
GROWTH   PROFIT     NEW   CREATE    MECH     ABST    MATH
GROWTH  35.1412  -8.2675  15.7402 -13.9069 -1.5588 -13.8694 -23.7101
PROFIT  -8.2675  31.6239  -2.7059   1.4128 -8.1820   6.5890 -19.5028
NEW     15.7402  -2.7059  21.6934 -13.2468 -0.1384 -10.4680 -19.0616
CREATE -13.9069   1.4128 -13.2468  10.5387 -0.1251   7.6430  14.2502
MECH    -1.5588  -8.1820  -0.1384  -0.1251  4.5610  -0.8843   7.2216
ABST   -13.8694   6.5890 -10.4680   7.6430 -0.8843   8.6865   7.9984
MATH   -23.7101 -19.5028 -19.0616  14.2502  7.2216   7.9984  43.0948

Partial Correlations Controlling all other Variables

GROWTH    PROFIT       NEW    CREATE   MECH      ABST      MATH
GROWTH     1.00000   0.24800  -0.57008   0.72265  0.12312   0.79383   0.60927
PROFIT     0.24800   1.00000   0.10331  -0.07739  0.68127  -0.39755   0.52829
NEW       -0.57008   0.10331   1.00000   0.87610  0.01392   0.76257   0.62342
CREATE     0.72265  -0.07739   0.87610   1.00000  0.01805  -0.79882  -0.66868
MECH       0.12312   0.68127   0.01392   0.01805  1.00000   0.14049  -0.51510
ABST       0.79383  -0.39755   0.76257  -0.79882  0.14049   1.00000  -0.41340
MATH       0.60927   0.52829   0.62342  -0.66868 -0.51510  -0.41340   1.00000

Kaiser's Measure of Sampling Adequacy: Over-all MSA = 0.61609232

GROWTH    PROFIT       NEW    CREATE      MECH      ABST      MATH
0.662688  0.782579  0.629694  0.409556  0.749798  0.417191  0.631877

Prior Communality Estimates: ONE

Eigenvalues of the Correlation Matrix:  Total = 7  Average = 1

1      2     3        4     5        6        7
Eigenvalue   5.0346  0.9335  0.4979  0.4212 0.0810  0.0203  0.0113
Difference   4.1011  0.4356  0.0767  0.3402 0.0607  0.0090
Proportion   0.7192  0.1334  0.0711  0.0602 0.0116  0.0029  0.0016
Cumulative   0.7192  0.8526  0.9237  0.9839 0.9955  0.9984  1.0000

1 factors will be retained by the MINEIGEN criterion.
Eigenvectors
GROWTH     0.43367
PROFIT     0.42021
NEW        0.42105
CREATE     0.29429
MECH       0.34909
ABST       0.28917
MATH       0.40740

Factor Pattern
FACTOR1
GROWTH     0.97307
PROFIT     0.94287
NEW        0.94475
CREATE     0.66032
MECH       0.78329
ABST       0.64883
MATH       0.91413

Variance explained by each factor
FACTOR1
5.034598

Final Communality Estimates: Total = 5.034598

GROWTH    PROFIT       NEW    CREATE      MECH      ABST      MATH
0.946864  0.889006  0.892553  0.436018  0.613542  0.420981  0.835633

Scoring Coefficients Estimated by Regression

Squared Multiple Correlations of the Variables with each Factor

FACTOR1
1.000000

Standardized Scoring Coefficients

FACTOR1
GROWTH     0.19328
PROFIT     0.18728
NEW        0.18765
CREATE     0.13116
MECH       0.15558
ABST       0.12887
MATH       0.18157

Residual Correlations With Uniqueness on the Diagonal

GROWTH    PROFIT      NEW    CREATE   MECH      ABST      MATH
GROWTH   0.05314   0.00860  -0.03531  -0.07050 -0.05412   0.04305   0.03780
PROFIT   0.00860   0.11099  -0.04825  -0.08109  0.00737  -0.14638   0.08239
NEW     -0.03531  -0.04825   0.10745   0.07653 -0.10254   0.02811  -0.01106
CREATE  -0.07050  -0.08109   0.07653   0.56398  0.07352  -0.28153  -0.19098
MECH    -0.05412   0.00737  -0.10254   0.07352  0.38646  -0.12227  -0.14147
ABST     0.04305  -0.14638   0.02811  -0.28153 -0.12227   0.57902  -0.02674
MATH     0.03780   0.08239  -0.01106  -0.19098 -0.14147  -0.02674   0.16437

Root Mean Square Off-diagonal Residuals: Over-all = 0.10322669

GROWTH    PROFIT       NEW    CREATE      MECH      ABST      MATH
0.045646  0.078788  0.058961  0.151951  0.094753  0.140826  0.104516

Partial Correlations Controlling Factors

GROWTH    PROFIT       NEW    CREATE
MECH      ABST      MATH
GROWTH  1.00000   0.11194  -0.46725  -0.40724 -0.37768   0.24544   0.40447
PROFIT  0.11194   1.00000  -0.44187  -0.32409  0.03558  -0.57739   0.60998
NEW    -0.46725  -0.44187   1.00000   0.31088 -0.50322   0.11268  -0.08320
CREATE -0.40724  -0.32409   0.31088   1.00000  0.15747  -0.49265  -0.62725
MECH   -0.37768   0.03558  -0.50322   0.15747  1.00000  -0.25848  -0.56133
ABST    0.24544  -0.57739   0.11268  -0.49265 -0.25848   1.00000  -0.08669
MATH    0.40447   0.60998  -0.08320  -0.62725 -0.56133  -0.08669   1.00000

Root Mean Square Off-diagonal Partials: Over-all = 0.38975152

GROWTH    PROFIT       NEW    CREATE      MECH      ABST      MATH
0.356644  0.412217  0.361262  0.414017  0.366023  0.347216  0.458011```
The procedure selects only one factor on which all the variables are fairly highly loaded. The factor is the first principal component of the correlation matrix.

Now I try insisting on two factors, using varimax rotation and then plotting the factor loadings before and after rotation.

`proc factor m=prin nfactor=2 rotate=v preplot plot all;`
The output
```           2 factors will be retained by the NFACTOR criterion.
Initial Factor Method: Principal Components

Factor Pattern

FACTOR1   FACTOR2
GROWTH     0.97307  -0.10798
PROFIT     0.94287   0.02830
NEW        0.94475   0.00889
CREATE     0.66032   0.64581
MECH       0.78329   0.28497
ABST       0.64883  -0.62066
MATH       0.91413  -0.19359

Variance explained by each factor

FACTOR1   FACTOR2
5.034598  0.933516

Final Communality Estimates: Total = 5.968114

GROWTH    PROFIT       NEW    CREATE      MECH      ABST      MATH
0.958522  0.889807  0.892632  0.853094  0.694751  0.806195  0.873111

Residual Correlations With Uniqueness on the Diagonal

GROWTH    PROFIT       NEW    CREATE   MECH      ABST      MATH
GROWTH  0.04148   0.01165  -0.03435  -0.00077 -0.02335  -0.02397   0.01690
PROFIT  0.01165   0.11019  -0.04851  -0.09936 -0.00070  -0.12881   0.08787
NEW    -0.03435  -0.04851   0.10737   0.07079 -0.10508   0.03362  -0.00934
CREATE -0.00077  -0.09936   0.07079   0.14691 -0.11052   0.11930  -0.06595
MECH   -0.02335  -0.00070  -0.10508  -0.11052  0.30525   0.05460  -0.08631
ABST   -0.02397  -0.12881   0.03362   0.11930  0.05460   0.19380  -0.14690
MATH    0.01690   0.08787  -0.00934  -0.06595 -0.08631  -0.14690   0.12689

Root Mean Square Off-diagonal Residuals: Over-all = 0.07538345

GROWTH    PROFIT       NEW    CREATE      MECH      ABST      MATH
0.021296  0.078182  0.058881  0.087256  0.075533  0.097545  0.083137

Root Mean Square Off-diagonal Partials: Over-all = 0.50908239

GROWTH    PROFIT       NEW    CREATE      MECH      ABST      MATH
0.277921  0.601054  0.443291  0.567641  0.386234  0.623235  0.565089

Plot of Factor Pattern for FACTOR1 and FACTOR2

FACTOR1
1
A   B
G    .9

.8       E

.7
F                                     D
.6

.5

.4

.3

.2
F
.1                               A
C
-1 -.9-.8-.7-.6-.5-.4-.3-.2-.1  0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0T
O
-.1                               R
2
-.2
GROWTH=A  PROFIT=B    NEW=B  CREATE=D  MECH=E ABST=F  MATH=G

Rotation Method: Varimax

Orthogonal Transformation Matrix
1         2
1      0.73145   0.68189
2     -0.68189   0.73145

Rotated Factor Pattern

FACTOR1   FACTOR2
GROWTH     0.78538   0.58455
PROFIT     0.67037   0.66364
NEW        0.68498   0.65072
CREATE     0.04261   0.92265
MECH       0.37862   0.74256
ABST       0.89781  -0.01155
MATH       0.80065   0.48174

Variance explained by each factor

FACTOR1   FACTOR2
3.127683  2.840431
Final Communality Estimates: Total = 5.968114

GROWTH    PROFIT       NEW    CREATE      MECH      ABST      MATH
0.958522  0.889807  0.892632  0.853094  0.694751  0.806195  0.873111

Scoring Coefficients Estimated by Regression
Squared Multiple Correlations of the Variables with each Factor

FACTOR1   FACTOR2
1.000000  1.000000

Standardized Scoring Coefficients

FACTOR1   FACTOR2
GROWTH     0.22024   0.04719
PROFIT     0.11632   0.14988
NEW        0.13076   0.13492
CREATE    -0.37581   0.59546
MECH      -0.09436   0.32938
ABST       0.54763  -0.39843
MATH       0.27422  -0.02788

Plot of Factor Pattern for FACTOR1 and FACTOR2

FACTOR1
1

F9

.8             G  A

.7                  C
B
.6

.5

.4                     E

.3

.2
F
.1                               A
D    C
-1 -.9-.8-.7-.6-.5-.4-.3-.2-.1  0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0T
O
-.1                               R
2
GROWTH=A  PROFIT=B    NEW=C  CREATE=D  MECH=E ABST=F  MATH=G```
With principal components factor analysis fitting a second factor does not change the first factor (before rotation). Creativity and abstract reasoning are loaded on the second factor with opposite signs which would appear to represent a difference between people on a dimension of creativity as opposed to abstract reasoning.

After rotation everything except creativity is loaded on factor 1, though Mechanical reasoning has a rather smaller loading. Abstraction is not loaded on factor 2.

Now I tried iterated principal factor analysis with varimax rotation. The option heywood permits iteration to continue if the estimated uniqueness of a variable drops below 0.

```proc factor m=prinit nfactor=2 rotate=v preplot plot all heywood;
run;```
```Initial Factor Method: Iterated Principal Factor Analysis

Prior Communality Estimates: ONE

2 factors will be retained by the NFACTOR criterion.
Iter    Change   Communalities

1   0.305249   0.95852 0.88981 0.89263 0.85309 0.69475 0.80620 0.87311
2   0.122396   0.96840 0.87423 0.87908 0.80520 0.60002 0.68380 0.85064
3   0.090342   0.98057 0.87429 0.88013 0.79735 0.57498 0.59346 0.85545
4   0.063814   0.98875 0.87617 0.88190 0.80757 0.56804 0.52964 0.86690
5   0.040187   0.99332 0.87790 0.88272 0.82483 0.56497 0.48946 0.87810
6   0.022953   0.99547 0.87917 0.88280 0.84403 0.56254 0.46650 0.88668
7   0.019141   0.99628 0.88000 0.88249 0.86317 0.56024 0.45397 0.89244
8   0.018399   0.99646 0.88051 0.88204 0.88157 0.55808 0.44705 0.89597
9   0.017475   0.99640 0.88084 0.88156 0.89905 0.55609 0.44302 0.89798
10   0.016551   0.99629 0.88107 0.88109 0.91560 0.55429 0.44044 0.89906
11   0.015686   0.99618 0.88126 0.88065 0.93128 0.55265 0.43862 0.89958
12   0.014897   0.99611 0.88142 0.88023 0.94618 0.55115 0.43722 0.89977
13   0.014181   0.99605 0.88158 0.87984 0.96036 0.54978 0.43605 0.89978
14   0.013532   0.99601 0.88172 0.87946 0.97389 0.54852 0.43503 0.89970
15   0.012944   0.99599 0.88186 0.87910 0.98684 0.54736 0.43411 0.89956
16   0.012408   0.99597 0.88200 0.87875 0.99925 0.54628 0.43327 0.89939
17   0.001004   0.99596 0.88213 0.87842 1.00000 0.54528 0.43250 0.89922
18   0.000295   0.99600 0.88224 0.87833 1.00000 0.54499 0.43220 0.89918

Convergence criterion satisfied.

Eigenvalues of the Reduced Correlation Matrix:
Total = 5.63350769  Average = 0.80478681

1       2       3       4     5       6      7
Eigenvalue    4.8786  0.7663  0.2047  0.0838 0.0051 -0.1182  -0.1868
Difference    4.1123  0.5616  0.1209  0.0787 0.1233  0.0686
Proportion    0.8660  0.1360  0.0363  0.0149 0.0009 -0.0210  -0.0332
Cumulative    0.8660  1.0020  1.0384  1.0532 1.0541  1.0332   1.0000

Eigenvectors
1         2
GROWTH     0.44687  -0.16852
PROFIT     0.42379  -0.08882
NEW        0.42404   0.03803
CREATE     0.30550   0.85228
MECH       0.32820   0.15943
ABST       0.26439  -0.34496
MATH       0.41224  -0.30243

Factor Pattern

FACTOR1   FACTOR2
GROWTH     0.98704  -0.14753
PROFIT     0.93605  -0.07775
NEW        0.93660   0.03329
CREATE     0.67478   0.74609
MECH       0.72492   0.13956
ABST       0.58396  -0.30198
MATH       0.91054  -0.26475

Variance explained by each factor

FACTOR1   FACTOR2
4.878597  0.766328

Final Communality Estimates: Total = 5.644925

GROWTH    PROFIT       NEW    CREATE      MECH      ABST      MATH
0.996003  0.882237  0.878330  1.011982  0.544987  0.432204  0.899183

Residual Correlations With Uniqueness on the Diagonal

GROWTH  0.00400  -0.00931  -0.03554   0.01607  0.01314   0.05347  -0.01049
PROFIT -0.00931   0.11776  -0.03159  -0.03212  0.07820  -0.10471   0.07140
NEW    -0.03554  -0.03159   0.12167   0.04352 -0.04614   0.10420   0.00857
CREATE  0.01607  -0.03212   0.04352  -0.01198 -0.00255  -0.02184  -0.00425
MECH    0.01314   0.07820  -0.04614  -0.00255  0.45501   0.00477  -0.04857
ABST    0.05347  -0.10471   0.10420  -0.02184  0.00477   0.56780  -0.04530
MATH   -0.01049   0.07140   0.00857  -0.00425 -0.04857  -0.04530   0.10082

Root Mean Square Off-diagonal Residuals: Over-all = 0.04822817

GROWTH    PROFIT       NEW    CREATE      MECH      ABST      MATH
0.028135  0.063630  0.053566  0.024783  0.042435  0.067370  0.040229

Partial Correlations Controlling Factors

GROWTH    PROFIT       NEW    CREATE    MECH      ABST      MATH
GROWTH  1.00000  -0.42908  -1.61171   0.00000  0.30811   1.12224  -0.52229
PROFIT -0.42908   1.00000  -0.26395   0.00000  0.33781  -0.40493   0.65524
NEW    -1.61171  -0.26395   1.00000   0.00000 -0.19609   0.39645   0.07734
CREATE  0.00000   0.00000   0.00000   0.00000  0.00000   0.00000   0.00000
MECH    0.30811   0.33781  -0.19609   0.00000  1.00000   0.00938  -0.22677
ABST    1.12224  -0.40493   0.39645   0.00000  0.00938   1.00000  -0.18934
MATH   -0.52229   0.65524   0.07734   0.00000 -0.22677  -0.18934   1.00000

Root Mean Square Off-diagonal Partials: Over-all = 0.51059853

GROWTH    PROFIT       NEW    CREATE      MECH      ABST      MATH
0.857210  0.400252  0.691481  0.000000  0.223239  0.519055  0.364095

Plot of Factor Pattern for FACTOR1 and FACTOR2

FACTOR1
A    1
B  C
G      .9

.8

.7   E
D
F        .6

.5

.4

.3

.2
F
.1                               A
C
-1 -.9-.8-.7-.6-.5-.4-.3-.2-.1  0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0T
O
-.1                               R
2
GROWTH=A  PROFIT=B    NEW=C  CREATE=D  MECH=E ABST=F  MATH=G

Rotation Method: Varimax

Orthogonal Transformation Matrix
1         2
1      0.84716   0.53133
2     -0.53133   0.84716

Rotated Factor Pattern
FACTOR1   FACTOR2
GROWTH     0.91457   0.39946
PROFIT     0.83430   0.43149
NEW        0.77577   0.52585
CREATE     0.17523   0.99059
MECH       0.53997   0.50340
ABST       0.65516   0.05445
MATH       0.91205   0.25951

Variance explained by each factor

FACTOR1   FACTOR2
3.717650  1.927275

Final Communality Estimates: Total = 5.644925

GROWTH    PROFIT       NEW    CREATE      MECH      ABST      MATH
0.996003  0.882237  0.878330  1.011982  0.544987  0.432204  0.899183

Scoring Coefficients Estimated by Regression

Squared Multiple Correlations of the Variables with each Factor

FACTOR1   FACTOR2
1.298471  1.311894

Standardized Scoring Coefficients

FACTOR1   FACTOR2

GROWTH     3.46196  -2.72182
PROFIT    -0.91749   1.49807
NEW        2.32759  -2.18108
CREATE    -2.03309   2.57935
MECH       0.08882  -0.22785
ABST      -1.46013   1.47280
MATH      -1.80143   1.46081
Rotation Method: Varimax

Plot of Factor Pattern for FACTOR1 and FACTOR2

FACTOR1
1

.9      G   A
B
.8              C

.7
F
.6
E
.5

.4

.3

.2                            D
F
.1                               A
C
-1 -.9-.8-.7-.6-.5-.4-.3-.2-.1  0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0T
O
-.1                               R
2
GROWTH=A  PROFIT=B    NEW=C  CREATE=D  MECH=E ABST=F  MATH=G```
The rotated factor loadings seem rather similar here. There seems to be a distinct latent variable which fully explains creativity and is at play in determining other variables a bit. There also seems to be a variable on which every variable except creativity is highly loaded. I am not really sure how to interpret this variable.

Now I tried maximum likelihood.

```proc factor m=ml nfactor=2 rotate=v preplot plot all heywood;
run;```
The output is
```Initial Factor Method: Maximum Likelihood
Prior Communality Estimates: SMC

GROWTH    PROFIT       NEW    CREATE      MECH      ABST      MATH
0.971543  0.968378  0.953903  0.905112  0.780748  0.884878  0.976795

Preliminary Eigenvalues:  Total = 148.339548  Average = 21.191364

1       2       3       4     5       6       7
Eigenvalue  130.3121  9.4410  6.5497  2.5446 0.7601 -0.5331 -0.7349
Difference  120.8711  2.8913  4.0051  1.7845 1.2932  0.2018
Proportion    0.8785  0.0636  0.0442  0.0172 0.0051 -0.0036 -0.0050
Cumulative    0.8785  0.9421  0.9863  1.0034 1.0085  1.0050  1.0000

2 factors will be retained by the NFACTOR criterion.

Iter Criterion    Ridge   Change   Communalities

1    4.18397    0.000  0.09489   0.96827 0.95946 0.92988 1.00000
0.73649 0.79560 0.96985

2    2.64993    0.000  0.37880   0.94441 0.91603 0.87751 1.00000
0.54254 0.41681 0.96161

3    2.63510    0.000  0.01113   0.93584 0.92687 0.87771 1.00000
0.53141 0.41206 0.96421

4    2.63204    0.000  0.01336   0.93342 0.92724 0.87616 1.00000
0.53049 0.39870 0.96884

5    2.63142    0.000  0.00620   0.93205 0.92916 0.87683 1.00000
0.52684 0.39250 0.96942

6    2.63130    0.000  0.00210   0.93149 0.92913 0.87648 1.00000
0.52659 0.39040 0.97035

7    2.63128    0.000  0.00125   0.93123 0.92951 0.87666 1.00000
0.52576 0.38915 0.97042

8    2.63127    0.000  0.00035   0.93112 0.92948 0.87658 1.00000
0.52575 0.38880 0.97061

Convergence criterion satisfied.

Significance tests based on 50 observations:

Test of H0: No common factors.
vs HA: At least one common factor.

Chi-square = 499.661   df = 21   Prob>chi**2 = 0.0001

Test of H0: 2 Factors are sufficient.
vs HA: More factors are needed.

Chi-square = 117.092   df = 8   Prob>chi**2 = 0.0001

Chi-square without Bartlett's correction = 128.93234776
Akaike's Information Criterion = 112.93234776
Schwarz's Bayesian Criterion = 97.636163717
Tucker and Lewis's Reliability Coefficient = 0.4017366379
Squared Canonical Correlations

FACTOR1   FACTOR2
1.000000  0.980050

Eigenvectors

1         2
GROWTH     0.57204   0.78496
PROFIT     0.03116   0.20891
NEW        0.41606  -0.17154
CREATE     0.70619  -0.54400
MECH       0.00000  -0.06982
ABST       0.00000  -0.03499
MATH       0.00000   0.09345

Eigenvalues of the Weighted Reduced Correlation Matrix:
Total = 49.1264918  Average = 8.18774864

1       2       3       4       5       6       7
Eigenvalue     .     49.1265  1.1504  0.7153 -0.2018 -0.7704 -0.8935
Difference     .     47.9761  0.4351  0.9171  0.5686  0.1231
Proportion     .      1.0000  0.0234  0.0146 -0.0041 -0.0157 -0.0182
Cumulative     .      1.0000  1.0234  1.0380  1.0339  1.0182  1.0000

Factor Pattern

FACTOR1   FACTOR2
GROWTH     0.57204   0.77709
PROFIT     0.54151   0.79768
NEW        0.70036   0.62138
CREATE     1.00000  -0.00000
MECH       0.59074   0.42024
ABST       0.14691   0.60579
MATH       0.41264   0.89462

Variance explained by each factor

FACTOR1   FACTOR2
Weighted   19.448504 49.126492
Unweighted  2.651787  2.970211

Final Communality Estimates and Variable Weights
Total Communality: Weighted = 68.574996   Unweighted = 5.621998

GROWTH    PROFIT       NEW    CREATE
Communality   0.931096  0.929532  0.876616  1.000000
Weight       14.518860 14.180085  8.102194   .

MECH      ABST      MATH
Communality   0.525572  0.388566  0.970616
Weight        2.108574  1.636117 34.029166

Residual Correlations With Uniqueness on the Diagonal

GROWTH    PROFIT      NEW   CREATE    MECH      ABST      MATH
GROWTH  0.06890  -0.00356   0.00050  0.00000  0.04359   0.11962  -0.00393
PROFIT -0.00356   0.07047  -0.03239  0.00000  0.09080  -0.09740   0.00722
NEW     0.00050  -0.03239   0.12338  0.00000 -0.03739   0.16178   0.00768
CREATE  0.00000   0.00000   0.00000  0.00000  0.00000   0.00000   0.00000
MECH    0.04359   0.09080  -0.03739  0.00000  0.47443   0.04459  -0.04516
ABST    0.11962  -0.09740   0.16178  0.00000  0.04459   0.61143  -0.03620
MATH   -0.00393   0.00722   0.00768  0.00000 -0.04516  -0.03620   0.02938

Root Mean Square Off-diagonal Residuals: Over-all = 0.05691855

GROWTH    PROFIT       NEW    CREATE      MECH      ABST      MATH
0.052019  0.056043  0.069134  0.000000  0.050941  0.094219  0.024073

Root Mean Square Off-diagonal Partials: Over-all = 0.27892670

GROWTH    PROFIT       NEW    CREATE      MECH      ABST      MATH
0.260769  0.320222  0.290900  0.000000  0.283366  0.405477  0.211487

Plot of Factor Pattern for FACTOR1 and FACTOR2

FACTOR1
D

.9

.8

.7                 C

.6           E
A
.5

.4                         G

.3

.2
F             F
.1                               A
C
-1 -.9-.8-.7-.6-.5-.4-.3-.2-.1  0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0T
O
-.1                               R
2
GROWTH=A  PROFIT=A    NEW=C  CREATE=D  MECH=E ABST=F  MATH=G

Rotation Method: Varimax

Orthogonal Transformation Matrix

1         2

1      0.14646   0.98922
2      0.98922  -0.14646

Rotated Factor Pattern

FACTOR1   FACTOR2

GROWTH     0.85249   0.45205
PROFIT     0.86839   0.41884
NEW        0.71725   0.60180
CREATE     0.14646   0.98922
MECH       0.50223   0.52282
ABST       0.62078   0.05660
MATH       0.94541   0.27716

Variance explained by each factor

FACTOR1   FACTOR2
Weighted   56.990420 11.584576
Unweighted  3.548142  2.073856

Final Communality Estimates and Variable Weights
Total Communality: Weighted = 68.574996   Unweighted = 5.621998

GROWTH    PROFIT       NEW    CREATE
Communality   0.931096  0.929532  0.876616  1.000000
Weight       14.518860 14.180085  8.102194   .

MECH      ABST      MATH
Communality   0.525572  0.388566  0.970616
Weight        2.108574  1.636117 34.029166

Scoring Coefficients Estimated by Regression

Squared Multiple Correlations of the Variables with each Factor

FACTOR1   FACTOR2
0.980478  0.999572

Standardized Scoring Coefficients

FACTOR1   FACTOR2
GROWTH     0.22265  -0.03297
PROFIT     0.22322  -0.03305
NEW        0.09935  -0.01471
CREATE    -0.43247   1.07493
MECH       0.01749  -0.00259
ABST       0.01956  -0.00290
MATH       0.60078  -0.08895

Plot of Factor Pattern for FACTOR1 and FACTOR2

FACTOR1
1
G
.9
BA
.8

.7                 C

.6F

.5              E

.4

.3

.2
D  F
.1                               A
C
-1 -.9-.8-.7-.6-.5-.4-.3-.2-.1  0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0T
O
-.1                               R
2
GROWTH=A  PROFIT=B    NEW=C  CREATE=D  MECH=E ABST=F  MATH=G```
The pattern is the same as for the other methods. Notice however, that this procedure factors S not R. This explains the totally different eigenvalues and so on.

Finally I ran proc glm regressing the sales figures on the psychological test scores.

``` proc glm ;
model growth profit new = create mech abst math;
manova h=_all_ /printh printe;
run;```
The output is:
```                      General Linear Models Procedure
Dependent Variable: GROWTH
Sum of          Mean
Source                  DF        Squares        Square  F Value    Pr > F
Model                    4     2520.49121     630.12280   241.32    0.0001
Error                   45      117.50399       2.61120
Corrected Total         49     2637.99520
R-Square           C.V.      Root MSE        GROWTH Mean
0.955457       1.634952       1.61592            98.8360
Source                  DF    Type III SS   Mean Square  F Value    Pr > F
CREATE                   1      61.431768     61.431768    23.53    0.0001
MECH                     1      25.570007     25.570007     9.79    0.0031
ABST                     1      92.585812     92.585812    35.46    0.0001
MATH                     1     548.009560    548.009560   209.87    0.0001

T for H0:     Pr > |T|    Std Error of
Parameter          Estimate     Parameter=0                  Estimate

INTERCEPT       68.95355221           51.00      0.0001      1.35196005
CREATE           0.35938316            4.85      0.0001      0.07409369
MECH             0.30033781            3.13      0.0031      0.09597644
ABST             0.79574727            5.95      0.0001      0.13363585
MATH             0.44315482           14.49      0.0001      0.03059014

Dependent Variable: PROFIT
Sum of          Mean
Source                  DF        Squares        Square  F Value    Pr > F
Model                    4     4852.96681    1213.24170   321.87    0.0001
Error                   45      169.61899       3.76931
Corrected Total         49     5022.58580
R-Square           C.V.      Root MSE        PROFIT Mean
0.966229       1.820892       1.94147            106.622
Source                  DF    Type III SS   Mean Square  F Value    Pr > F
CREATE                   1        6.79727       6.79727     1.80    0.1860
MECH                     1      213.27427     213.27427    56.58    0.0001
ABST                     1       48.81780      48.81780    12.95    0.0008
MATH                     1     1764.23024    1764.23024   468.05    0.0001
T for H0:     Pr > |T|    Std Error of
Parameter          Estimate     Parameter=0                  Estimate
INTERCEPT       75.41979427           46.43      0.0001      1.62433197
CREATE           0.11954420            1.34      0.1860      0.08902094
MECH             0.86738881            7.52      0.0001      0.11531228
ABST            -0.57781927           -3.60      0.0008      0.16055873
MATH             0.79513166           21.63      0.0001      0.03675297

Dependent Variable: NEW
Sum of          Mean
Source                  DF        Squares        Square  F Value    Pr > F
Model                    4     1013.57078     253.39270   153.11    0.0001
Error                   45       74.47422       1.65498
Corrected Total         49     1088.04500
R-Square           C.V.      Root MSE           NEW Mean
0.931552       1.251300       1.28646            102.810
Source                  DF    Type III SS   Mean Square  F Value    Pr > F
CREATE                   1     153.408261    153.408261    92.69    0.0001
MECH                     1       1.846673      1.846673     1.12    0.2965
ABST                     1      63.380815     63.380815    38.30    0.0001
MATH                     1     150.951693    150.951693    91.21    0.0001

T for H0:     Pr > |T|    Std Error of
Parameter          Estimate     Parameter=0                  Estimate
INTERCEPT       83.70817574           77.77      0.0001      1.07631783
CREATE           0.56791793            9.63      0.0001      0.05898722
MECH            -0.08071229           -1.06      0.2965      0.07640844
ABST             0.65838791            6.19      0.0001      0.10638972
MATH             0.23258431            9.55      0.0001      0.02435332

Multivariate Analysis of Variance
E = Error SS&CP Matrix
GROWTH            PROFIT               NEW
GROWTH      117.50398618      32.670138508       -52.8579666
PROFIT      32.670138508      169.61898601       -5.37650348
NEW          -52.8579666       -5.37650348      74.474217358

Partial Correlation Coefficients from the Error SS&CP Matrix / Prob > |r|
DF = 45       GROWTH    PROFIT       NEW
GROWTH      1.000000  0.231413 -0.565043
0.0001    0.1218    0.0001
PROFIT      0.231413  1.000000 -0.047837
0.1218    0.0001    0.7522
NEW        -0.565043 -0.047837  1.000000
0.0001    0.7522    0.0001

H = Type III SS&CP Matrix for CREATE

GROWTH            PROFIT               NEW
GROWTH      61.431768316      20.434489514      97.078013797
PROFIT      20.434489514      6.7972707468      32.291755705
NEW         97.078013797      32.291755705      153.40826125

Characteristic Roots and Vectors of: E Inverse * H, where
H = Type III SS&CP Matrix for CREATE   E = Error SS&CP Matrix
Characteristic   Percent        Characteristic Vector  V'EV=1
Root
GROWTH         PROFIT       NEW
5.55606041    100.00   0.09060203  -0.00667035  0.13437927
0.00000000      0.00  -0.02427392   0.07898107 -0.00126443
0.00000000      0.00   0.06725495   0.00372078 -0.04334269

Manova Test Criteria and Exact F Statistics for
the Hypothesis of no Overall CREATE Effect
H = Type III SS&CP Matrix for CREATE   E = Error SS&CP Matrix
S=1    M=0.5    N=20.5
Statistic                    Value          F      Num DF    Den DF  Pr > F
Wilks' Lambda             0.15253063    79.6369         3        43  0.0001
Pillai's Trace            0.84746937    79.6369         3        43  0.0001
Hotelling-Lawley Trace    5.55606041    79.6369         3        43  0.0001
Roy's Greatest Root       5.55606041    79.6369         3        43  0.0001
H = Type III SS&CP Matrix for MECH
GROWTH            PROFIT               NEW
GROWTH      25.570006621      73.847304527      -6.871641257
PROFIT      73.847304527      213.27426569      -19.84560239
NEW         -6.871641257      -19.84560239      1.8466734979

Manova Test Criteria and Exact F Statistics for
the Hypothesis of no Overall MECH Effect
S=1    M=0.5    N=20.5
Statistic                    Value          F      Num DF    Den DF  Pr > F
Wilks' Lambda             0.43419910    18.6776         3        43  0.0001
Pillai's Trace            0.56580090    18.6776         3        43  0.0001
Hotelling-Lawley Trace    1.30309091    18.6776         3        43  0.0001
Roy's Greatest Root       1.30309091    18.6776         3        43  0.0001
H = Type III SS&CP Matrix for ABST
GROWTH            PROFIT               NEW
GROWTH      92.585811859      -67.22972077         76.603944
PROFIT      -67.22972077      48.817796856      -55.62474057
NEW            76.603944      -55.62474057      63.380815252

Manova Test Criteria and Exact F Statistics for
the Hypothesis of no Overall ABST Effect
S=1    M=0.5    N=20.5
Statistic                    Value          F      Num DF    Den DF  Pr > F
Wilks' Lambda             0.17661640    66.8218         3        43  0.0001
Pillai's Trace            0.82338360    66.8218         3        43  0.0001
Hotelling-Lawley Trace    4.66198821    66.8218         3        43  0.0001
Roy's Greatest Root       4.66198821    66.8218         3        43  0.0001

H = Type III SS&CP Matrix for MATH
GROWTH            PROFIT               NEW
GROWTH      548.00955988      983.26753203      287.61601248
PROFIT      983.26753203       1764.230244      516.05575425
NEW         287.61601248      516.05575425      150.95169261

Manova Test Criteria and Exact F Statistics for
the Hypothesis of no Overall MATH Effect
S=1    M=0.5    N=20.5
Statistic                    Value          F      Num DF    Den DF  Pr > F
Wilks' Lambda             0.04524661   302.4492         3        43  0.0001
Pillai's Trace            0.95475339   302.4492         3        43  0.0001
Hotelling-Lawley Trace   21.10110519   302.4492         3        43  0.0001
Roy's Greatest Root      21.10110519   302.4492         3        43  0.0001```

All the variables are very significant predictors of the sales indices. The Type 3 Sums of Squares, which adjust for all other variables in the model are the relevant ones and show that from a multivariate point of view no variables can be deleted. However, in the univariate regression for PROFIT we can probability drop Creativity while for predicting NEW sales Mechanical reasoning appears unimportant. All 4 must be retained for prediction of sales growth.

So far as I can see this data set is relatively well understood from this regression output, though the correlation structure is of some interest too.