Main Steps to
Exploratory Factor
Analysis
- Extraction
- Rotation
- Interpretation
Introduction to Factor Analysis
& Exploratory Factor
Analysis
Overview
- Introduction to Factor Analysis
- An Aside About Principal Component Analysis
- Exploratory Factor Analysis
- Concept
- General Steps
Introduction to Factor Analysis
Introduction
- “Measure one thing, and measure it well.”
- Internal consistency of an instrument:
- Testing the unitary nature of an instrument
- Internal consistency of an instrument:
- But what if it doesn’t measure one thing well?
- What if an instrument measures more than one thing?
Factors
- Often, instruments measure more than one thing
- E.g., a test of content knowledge may measure more than one content
area:
- Health promotion & maintenance
- Psychosocial integrity
- Physiological integrity
- Safe & effective care environment
- E.g., a test of content knowledge may measure more than one content
area:
Factors (cont.)
- A respondent may know one content area better than others
- E.g., know “psychosocial integrity” better than “health promotion & maintenance”
- So items related to “psychosocial integrity” should correlate
together well
- And correlate less well with items about “health promotion & maintenance”
Factors (cont.)
- Items that intercorrelate are assumed to measure the same,
underlying trait
- I.e., sample the same domain
- A domain, of course, is a theoretical, non-ostensible (endogenous) construct
- I.e., sample the same domain
- The collection of measured items, though, creates a score
- The linear combination of related items is a factor score
Factors (cont.)
- One can therefore measure if all items on an instrument indeed measure the same factor
- And whether, instead, an instrument measures more than one factor
- By testing how well items correlated with—“load”
onto—a given factor
- Factor loadings are thus the correlations / standardized regression coefficients between items and the factors they load onto
Factors (cont.)
- A factor, then, is that which related items measure
- Factor vs. domain:
- Both are not directly measured
- A factor is nonetheless indirectly quantifiable through the items
- A domain is only sampled, and rarely fully measurable
- The nature of that domain is investigated through content (and construct) analysis
Factors (cont.)
- Factor as linear combination
\[\text{Whole Instrument} = x_{1} + x_{2}
+ x_{3} + x_{4}\]
\[\text{Whole
Instrument} = 1x_{1} + 1x_{2} + 1x_{3} + 1x_{4}\]
\[\text{Factor}_{1} = 1x_{1} +
1x_{2} + 0x_{3} + 0x_{4}\]
\[\text{Factor}_{2} = 0x_{1} + 0x_{2} + 1x_{3} +
1x_{4}\]
Factors (end)
- Factor as linear combination (cont.)
\[\text{Whole Instrument} = x_{1} + x_{2}
+ x_{3} + x_{4}\]
\[\text{Whole
Instrument} = 1x_{1} + 1x_{2} + 1x_{3} + 1x_{4}\]
\[\text{Factor}_{1} = .7x_{1} + .5x_{2} +
.1x_{3} + .2x_{4}\]
\[\text{Factor}_{2} = .1x_{1} + .1x_{2} + .6x_{3} +
.8x_{4}\]
Types of Factor Analysis
- Factor analysis is, in fact, a very broad field
- Generally (and here), though, we will focus on two types:
- Exploratory factor analysis
- Letting the data suggest a possible factor structure
- Confirmatory factor analysis
- Using hypotheses to test a factor structure
- Exploratory factor analysis
Nearly as an Aside…
EFA & PCA
- Factor analysis is an example of “data reduction”
- I.e., attempting to find a more parsimonious explanation for a
series of responses & their concomitant data
- Reduce the number of explanatory variables while loosing the least important information
EFA & PCA (cont.)
- Factor analysis is similar to principal
component anlaysis (PCA)
- Which differs from factor analysis in not attempting to find an underlying, theoretical structure
- PCA also assumes there is no unique variance to items
- That all variance is shared variance
- Factor analysis assumes there is shared variance
& unique variance
- The “unique” being those underlying factors
EFA & PCA (end)
| Goal of Common Factor Analysis |
Goal of Principal Component Analysis |
|---|---|
| Explore and test the presence & structure of possible latent (non-ostensible) constructs | Reduce list of items (or variables) to a linear combination of a smaller set of components |
| I.e., make inferences about what the items measure | I.e., explain data more succinctly |
Exploratory Factor Analysis
Main Steps to
Factor Extraction
EFA: Extraction
- Using the data to estimate the number of underlying factors
- Typically try to determine the fewest justifiable factors
- Although numerical conventions exist,
- It still is a bit of an art
EFA: Extraction (cont.)
- Extraction criteria
- Eigenvalues ⮜
- Scree plot
EFA: Extraction (cont.)
- Eigenvalues
- Remember this:
\[\text{Factor}_{1} = .7x_{1} + .5x_{2} + .1x_{3} + .2x_{4}\]
- An eigenvalue is simply the sum of these squared loadings:
\[\text{Eigenvalue}_{1} = .7^{2} + .5^{2}
+ .1^{2} + .2^{2}\]
\[\text{Eigenvalue}_{1} = .49 + .25 + .01 + .04 =
.79\]
EFA: Extraction (cont.)
- But what does that mean?
- It describes how much of the total variance is explained by that factor
- Each item instrument has an eigenvalue of 1
- So values >1 contribute more than a single item
- The mean eigenvalue for an instrument is 1
- So >1 also provides above-average information
EFA: Extraction (cont.)
- Because we are interested in explaining as much variance as possible
- With the fewest latent factors
- We can decide to retain only those latent factors with sufficiently high eigenvalues
EFA: Extraction (cont.)
- Extraction criteria
- Eigenvalues
- Scree plot ⮜
EFA: Extraction (cont.)
Actual scree around the bottom of a cliff
EFA: Extraction (end)
Scree plot of factors
Rotation
EFA: Rotation
- Factors may be naturally orthogonal:
\[\text{Factor}_{1} = 1x_{1} + 1x_{2} +
0x_{3} + 0x_{4}\]
\[\text{Factor}_{2} = 0x_{1} + 0x_{2} + 1x_{3} +
1x_{4}\]
- But more often, they are not entirely so:
\[\text{Factor}_{1} = .7x_{1} + .5x_{2} +
.1x_{3} + .2x_{4}\]
\[\text{Factor}_{2} = .1x_{1} + .1x_{2} + .6x_{3} +
.8x_{4}\]
EFA: Rotation (cont.)
- Orthogonal factors are at right angles to each other
- If the extracted factors are intercorrelated,
- We can “rotate” them to be uncorrelated
EFA: Rotation (cont.)
- Non-orthogonal factors:
- The factors intercorrelate
- The level of intercorrelation can be chosen
- This is called an oblique solution
EFA: Rotation (end)
- There are many
methods one can use to rotate factors
- And thus the items that are loaded onto each factor
- Researchers can try out several
rotations,
- But usually trying out a couple suffices
- E.g., one or two each of oblique & orthogonal
- Exploring rotations can help interpret factors
Common Orthogonal Rotations
- Varimax
- Attempts to simplify interpretations of factors
- By making the factor loadings for each item as close to 0 or as close to 1 as possible
- Attempts to simplify interpretations of factors
- Quartimax
- Attempts to simplify interpretation of items
- By maximizing the variance of the loadings between factors
- Quartimax accentutates differenecs between factors
- Changing proportion of variance factors explains
- Attempts to simplify interpretation of items
Common Oblique Rotations
- Promax
- Tends to give rather easily-interpreted loadings onto the factors
- Direct Oblimin
- Attempts to find the best axis for each factor
- Promax tends to produce more orthogonal factors than direct oblimin
Interpretation
EFA: Interpretation
- EFA does not directly test what domains are being represented
- It can only suggest which items are most related to the proffered factors
- It is up to us to decide what constructs underlie the factors
- In large part by evaluating items loadings
- E.g., from different extractions & rotations
- In large part by evaluating items loadings
Final Thoughts on EFAs
- EFA is highly data driven
- And based on subjectivity
- It is therefore best for initial analyses
- And, well, exploration
- EFA is not to be discounted
- But it is, frankly, often abused
Please
Don’t abuse EFAs
The End