Tests of Mean Differences

• t- and F-tests are tests of mean differences
  • t-tests are used for two means
  • F-tests are used for three or more means
• I.e., They are used to test whether two means are significantly different from each other
• Nonetheless, they can also be used to test if means are different than zero

t-Tests

• Invented by William Gosset
  • Devised to test differences in small samples
• Tests the size of a mean difference against a distribution of size differences one would expect if there was no real difference
  • I.e., if the mean difference we got was just by chance
  • If the mean difference is big enough—given that sample size—then we reject that the mean difference we got is part of the null distribution
  • And thus significant
• The distribution tested against is a t-distribution
  • Which approximates a normal distribution

F-Tests

• Invented by Ronald Fisher
  • Called “F” in his honor by George Snedecor, who used it in his contributions to creating ANOVA family tests
• Devised to test signal-to-noise ratios
  • Thus seeing if enough variance is accounted for by a variable to be considered significant
• Turns out to be simply the square of a t-score (F = t²)
  • (And likewise tested against an F-distribution that also approximates a normal dist.)
• Note that F-tests are relatively sensitive to deviations from normality
  • t-tests are a bit less sensitive

Assumptions of ANOVAs etc.

• A group of similar tests
• That all follow the basic idea of a linear regression
  • I.e., assuming the relationship between the predictors (IVs) and outcome (DV) is linear
  • And assuming any deviations from linearity are entirely due to error
• Also assumes the variables and error all all normally distributed
  • Since it uses F-tests, this assumption is worth investigating more than usual

What ANOVAs Test

• ANOVAs are designed to test if two or more groups differ in the mean value of the outcome
  • E.g., if patients of different races tend to have different mean levels of blood lipids (e.g., LDLs)
• So:
  • The outcome variable (DV) is continuous
  • The main predictors are nominal
  • But ANOVAs can contain other types of predictors that are continuous, etc.
    • These are called by different names (discussed below)

What ANOVAs Test (cont.)

• ANOVAs are also designed to conduct “omnibus” tests
  • If a predictor is found to be significant, it means that there is one or more significant difference somewhere in that variable
    • E.g., that one or more racial groups have different mean levels of blood LDLs

What ANOVAs Test (cont.)

• The use of an initial, omnibus is intentional:
  • The initial omnibus test is used to control for conducting too many significance tests
  • And thus helping control for false positives (Type 1 errors)
• Tests of one variable alone is called a test of that variables main effect
  • We can also test the interactions between two (or more) variables

The ANOVA Family

• There is a “family” of tests that all are related mathematically
  • And all given names similar to “ANOVA”
• All use ordinary least squares to compute the linear regression
  • And similar (but not identical) methods to estimate error
  • Differences in computing the error term is the main way they differ

Types of ANOVAs

N.b., we can also have, e.g., a repeated measures MANCOVA