Descriptives vs. Inferentials

• Descriptives good
  • Simple & intuitive
    • Can efficiently describe the sample
  • Robust
    • Because they make no assumptions about the population
• Mean & SD
  • SD as average distance from the mean
  • SD as a standard unit of measurement
    • Standardized (z) scores
      • Why correlation is so popular
      • And covariance isn’t

Descriptives vs. Inferentials (cont.)

• Inferentials
  • Make assumptions about the population
  • Most importantly the distribution
    • Often assume it approximates a normal distribution
      • But we know we’re wrong
    • Assumptions are often most robust against:
      • Kurtosis (& skew a bit)
      • Non-independence of measures (“multicollinearity”)
      • Changes in variance over time (“heteroscedasticity”)

Descriptives vs. Inferentials (cont.)

• Inferentials (cont.)
  • Assumptions not robust against:
    • Non-independence of participants
    • Multi-modality (more than one “hump” in the distribution)

Central Limit Theorum

• “Multiple ‘draws’ of samples”
  • Sample stats never equal population stats
    • A sample stat assumed to always have some error to its measurement
      • Observed = True + Error
  • But! (assuming consistent sampling techniques)
    • The error of measurement of sample stats:
      • Tends to be normally distributed
      • Has a mean of zero (modulo bias)
    • This leads to the Central Limit Theorem
      • Which under-girds—even allows for—nearly every statistic you’ll use

Variance & Covariance

• Variance = Information
• We seek to understand that information
  • The more we understand, the better
  • Often quantify “how much we understand” as a signal-to-noise ratio
    • \(\text{Variance understood} = \frac{\text{Variance accounted for}}{\text{Variance }not \text{ accounted for}}\)

Variance & Covariance (cont.)

• Variance = Information
• Seek to understand that information
  • The more we understand, the better (cont.)

Partialing Out Variance

• Ways to increase the size of the signal to the size of the noise:
• Increase size of signal
  • Bigger effects
  • Greater range of measurements of effects

Ways of Communicating

• Merging visuals with text
  • What is best described where
  • Visuals as “conversation pieces”
    • Text as highlighting what to focus on in visuals
    • Especially vis-à-vis theory

Basic Strategy

1. Assume the relationships between variables are linear
2. Find a line that best accounts for all variables in the model
   • Either via ordinary least squares (OLS)
   • Or maximum likelihood estimation (MLE)
3. If the line—the linear regression—accounts for enough of the total enough, declare significance of model
   • Measured, e.g., with R²
4. And/or look at the individual predictors to see which of them are significant contributor to that linear regression
   • Amount of variance accounted for by those variables vs. total variance

Partialing Out Variance

• At minimum, we separate out the variance associated with our predictor(s) from “error”
  • And, perhaps, the “intercept,” the starting, pre-intervention values for each participant

Partialing Out Variance (cont.)

• N.b., error can also be separated
  • E.g., if we know the sources of those errors (same hospital, neighborhood, etc.)

Common Sources of Variance

• Similar predictors may share too much variance
  • If left un-addressed, can lead to “multicollinearity”
  • Which leads to unstable models terms
    • E.g., terms will flip from being significant to not & back depending on what other terms are added to the model
• Usually addressed by removing one of the multicollinear terms

Combining Sources of Variance (cont.)

• We do this all the time, in fact
  • Adding up responses to items on the same survey
  • Taking the average of a results from a blood draw

Combining Sources of Variance (end)

• We can group them into “families” of variables within the model…

Ostensible & Non-Ostensible

• Some Things We Can See…
  • Neighborhoods & paychecks
  • Blood pressure & adipose tissue
  • Smiles and cortisol levels

Ostensible & Non-Ostensible (cont.)

• Things we can observe empirically are sometimes called ostensible
• While the underlying “construct” they are manifestations of are non-ostensible

Ostensible & Non-Ostensible (end)

• It is quite common to use ostensible variables to represent non-ostensible constructs
  • Even if it is less common to realize (or least acknowledge) that is what one is doing
• However, it is indeed important that the ostensible variable(s) well represent the construct of actual interest

Comparing Model Fits

• If that difference is significant, then that “family” of variables in the second model significantly improves our understanding of the outcome

Comparing Model Fits (cont.)

• Common information criterion:
  • -2 \(\times\) log-likelihood (-2LL)
    • This is a “raw” information criterion from which others are derived
    • Usually not itself used
      • But is sometimes reported, so it’s good to know
  • Akaike Information Criterion (AIC)
  • Bayesian Information Criterion (BIC)
• AIC & BIC both adjust for number of terms in the model
  • More than does R²—an other reason to use them
  • But BIC adjusts more aggressively
    • So use BIC when there are a lot of terms in the model

Comparing Model Fits (cont.)

• Thus want to see that the model with the “family” of interest has a smaller information criterion
  • I.e.:

• Information criteria can be large, so this may be, e.g.:

Model Testing (cont.)

• We test this difference against a χ² with degrees of freedom equal to the difference in dfs between the models
• E.g.:

\(\ \ \ \ b_{1}X_{1}\)

\(\ \ \ \ b_{1}X_{1} + ( b_{ZIP}X_{ZIP} + b_{Salary}X_{Salary} )\)

• The second model as 2 more dfs than the first¹
  • So we test a χ² = 20 to see if it is significant
    • Against a χ² distribution based on 2 df
  • Which would be significant since the critical χ² \(\approx\) 10
    • (Since for a χ² dist., the mean = df and SD = 2df)