Inhoudsopgave
Midterm week 1 ............................................................................................................................................... 5
Chapter 2: The Simple Regression Model ........................................................................................................... 5
2.1 Scatterplots and conditional distributions ..................................................................................................... 5
2.1.1. Scatterplots .......................................................................................................................................... 5
2.1.2. A line through conditional means ........................................................................................................ 5
2.1.3 Errors of Estimate ................................................................................................................................. 5
2.2 The Simple regression model ........................................................................................................................ 5
2.2.1. The Regression Line ............................................................................................................................. 5
2.2.2. Variance, Covariance, and correlation ................................................................................................. 5
2.2.3 Finding the Regression Line .................................................................................................................. 6
2.2.4. Example computations......................................................................................................................... 6
2.2.5. Linear regression analysis by computer ............................................................................................... 6
2.3 The regression coefficient versus the correlation coefficient ......................................................................... 6
2.3.1. Properties of the Regression and Correlation Coefficients .................................................................. 6
2.3.2. Uses of the regression and correlation coefficients ............................................................................. 7
2.4 Residuals ....................................................................................................................................................... 7
2.4.1 The three components of Y ................................................................................................................... 7
2.4.2. Algebraic properties of residuals.......................................................................................................... 7
2.4.3. Residuals as Y adjusted for differences in X ......................................................................................... 7
2.4.4. Residual analysis .................................................................................................................................. 7
Chapter 3: Partial Relationship and the Multiple Regression Model .................................................................... 8
3.1. Regression analysis with more than one predictor variable ......................................................................... 8
3.1.1. An Example .......................................................................................................................................... 8
3.1.2. Regressors ............................................................................................................................................ 8
3.1.3. Models ................................................................................................................................................. 8
3.1.4. Representing a model geometrically .................................................................................................... 8
3.1.5. Model errors ........................................................................................................................................ 8
3.1.6. An alternative view of the model ......................................................................................................... 8
3.2. The Best-Fitting Model ................................................................................................................................. 8
3.2.1. Model estimation with Computer Software ......................................................................................... 8
3.2.2. Partial regression coefficients .............................................................................................................. 8
3.2.3. The regression constant ....................................................................................................................... 8
3.2.4. Problems with three or more regressors ............................................................................................. 8
3.2.5. The multiple correlation R .................................................................................................................... 8
3.3.3. The standardized regression coefficient .............................................................................................. 8
4.2 The ANOVA summary table........................................................................................................................... 8
4.2.1. Data = model + error ............................................................................................................................ 8
4.2.2. Total and regression sums of squares .................................................................................................. 8
4.2.3. Degrees of Freedom .................................................................................................................................. 9
4.2.4. Mean squares ......................................................................................................................................... 10
4.3 Inference about the multiple correlation..................................................................................................... 11
4.3.1 Biased and less biased estimation of Rsquared................................................................................... 11
4.2.3 Testing a hypothesis about tR ............................................................................................................. 11
Installing & loading packages ......................................................................................................................... 13
,How to… ......................................................................................................................................................... 13
Adding a Column to Your Datafile: .................................................................................................................... 13
Manually Calculating the Mean: ....................................................................................................................... 13
Manually Calculating the Predicted Mean: ....................................................................................................... 13
Manually Calculating the Residual: ................................................................................................................... 13
Manually Calculating the Mean Residual: ......................................................................................................... 13
Manually Calculating the Squared Residuals: ................................................................................................... 13
Manually Calculating the Mean Squared Residual: ........................................................................................... 13
Manually Calculating SSE (Sum of Squared Residuals): ..................................................................................... 13
Manually Calculating TSS (Total Sum of Squares): ............................................................................................ 14
Manually Calculating RSS (Regression Sum of Squares): ................................................................................... 14
Manually Calculating R-squared (R2): ............................................................................................................... 14
Manually calculating the F-statistic .................................................................................................................. 14
Midterm week 2 ............................................................................................................................................. 16
Installing & loading packages ......................................................................................................................... 16
How to… ......................................................................................................................................................... 16
Assumptions ..................................................................................................................................................... 16
Chapter 4: ...................................................................................................................................................... 18
4.1.2. Assumptions for Proper Inference ..................................................................................................... 18
4.4. The Distribution of and Inference about a partial regression coefficient .................................................. 18
4.4.1 Testing a Null hypothesis about Tb ..................................................................................................... 18
4.4.2 Interval Estimates for Tb ..................................................................................................................... 18
4.4.3 Factors Affecting the Standard Error of b ........................................................................................... 19
4.4.4 Tolerance ............................................................................................................................................ 19
4.7 Miscellaneous Issues in Inference ............................................................................................................... 21
4.7.1 How Great a Drawback is Collinearity? ............................................................................................... 21
4.7.2 Contradicting Inferences ..................................................................................................................... 21
4.7.3 Sample Size and Nonsignificant Covariates ........................................................................................ 21
4.7.4 Inference in Simple Regression (when k=1) ........................................................................................ 22
Chapter 5: Extending Regression Analysis Principles ...................................................................................... 22
5.1 Dichotomous regressors ............................................................................................................................. 22
5.1.1 Indicator or dummy variables ............................................................................................................. 22
5.1.2 Estimates of Y are Group Means......................................................................................................... 22
5.1.3. The regression coefficitien for an indicator is a Difference ................................................................ 22
5.1.4 A graphic representation .................................................................................................................... 22
5.1.5 A Caution About Standardized Regression Coefficients For Dichotomous Regressors ....................... 22
5.1.6 Artificial categorization of numerical variables................................................................................... 23
Chapter 7: ...................................................................................................................................................... 23
7.3 Selection Predictor Variables ...................................................................................................................... 23
7.3.1. Stepwise regression ........................................................................................................................... 23
7.3.2. All subsets regression ........................................................................................................................ 24
7.3.3 How Do Variable Selection Methods Perform? .................................................................................. 24
,Chapter 8: Assessing The Importance Of Regressors....................................................................................... 24
8.1 What Does It Mean For A Variable To Be Important? ................................................................................. 24
8.1.1. Variable Importance in Substantive or Applied Terms ....................................................................... 24
8.1.2. Variable Importance in Statistical Terms ............................................................................................ 24
8.3 Determining the Relative Importance of Regressors in a Single Regression Model .................................... 25
8.3.1 The Limitations of the Standardized Regression Coefficient .............................................................. 25
8.3.2 The Advantage of the Semipartial Correlation ................................................................................... 25
8.3.3. Some Equivalences among measures ................................................................................................ 25
8.3.4. Eta-Squared, Partial Eta-Squared, and Cohen’s f-Squared ................................................................. 26
8.3.5. Comparing Two Regression Coefficients in the Same Model ............................................................ 27
Chapter 9: Multicategorical Regressors .......................................................................................................... 28
9.1. Multicategorical variables as sets ............................................................................................................. 28
9.1.1. Indicator coding ................................................................................................................................. 28
9.1.2. Constructing Indicator Variables ........................................................................................................ 28
9.1.3. The Reference Category ..................................................................................................................... 28
9.1.4. Testing the equality of several means................................................................................................ 29
9.1.5. Parallels with Analysis of Variance ..................................................................................................... 29
9.1.6. Interpreting Estimated Y and the Regression Coefficients ................................................................. 29
9.2 Multicategorical regressors as or with covariates ...................................................................................... 29
9.2.1 Multicategorical Variables as Covariates ............................................................................................ 29
9.2.2 Comparing Groups and Statistical Control .......................................................................................... 29
9.2.3 Interpretation of regression coefficients ............................................................................................ 30
9.2.4. Adjusted Means ................................................................................................................................. 30
9.2.5. Parallels with ANCOVA ....................................................................................................................... 30
9.2.6. More Than One Covariate.................................................................................................................. 30
Chapter 16: Detecting and Managing Irregularities ........................................................................................ 30
16.1 Regression diagnostics ............................................................................................................................. 30
16.1.1. Shortcomings of eyeballing the Data ............................................................................................... 30
16.1.2. Types of Extreme Cases ................................................................................................................... 30
16.1.3 Quantifying leverage, distance, and influence .................................................................................. 30
Midterm week 3 ............................................................................................................................................. 32
Theorie ........................................................................................................................................................... 32
Difference correlation and causation ............................................................................................................... 32
Spurious effect .................................................................................................................................................. 32
Theory – Mediation .......................................................................................................................................... 32
Difference spurious effect & mediation effect .................................................................................................. 33
Different types of methods to test mediation ................................................................................................... 34
Baron & Kenny ............................................................................................................................................. 34
How to calculate mediation: ........................................................................................................................ 34
Sobeltest ...................................................................................................................................................... 35
Bootstrap ..................................................................................................................................................... 35
Bootstrapping and Confidence Intervals for Effect Sizes: ............................................................................ 36
Different Effect Size Metrics:........................................................................................................................ 36
Relative Effect Sizes:..................................................................................................................................... 37
Stability and Sample Size: ............................................................................................................................ 37
𝑅² (Proportion of Variance Explained by Indirect Effect): ............................................................................ 37
Evaluation of methods ...................................................................................................................................... 38
, Midterm week 4 ............................................................................................................................................. 39
Theory ............................................................................................................................................................ 39
Moderation:...................................................................................................................................................... 39
Significance Testing: ......................................................................................................................................... 39
Modeling with Interaction: ............................................................................................................................... 40
Moderation with the PROCESS function ......................................................................................................... 40
Moderation with the PROCESS function - Interpretation .................................................................................. 40
Moderation through hierarchical regression analysis ..................................................................................... 41
Visualization ................................................................................................................................................... 41
Plotting Regression Coefficients: ...................................................................................................................... 41
Plotting Conditional Effects:.............................................................................................................................. 41
Using Johnson-Neyman Plot: ............................................................................................................................ 42
The four primary levels of measurement........................................................................................................ 43
Open question example answer ..................................................................................................................... 44
Midterm week 5 ............................................................................................................................................. 45
Analysis of Variance (ANOVA)........................................................................................................................... 45
Assumptions ..................................................................................................................................................... 46
Contrasts .......................................................................................................................................................... 47
Choosing Contrasts in ANOVA ..................................................................................................................... 47
Additional Testing Approaches .................................................................................................................... 47
Splitting the Variance ................................................................................................................................... 48
Non-Orthogonal Contrasts........................................................................................................................... 48
Post-Hoc Tests and Corrections ................................................................................................................... 48
Variance ............................................................................................................................................................ 49
Mean squares & F-test ..................................................................................................................................... 51
Effect sizes ........................................................................................................................................................ 52
Marginal vs. Estimated marginal means .......................................................................................................... 52
Midterm week 6 ............................................................................................................................................. 53
16.2 When to use MANOVA ........................................................................................................................... 53
Slides:............................................................................................................................................................. 60
Plot uitleg ....................................................................................................................................................... 66
Tutorial........................................................................................................................................................... 69
Midterm week 1 ............................................................................................................................................... 5
Chapter 2: The Simple Regression Model ........................................................................................................... 5
2.1 Scatterplots and conditional distributions ..................................................................................................... 5
2.1.1. Scatterplots .......................................................................................................................................... 5
2.1.2. A line through conditional means ........................................................................................................ 5
2.1.3 Errors of Estimate ................................................................................................................................. 5
2.2 The Simple regression model ........................................................................................................................ 5
2.2.1. The Regression Line ............................................................................................................................. 5
2.2.2. Variance, Covariance, and correlation ................................................................................................. 5
2.2.3 Finding the Regression Line .................................................................................................................. 6
2.2.4. Example computations......................................................................................................................... 6
2.2.5. Linear regression analysis by computer ............................................................................................... 6
2.3 The regression coefficient versus the correlation coefficient ......................................................................... 6
2.3.1. Properties of the Regression and Correlation Coefficients .................................................................. 6
2.3.2. Uses of the regression and correlation coefficients ............................................................................. 7
2.4 Residuals ....................................................................................................................................................... 7
2.4.1 The three components of Y ................................................................................................................... 7
2.4.2. Algebraic properties of residuals.......................................................................................................... 7
2.4.3. Residuals as Y adjusted for differences in X ......................................................................................... 7
2.4.4. Residual analysis .................................................................................................................................. 7
Chapter 3: Partial Relationship and the Multiple Regression Model .................................................................... 8
3.1. Regression analysis with more than one predictor variable ......................................................................... 8
3.1.1. An Example .......................................................................................................................................... 8
3.1.2. Regressors ............................................................................................................................................ 8
3.1.3. Models ................................................................................................................................................. 8
3.1.4. Representing a model geometrically .................................................................................................... 8
3.1.5. Model errors ........................................................................................................................................ 8
3.1.6. An alternative view of the model ......................................................................................................... 8
3.2. The Best-Fitting Model ................................................................................................................................. 8
3.2.1. Model estimation with Computer Software ......................................................................................... 8
3.2.2. Partial regression coefficients .............................................................................................................. 8
3.2.3. The regression constant ....................................................................................................................... 8
3.2.4. Problems with three or more regressors ............................................................................................. 8
3.2.5. The multiple correlation R .................................................................................................................... 8
3.3.3. The standardized regression coefficient .............................................................................................. 8
4.2 The ANOVA summary table........................................................................................................................... 8
4.2.1. Data = model + error ............................................................................................................................ 8
4.2.2. Total and regression sums of squares .................................................................................................. 8
4.2.3. Degrees of Freedom .................................................................................................................................. 9
4.2.4. Mean squares ......................................................................................................................................... 10
4.3 Inference about the multiple correlation..................................................................................................... 11
4.3.1 Biased and less biased estimation of Rsquared................................................................................... 11
4.2.3 Testing a hypothesis about tR ............................................................................................................. 11
Installing & loading packages ......................................................................................................................... 13
,How to… ......................................................................................................................................................... 13
Adding a Column to Your Datafile: .................................................................................................................... 13
Manually Calculating the Mean: ....................................................................................................................... 13
Manually Calculating the Predicted Mean: ....................................................................................................... 13
Manually Calculating the Residual: ................................................................................................................... 13
Manually Calculating the Mean Residual: ......................................................................................................... 13
Manually Calculating the Squared Residuals: ................................................................................................... 13
Manually Calculating the Mean Squared Residual: ........................................................................................... 13
Manually Calculating SSE (Sum of Squared Residuals): ..................................................................................... 13
Manually Calculating TSS (Total Sum of Squares): ............................................................................................ 14
Manually Calculating RSS (Regression Sum of Squares): ................................................................................... 14
Manually Calculating R-squared (R2): ............................................................................................................... 14
Manually calculating the F-statistic .................................................................................................................. 14
Midterm week 2 ............................................................................................................................................. 16
Installing & loading packages ......................................................................................................................... 16
How to… ......................................................................................................................................................... 16
Assumptions ..................................................................................................................................................... 16
Chapter 4: ...................................................................................................................................................... 18
4.1.2. Assumptions for Proper Inference ..................................................................................................... 18
4.4. The Distribution of and Inference about a partial regression coefficient .................................................. 18
4.4.1 Testing a Null hypothesis about Tb ..................................................................................................... 18
4.4.2 Interval Estimates for Tb ..................................................................................................................... 18
4.4.3 Factors Affecting the Standard Error of b ........................................................................................... 19
4.4.4 Tolerance ............................................................................................................................................ 19
4.7 Miscellaneous Issues in Inference ............................................................................................................... 21
4.7.1 How Great a Drawback is Collinearity? ............................................................................................... 21
4.7.2 Contradicting Inferences ..................................................................................................................... 21
4.7.3 Sample Size and Nonsignificant Covariates ........................................................................................ 21
4.7.4 Inference in Simple Regression (when k=1) ........................................................................................ 22
Chapter 5: Extending Regression Analysis Principles ...................................................................................... 22
5.1 Dichotomous regressors ............................................................................................................................. 22
5.1.1 Indicator or dummy variables ............................................................................................................. 22
5.1.2 Estimates of Y are Group Means......................................................................................................... 22
5.1.3. The regression coefficitien for an indicator is a Difference ................................................................ 22
5.1.4 A graphic representation .................................................................................................................... 22
5.1.5 A Caution About Standardized Regression Coefficients For Dichotomous Regressors ....................... 22
5.1.6 Artificial categorization of numerical variables................................................................................... 23
Chapter 7: ...................................................................................................................................................... 23
7.3 Selection Predictor Variables ...................................................................................................................... 23
7.3.1. Stepwise regression ........................................................................................................................... 23
7.3.2. All subsets regression ........................................................................................................................ 24
7.3.3 How Do Variable Selection Methods Perform? .................................................................................. 24
,Chapter 8: Assessing The Importance Of Regressors....................................................................................... 24
8.1 What Does It Mean For A Variable To Be Important? ................................................................................. 24
8.1.1. Variable Importance in Substantive or Applied Terms ....................................................................... 24
8.1.2. Variable Importance in Statistical Terms ............................................................................................ 24
8.3 Determining the Relative Importance of Regressors in a Single Regression Model .................................... 25
8.3.1 The Limitations of the Standardized Regression Coefficient .............................................................. 25
8.3.2 The Advantage of the Semipartial Correlation ................................................................................... 25
8.3.3. Some Equivalences among measures ................................................................................................ 25
8.3.4. Eta-Squared, Partial Eta-Squared, and Cohen’s f-Squared ................................................................. 26
8.3.5. Comparing Two Regression Coefficients in the Same Model ............................................................ 27
Chapter 9: Multicategorical Regressors .......................................................................................................... 28
9.1. Multicategorical variables as sets ............................................................................................................. 28
9.1.1. Indicator coding ................................................................................................................................. 28
9.1.2. Constructing Indicator Variables ........................................................................................................ 28
9.1.3. The Reference Category ..................................................................................................................... 28
9.1.4. Testing the equality of several means................................................................................................ 29
9.1.5. Parallels with Analysis of Variance ..................................................................................................... 29
9.1.6. Interpreting Estimated Y and the Regression Coefficients ................................................................. 29
9.2 Multicategorical regressors as or with covariates ...................................................................................... 29
9.2.1 Multicategorical Variables as Covariates ............................................................................................ 29
9.2.2 Comparing Groups and Statistical Control .......................................................................................... 29
9.2.3 Interpretation of regression coefficients ............................................................................................ 30
9.2.4. Adjusted Means ................................................................................................................................. 30
9.2.5. Parallels with ANCOVA ....................................................................................................................... 30
9.2.6. More Than One Covariate.................................................................................................................. 30
Chapter 16: Detecting and Managing Irregularities ........................................................................................ 30
16.1 Regression diagnostics ............................................................................................................................. 30
16.1.1. Shortcomings of eyeballing the Data ............................................................................................... 30
16.1.2. Types of Extreme Cases ................................................................................................................... 30
16.1.3 Quantifying leverage, distance, and influence .................................................................................. 30
Midterm week 3 ............................................................................................................................................. 32
Theorie ........................................................................................................................................................... 32
Difference correlation and causation ............................................................................................................... 32
Spurious effect .................................................................................................................................................. 32
Theory – Mediation .......................................................................................................................................... 32
Difference spurious effect & mediation effect .................................................................................................. 33
Different types of methods to test mediation ................................................................................................... 34
Baron & Kenny ............................................................................................................................................. 34
How to calculate mediation: ........................................................................................................................ 34
Sobeltest ...................................................................................................................................................... 35
Bootstrap ..................................................................................................................................................... 35
Bootstrapping and Confidence Intervals for Effect Sizes: ............................................................................ 36
Different Effect Size Metrics:........................................................................................................................ 36
Relative Effect Sizes:..................................................................................................................................... 37
Stability and Sample Size: ............................................................................................................................ 37
𝑅² (Proportion of Variance Explained by Indirect Effect): ............................................................................ 37
Evaluation of methods ...................................................................................................................................... 38
, Midterm week 4 ............................................................................................................................................. 39
Theory ............................................................................................................................................................ 39
Moderation:...................................................................................................................................................... 39
Significance Testing: ......................................................................................................................................... 39
Modeling with Interaction: ............................................................................................................................... 40
Moderation with the PROCESS function ......................................................................................................... 40
Moderation with the PROCESS function - Interpretation .................................................................................. 40
Moderation through hierarchical regression analysis ..................................................................................... 41
Visualization ................................................................................................................................................... 41
Plotting Regression Coefficients: ...................................................................................................................... 41
Plotting Conditional Effects:.............................................................................................................................. 41
Using Johnson-Neyman Plot: ............................................................................................................................ 42
The four primary levels of measurement........................................................................................................ 43
Open question example answer ..................................................................................................................... 44
Midterm week 5 ............................................................................................................................................. 45
Analysis of Variance (ANOVA)........................................................................................................................... 45
Assumptions ..................................................................................................................................................... 46
Contrasts .......................................................................................................................................................... 47
Choosing Contrasts in ANOVA ..................................................................................................................... 47
Additional Testing Approaches .................................................................................................................... 47
Splitting the Variance ................................................................................................................................... 48
Non-Orthogonal Contrasts........................................................................................................................... 48
Post-Hoc Tests and Corrections ................................................................................................................... 48
Variance ............................................................................................................................................................ 49
Mean squares & F-test ..................................................................................................................................... 51
Effect sizes ........................................................................................................................................................ 52
Marginal vs. Estimated marginal means .......................................................................................................... 52
Midterm week 6 ............................................................................................................................................. 53
16.2 When to use MANOVA ........................................................................................................................... 53
Slides:............................................................................................................................................................. 60
Plot uitleg ....................................................................................................................................................... 66
Tutorial........................................................................................................................................................... 69
