Statisitcs
Contents
Language of data ..................................................................................................................................... 7
Stages in Quantitative Research.............................................................................................................. 7
Quantitative Research Question ............................................................................................................. 7
Quantitative vs. Qualitative research .................................................................................................. 7
Random experiment and variables ..................................................................................................... 7
Population ........................................................................................................................................... 8
Variables .............................................................................................................................................. 8
Classification of the variables .......................................................................................................... 9
Collect Data ........................................................................................................................................... 10
Population vs. Sample ....................................................................................................................... 10
Proper Sampling ................................................................................................................................ 10
Different Time Patterns ..................................................................................................................... 11
Data Matrix ........................................................................................................................................ 11
Descriptive Statistics ............................................................................................................................. 11
Qualitative Variables ......................................................................................................................... 11
Distribution of Outcomes: Frequency Analysis ............................................................................. 11
Summary Measures ....................................................................................................................... 12
Cumulative frequencies................................................................................................................. 12
Graphical Representation of Distribution of Outcomes ................................................................ 12
Quantitative Statistics ....................................................................................................................... 14
Distribution of outcomes and summary measures ....................................................................... 14
Distribution of Outcomes .............................................................................................................. 14
Discrete Variable (small number of outcomes)......................................................................... 14
Continuous variables or discrete with large number of outcomes ........................................... 15
Numerical Representation (Bins) .......................................................................................... 15
Graphical Representation...................................................................................................... 15
Summary Measures ....................................................................................................................... 17
Mean.......................................................................................................................................... 17
Variance and Standard Deviation .............................................................................................. 18
Skewness ................................................................................................................................... 19
Kurtosis ...................................................................................................................................... 19
Several measures for the center ............................................................................................... 19
Several measures for the spread ............................................................................................... 20
1
, Cumulative Distribution ................................................................................................................ 20
Interpretation ............................................................................................................................ 21
Interpolation.............................................................................................................................. 21
Quantile Function .......................................................................................................................... 22
Determining the kth- percentile with SPSS ................................................................................ 23
Quantiles to determine Summary Measures ............................................................................ 23
Distribution of Outcomes: BOXPLOT ..................................................................................... 24
Transformation of Data ................................................................................................................. 25
Logarithmic transformation ...................................................................................................... 25
Linear Transformation ............................................................................................................... 25
Summary Measures for Transformed Variables........................................................................ 25
Z-Score ....................................................................................................................................... 26
Binned Data ................................................................................................................................... 26
Cumulative Frequencies ............................................................................................................ 27
Interpolation for cumulative frequencies ............................................................................. 27
Percentiles ............................................................................................................................. 27
Summary Measures ................................................................................................................... 28
Relationships between variables....................................................................................................... 29
Distribution of combination of outcomes ..................................................................................... 29
Overview: Bivariate Descriptive Statistics ..................................................................................... 30
Relationships ................................................................................................................................. 31
Relationships between 2 Quantitative Variables .............................................................................. 31
Relationship between Quantitative and Qualitative Variables ......................................................... 33
Relationship between Quantitative Variables................................................................................... 33
Scatterplot ..................................................................................................................................... 33
Linear Association...................................................................................................................... 33
Covariance ................................................................................................................................. 34
Correlation................................................................................................................................. 34
Linear Combination of Variables ....................................................................................................... 35
Linear Transformation ................................................................................................................... 35
Linear Combination & Summary Measures .................................................................................. 36
Mean and variance for linear combinations ............................................................................. 36
Probability Theory ................................................................................................................................. 37
Definition ........................................................................................................................................... 37
Rules to determine probabilities ....................................................................................................... 37
Rule Laplace ................................................................................................................................... 37
2
, Counting Rules............................................................................................................................... 38
Axioms of Kolmogorov .................................................................................................................. 39
Conditional Probabilities ................................................................................................................... 40
General Multiplication Rule........................................................................................................... 40
Independence ................................................................................................................................ 41
Discrete Random Variables ............................................................................................................... 41
Random Variables.......................................................................................................................... 41
Probability Distribution for a Discrete Random Variable .............................................................. 42
Probability ..................................................................................................................................... 43
Cumulative Distribution ................................................................................................................ 43
Percentiles ..................................................................................................................................... 44
Summary Measures ....................................................................................................................... 44
Population Mean ....................................................................................................................... 44
Population Variance – Standard Deviation ............................................................................... 45
Binominal Distribution................................................................................................................... 45
Probability Distribution ............................................................................................................. 46
Expected Value (=Mean) and Variance ..................................................................................... 47
How the probability distribution looks like graphically............................................................. 47
Continuous Random Variable............................................................................................................ 47
Density ........................................................................................................................................... 47
Probability ..................................................................................................................................... 49
Cumulative Distribution ................................................................................................................ 49
Summary Measures ....................................................................................................................... 49
Continuous Distribution Models ................................................................................................... 51
Normal Distribution ................................................................................................................... 51
Parameter: ............................................................................................................................. 51
Probabilities:.......................................................................................................................... 51
Quantiles: .............................................................................................................................. 52
Normal Approximation: Normal Distribution for Binomial Distribution ................................... 52
Standard Normal Distribution ................................................................................................... 52
Linear Transformation & Combination for Discrete/continuous Random variables ........................ 54
Linear Transformation ................................................................................................................... 54
Linear critiions ............................................................................................................................... 54
Sums and Averages........................................................................................................................ 55
Expectation / Mean ................................................................................................................... 56
Variance ..................................................................................................................................... 56
3
, Distribution ................................................................................................................................ 57
Central Limit Theorem ................................................................................................................... 58
Conditions for applying the CLT ................................................................................................ 59
Foundation for Inference ...................................................................................................................... 60
Point Estimator .................................................................................................................................. 60
Confidence Interval ........................................................................................................................... 61
Ingredients..................................................................................................................................... 62
Determine the CI for mean µ ........................................................................................................ 62
Interpretation ................................................................................................................................ 63
Width of CI ..................................................................................................................................... 63
Assumptions .................................................................................................................................. 64
Basics of Hypothesis Tests ................................................................................................................. 64
All Steps ......................................................................................................................................... 65
Formulate hypotheses............................................................................................................... 65
Suitable Descriptive Statistics.................................................................................................... 66
P-value ....................................................................................................................................... 67
Conclusion ................................................................................................................................. 67
Assumptions .............................................................................................................................. 67
Two-Sided Test .......................................................................................................................... 67
Two-Sided Test with Confidence Intervals ............................................................................ 68
Right-Sided Test......................................................................................................................... 68
Critical Values and Power (Type 2 error)................................................................................... 68
Hypothesis Testing ................................................................................................................................ 70
H1Inference about means and correlations...................................................................................... 70
Hypothesis testing for one mean (sigma unknown)...................................................................... 70
Critical Value(s) ...................................................................................................................... 71
Confidence Interval ............................................................................................................... 72
SPSS ....................................................................................................................................... 72
Testing Normality .......................................................................................................................... 72
Kolmogorov-Smirnov test.......................................................................................................... 73
Shapiro-Wilk .............................................................................................................................. 75
Hypothesis testing to compare two means................................................................................... 75
Paired versus Unpaired ............................................................................................................. 75
Paired T-test → turns into t-test for one mean ........................................................................ 76
Confidence Interval ............................................................................................................... 77
Unpaired T-test → compares two means ................................................................................. 77
4
, (1) Test Statistics (known sigma): .......................................................................................... 78
(2) Test Statistics (unknown sigma):...................................................................................... 78
(3) Test statistic for (unknown) σ1= σ2:.................................................................................. 78
T-test to compare 2 means in SPSS ....................................................................................... 79
Confidence Intervals (unpaired means) ................................................................................ 79
Hypothesis Testing Compare More Than 2 Means – ANOVA ................................................... 80
Post-Hoc Test......................................................................................................................... 83
T-Test for Pearson Correlation .................................................................................................. 85
H2 Chi Square Test for goodness of Fit and Independence .............................................................. 87
Goodness Of Fit Test (Chi-Test) ..................................................................................................... 87
Independence (Chi Square Test) ................................................................................................... 90
5 Simple Linear Regression................................................................................................................ 93
SR_A Model Specification.............................................................................................................. 93
Assumptions .............................................................................................................................. 95
Nonlinear relationships ............................................................................................................. 96
Linear Model.......................................................................................................................... 98
Polynomial model: quadratic ................................................................................................ 98
Polynomial model: cubic ....................................................................................................... 98
Log transformation ................................................................................................................ 98
Log Linear Model ................................................................................................................... 99
Linear Log Model ................................................................................................................... 99
Log Log Model ....................................................................................................................... 99
Dummy Variable .................................................................................................................... 99
SR_B Inference about the regression parameters ...................................................................... 100
Fitted Model ............................................................................................................................ 100
Population vs. Estimated model .......................................................................................... 100
Least Squares Principles ...................................................................................................... 101
Residuals .............................................................................................................................. 101
Calculation LS (least squared) estimates ............................................................................. 102
Interpretation ...................................................................................................................... 102
Least Squares Prediction ..................................................................................................... 102
Estimate standard error of the model σ.............................................................................. 102
Properties of LSE...................................................................................................................... 103
Bi as a random variable ....................................................................................................... 103
Expected Value .................................................................................................................... 104
Variance ............................................................................................................................... 104
5
, Standard Error of the Parameters ....................................................................................... 104
Density ................................................................................................................................. 104
Gauss Markov theorem ....................................................................................................... 105
Interval estimation and hypothesis testing ............................................................................. 105
Confidence Interval for β1 ................................................................................................... 105
Hypothesis Testing .............................................................................................................. 106
SR_C Goodness of Fit ................................................................................................................... 108
Decomposition of variance and R-square ............................................................................... 108
Measuring Goodness of Fit.................................................................................................. 108
SR_D Checking Assumptions ....................................................................................................... 111
Assumption 1 ........................................................................................................................... 111
Assumption A2 ........................................................................................................................ 112
Assumption A3 ........................................................................................................................ 112
Assumption A4 ........................................................................................................................ 113
Assumption A5 ........................................................................................................................ 113
6
Contents
Language of data ..................................................................................................................................... 7
Stages in Quantitative Research.............................................................................................................. 7
Quantitative Research Question ............................................................................................................. 7
Quantitative vs. Qualitative research .................................................................................................. 7
Random experiment and variables ..................................................................................................... 7
Population ........................................................................................................................................... 8
Variables .............................................................................................................................................. 8
Classification of the variables .......................................................................................................... 9
Collect Data ........................................................................................................................................... 10
Population vs. Sample ....................................................................................................................... 10
Proper Sampling ................................................................................................................................ 10
Different Time Patterns ..................................................................................................................... 11
Data Matrix ........................................................................................................................................ 11
Descriptive Statistics ............................................................................................................................. 11
Qualitative Variables ......................................................................................................................... 11
Distribution of Outcomes: Frequency Analysis ............................................................................. 11
Summary Measures ....................................................................................................................... 12
Cumulative frequencies................................................................................................................. 12
Graphical Representation of Distribution of Outcomes ................................................................ 12
Quantitative Statistics ....................................................................................................................... 14
Distribution of outcomes and summary measures ....................................................................... 14
Distribution of Outcomes .............................................................................................................. 14
Discrete Variable (small number of outcomes)......................................................................... 14
Continuous variables or discrete with large number of outcomes ........................................... 15
Numerical Representation (Bins) .......................................................................................... 15
Graphical Representation...................................................................................................... 15
Summary Measures ....................................................................................................................... 17
Mean.......................................................................................................................................... 17
Variance and Standard Deviation .............................................................................................. 18
Skewness ................................................................................................................................... 19
Kurtosis ...................................................................................................................................... 19
Several measures for the center ............................................................................................... 19
Several measures for the spread ............................................................................................... 20
1
, Cumulative Distribution ................................................................................................................ 20
Interpretation ............................................................................................................................ 21
Interpolation.............................................................................................................................. 21
Quantile Function .......................................................................................................................... 22
Determining the kth- percentile with SPSS ................................................................................ 23
Quantiles to determine Summary Measures ............................................................................ 23
Distribution of Outcomes: BOXPLOT ..................................................................................... 24
Transformation of Data ................................................................................................................. 25
Logarithmic transformation ...................................................................................................... 25
Linear Transformation ............................................................................................................... 25
Summary Measures for Transformed Variables........................................................................ 25
Z-Score ....................................................................................................................................... 26
Binned Data ................................................................................................................................... 26
Cumulative Frequencies ............................................................................................................ 27
Interpolation for cumulative frequencies ............................................................................. 27
Percentiles ............................................................................................................................. 27
Summary Measures ................................................................................................................... 28
Relationships between variables....................................................................................................... 29
Distribution of combination of outcomes ..................................................................................... 29
Overview: Bivariate Descriptive Statistics ..................................................................................... 30
Relationships ................................................................................................................................. 31
Relationships between 2 Quantitative Variables .............................................................................. 31
Relationship between Quantitative and Qualitative Variables ......................................................... 33
Relationship between Quantitative Variables................................................................................... 33
Scatterplot ..................................................................................................................................... 33
Linear Association...................................................................................................................... 33
Covariance ................................................................................................................................. 34
Correlation................................................................................................................................. 34
Linear Combination of Variables ....................................................................................................... 35
Linear Transformation ................................................................................................................... 35
Linear Combination & Summary Measures .................................................................................. 36
Mean and variance for linear combinations ............................................................................. 36
Probability Theory ................................................................................................................................. 37
Definition ........................................................................................................................................... 37
Rules to determine probabilities ....................................................................................................... 37
Rule Laplace ................................................................................................................................... 37
2
, Counting Rules............................................................................................................................... 38
Axioms of Kolmogorov .................................................................................................................. 39
Conditional Probabilities ................................................................................................................... 40
General Multiplication Rule........................................................................................................... 40
Independence ................................................................................................................................ 41
Discrete Random Variables ............................................................................................................... 41
Random Variables.......................................................................................................................... 41
Probability Distribution for a Discrete Random Variable .............................................................. 42
Probability ..................................................................................................................................... 43
Cumulative Distribution ................................................................................................................ 43
Percentiles ..................................................................................................................................... 44
Summary Measures ....................................................................................................................... 44
Population Mean ....................................................................................................................... 44
Population Variance – Standard Deviation ............................................................................... 45
Binominal Distribution................................................................................................................... 45
Probability Distribution ............................................................................................................. 46
Expected Value (=Mean) and Variance ..................................................................................... 47
How the probability distribution looks like graphically............................................................. 47
Continuous Random Variable............................................................................................................ 47
Density ........................................................................................................................................... 47
Probability ..................................................................................................................................... 49
Cumulative Distribution ................................................................................................................ 49
Summary Measures ....................................................................................................................... 49
Continuous Distribution Models ................................................................................................... 51
Normal Distribution ................................................................................................................... 51
Parameter: ............................................................................................................................. 51
Probabilities:.......................................................................................................................... 51
Quantiles: .............................................................................................................................. 52
Normal Approximation: Normal Distribution for Binomial Distribution ................................... 52
Standard Normal Distribution ................................................................................................... 52
Linear Transformation & Combination for Discrete/continuous Random variables ........................ 54
Linear Transformation ................................................................................................................... 54
Linear critiions ............................................................................................................................... 54
Sums and Averages........................................................................................................................ 55
Expectation / Mean ................................................................................................................... 56
Variance ..................................................................................................................................... 56
3
, Distribution ................................................................................................................................ 57
Central Limit Theorem ................................................................................................................... 58
Conditions for applying the CLT ................................................................................................ 59
Foundation for Inference ...................................................................................................................... 60
Point Estimator .................................................................................................................................. 60
Confidence Interval ........................................................................................................................... 61
Ingredients..................................................................................................................................... 62
Determine the CI for mean µ ........................................................................................................ 62
Interpretation ................................................................................................................................ 63
Width of CI ..................................................................................................................................... 63
Assumptions .................................................................................................................................. 64
Basics of Hypothesis Tests ................................................................................................................. 64
All Steps ......................................................................................................................................... 65
Formulate hypotheses............................................................................................................... 65
Suitable Descriptive Statistics.................................................................................................... 66
P-value ....................................................................................................................................... 67
Conclusion ................................................................................................................................. 67
Assumptions .............................................................................................................................. 67
Two-Sided Test .......................................................................................................................... 67
Two-Sided Test with Confidence Intervals ............................................................................ 68
Right-Sided Test......................................................................................................................... 68
Critical Values and Power (Type 2 error)................................................................................... 68
Hypothesis Testing ................................................................................................................................ 70
H1Inference about means and correlations...................................................................................... 70
Hypothesis testing for one mean (sigma unknown)...................................................................... 70
Critical Value(s) ...................................................................................................................... 71
Confidence Interval ............................................................................................................... 72
SPSS ....................................................................................................................................... 72
Testing Normality .......................................................................................................................... 72
Kolmogorov-Smirnov test.......................................................................................................... 73
Shapiro-Wilk .............................................................................................................................. 75
Hypothesis testing to compare two means................................................................................... 75
Paired versus Unpaired ............................................................................................................. 75
Paired T-test → turns into t-test for one mean ........................................................................ 76
Confidence Interval ............................................................................................................... 77
Unpaired T-test → compares two means ................................................................................. 77
4
, (1) Test Statistics (known sigma): .......................................................................................... 78
(2) Test Statistics (unknown sigma):...................................................................................... 78
(3) Test statistic for (unknown) σ1= σ2:.................................................................................. 78
T-test to compare 2 means in SPSS ....................................................................................... 79
Confidence Intervals (unpaired means) ................................................................................ 79
Hypothesis Testing Compare More Than 2 Means – ANOVA ................................................... 80
Post-Hoc Test......................................................................................................................... 83
T-Test for Pearson Correlation .................................................................................................. 85
H2 Chi Square Test for goodness of Fit and Independence .............................................................. 87
Goodness Of Fit Test (Chi-Test) ..................................................................................................... 87
Independence (Chi Square Test) ................................................................................................... 90
5 Simple Linear Regression................................................................................................................ 93
SR_A Model Specification.............................................................................................................. 93
Assumptions .............................................................................................................................. 95
Nonlinear relationships ............................................................................................................. 96
Linear Model.......................................................................................................................... 98
Polynomial model: quadratic ................................................................................................ 98
Polynomial model: cubic ....................................................................................................... 98
Log transformation ................................................................................................................ 98
Log Linear Model ................................................................................................................... 99
Linear Log Model ................................................................................................................... 99
Log Log Model ....................................................................................................................... 99
Dummy Variable .................................................................................................................... 99
SR_B Inference about the regression parameters ...................................................................... 100
Fitted Model ............................................................................................................................ 100
Population vs. Estimated model .......................................................................................... 100
Least Squares Principles ...................................................................................................... 101
Residuals .............................................................................................................................. 101
Calculation LS (least squared) estimates ............................................................................. 102
Interpretation ...................................................................................................................... 102
Least Squares Prediction ..................................................................................................... 102
Estimate standard error of the model σ.............................................................................. 102
Properties of LSE...................................................................................................................... 103
Bi as a random variable ....................................................................................................... 103
Expected Value .................................................................................................................... 104
Variance ............................................................................................................................... 104
5
, Standard Error of the Parameters ....................................................................................... 104
Density ................................................................................................................................. 104
Gauss Markov theorem ....................................................................................................... 105
Interval estimation and hypothesis testing ............................................................................. 105
Confidence Interval for β1 ................................................................................................... 105
Hypothesis Testing .............................................................................................................. 106
SR_C Goodness of Fit ................................................................................................................... 108
Decomposition of variance and R-square ............................................................................... 108
Measuring Goodness of Fit.................................................................................................. 108
SR_D Checking Assumptions ....................................................................................................... 111
Assumption 1 ........................................................................................................................... 111
Assumption A2 ........................................................................................................................ 112
Assumption A3 ........................................................................................................................ 112
Assumption A4 ........................................................................................................................ 113
Assumption A5 ........................................................................................................................ 113
6
