STATISTICS II
Course 19407 – Bachelors in Management and Technology
Fernando Alfayate Fernández (100498308)
Pablo Rodríguez Martos (100498180)
,Tabla de contenido
1. INFERENCE IN ONE POPULATION ............................................................................................................................2
1.1 INTRODUCTION: STATISTICAL INFERENCE ......................................................................................................2
1.2 POINT ESTIMATORS ........................................................................................................................................2
1.3 ESTIMATION USING CONFIDENCE INTERVALS ................................................................................................3
1.4 SUMMARY TABLE ............................................................................................................................................9
2. HYPOTHESIS TESTING IN ONE POPULATION ..........................................................................................................10
2.1 INTRODUCTION, THE NULL AND ALTERNATIVE HYPOTHESES .......................................................................10
2.2 HYPOTHESIS TESTING PROCESS.....................................................................................................................10
2.3 TYPE I AND TYPE II ERRORS, POWER .............................................................................................................11
2.4 TEST STATISTIC, LEVEL OF SIGNIFICANCE AND REJECTION/ACCEPTANCE REGIONS IN UPPER-, LOWER- AND
TWO-TAIL TESTS ........................................................................................................................................................12
2.5 PROCEDUE FOR THE HYPOTHESIS TESTING...................................................................................................13
2.6 P-VALUE ........................................................................................................................................................15
2.7 TWO-TAIL TESTS AND CONFIDENCE INTERVALS ...........................................................................................17
2.8 EXAMPLES WITH VARIOUS PARAMETERS .....................................................................................................17
2.9 POWER AND SAMPLE SIZE CALCULATIONS ...................................................................................................17
3. COMPARING TWO POPULATIONS .........................................................................................................................19
3.1 HYPOTHESIS FOR THE DIFFERENCE BETWEEN TWO POPULATION MEANS: MATCHED PAIRS ......................19
3.2 HYPOTHESIS FOR THE DIFFERENCE BETWEEN TWO POPULATION MEANS: INDEPENDENT SAMPLES .........20
3.3 HYPOTHESIS FOR THE RATIO OF TWO POPULATION VARIANCES: INDEPENDENT SAMPLES ........................24
4. SIMPLE LINEAR REGRESSION .................................................................................................................................27
4.1 INTRODUCTION .............................................................................................................................................27
4.2 MEASURES OF LINEAR DEPENDENCE ............................................................................................................28
4.3 SIMPLE LINEAR REGRESSION MODEL ............................................................................................................28
4.4 STATISTICAL INFERENCE IN SIMPLE LINEAR REGRESSION MODEL ................................................................29
4.5 AVERAGE AND INDIVIDUAL PREDICTIONS ....................................................................................................32
4.6 SCALE CHANGES IN THE SIMPLE REGRESSION MODEL .................................................................................33
4.7 REGRESSION LINE: R-SQUARED AND VARIABILITY DECOMPOSITION ...........................................................33
5. COMPLEX LINEAR REGRESSION .............................................................................................................................34
5.1 REGRESSION DIAGNOSTICS & RESIDUAL ANALYSIS.......................................................................................34
5.2 ANOVA DECOMPOSITION..............................................................................................................................35
5.3 NONLINEAR RELATIONSHIPS & LINEARIZING TRANSFORMATION ................................................................36
5.4 MATRIX TREATMENT OF LINEAR REGRESSION .............................................................................................36
5.5 MULTIPLE LINEAR REGRESSION ....................................................................................................................37
STATISTICS II
,1. INFERENCE IN ONE POPULATION
1.1 INTRODUCTION: STATISTICAL INFERENCE
In the first part of the course, we undertake situations that require to estimate a population
parameter. We are going to discuss procedures to estimate mean and variance of a population,
and later we will introduce the concept of confidence intervals.
• Population: group of individuals or objects that share a common characteristic. In statistics,
a definition can be: “set of all items that interest an investigator” (Newbold, 2013).
Population size is commonly described with 𝑁 However, a Sample is an observed portion
of a population with sample 𝑛, and its used to make inferences about the population.
A Parameter is a numerical value that describes a characteristic of a population.
“An Estimator of a population parameter is a random variable that depends on the sample
information; its value provides approximations of this unknown parameter. A specific value of that
random variable is called an estimate” (Newbold, 2013).
Other relevant concepts are Confidence Interval, which is a range of values likely to contain the
true value of a population parameter with a certain confidence level, and Sampling distribution,
defined as the distribution of sample statistics that would be obtained if all possible samples of n
size were taken from a population.
1.2 POINT ESTIMATORS
Now we have learned the basics, we must be aware that the value of the population parameter will
always be unknown, and that is why our objective by sampling is to estimate its value.
“Any inference drawn about the population will be based on sample statistics. The choice of
appropriate statistics will depend on which population parameter is of interest.” (Newbold, 2013).
A point estimator of a population parameter is a function, of the sample information that yields a
single number, called point estimate.
POPULATION ESTIMATOR ESTIMATE
𝑻(𝑿𝒏 )
PARAMETER NOTATION NOTATION
#! $⋯$#"
Mean 𝝁𝑿 Sample mean &
𝑋' = 𝜇̂ # 𝑥̅
Proportion 𝒑𝑿 Sample proportion 𝑝̂ # 𝑝̂'
∑ ##$ *&(#,)$
Sample variance 𝜎1#. 𝜎1'.
&
Variance 𝝈𝟐𝑿
∑ ##$ *&(#,)$ &
Sample quasi variance = 𝜎1 . 𝑠#. 𝑠'.
&*/ &*/ #
… … … …
In general, 𝜽𝑿 … 𝜃5# 𝜃5'
STATISTICS II
, AN IMPORTANT REMARK ABOUT POINT ESTIMATORS IS THAT WE EVALUATE THEM BASED ON THREE
IMPORTANT ASPECTS : UNBIASEDNESS , EFFICIENCY , AND THE MEAN SQUARED ERROR .
A point estimator is an unbiased estimator of a population parameter if its expected value is equal
to that parameter it is estimating. Also, we can think of the expected value of 𝜃5 as the average of 𝜃$
values for all possible samples.
That means that, if the procedure of sampling is repeated infinite times, then, on average, the value
obtained for an unbiased estimator will be equal to the population parameter.
Thinking of bias in mathematical terms:
𝐵𝑖𝑎𝑠[𝜃$ 𝑋 ] = 𝐸[𝜃$ 𝑋 ] − 𝜃𝑋
Unbiasedness does not mean that a particular value of the sample estimator must be exactly the
correct value of the population estimate, but that “an unbiased estimator has the capability of
estimating the population parameter correctly on average”. (Newbold, 2013)
POPULATION ESTIMATOR MINIMUM VARIANCE
BIAS UNBIASED?
PARAMETER 𝑻(𝑿𝒏 ) UNBIASED ESTIMATOR?
Mean 𝝁𝑿 𝑋' 𝐸[𝑋'] − 𝜇' = 0 Yes Yes, if 𝑋 normal
Proportion 𝒑𝑿 𝑝̂# 𝐸[𝑝̂# ] − 𝑝' = 0 Yes Yes
𝜎1#. 𝐸[𝜎1#. ] − 𝜎#. ≠ 0 No No
Variance 𝝈𝟐𝑿
𝑠#. 𝐸[𝑠#. ] − 𝜎#. ≠ 0 Yes Yes, if 𝑋 normal
In general, 𝜽𝑿 <𝑿
𝜽 𝐸=𝜃5# > − 𝜃# Often Rarely
Efficiency is measured by the estimator´s variance. Estimators with smaller variance are more
efficient. Relative efficiency of two unbiased estimators is:
$ 𝑋,1 ]
𝑉𝑎𝑟[𝜃
𝑅𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑒𝑓𝑓𝑖𝑐𝑖𝑒𝑛𝑐𝑦 4𝜃$ 𝑋,1 ; 𝜃$ 𝑋,2 5 =
$ 𝑋,2 ]
𝑉𝑎𝑟[𝜃
A general criterion to select estimators is the Mean Squared Error, defined mathematically as:
𝑀𝑆𝐸[𝜃$ 𝑋 ] = 𝐸[(𝜃$ 𝑋 − 𝜃𝑋 )2 ] = 𝑉𝑎𝑟[𝜃$ 𝑋 ] + (𝐵𝑖𝑎𝑠[𝜃$ 𝑋 ])2
Important remarks
1. The mean squared error of an unbiased estimator equals its variance.
2. An estimator with smaller MSE is better
3. The minimum variance unbiased estimator has the smallest variance/MSE among all
1.3 ESTIMATION USING CONFIDENCE INTERVALS
So far, we have considered the point estimation of an unknown population parameter which,
assuming we had an SRS sample of 𝑛 observations from 𝑋, would produce an educated guess about
that unknown parameter.
Point estimates, however, do not consider the variability.
STATISTICS II
Course 19407 – Bachelors in Management and Technology
Fernando Alfayate Fernández (100498308)
Pablo Rodríguez Martos (100498180)
,Tabla de contenido
1. INFERENCE IN ONE POPULATION ............................................................................................................................2
1.1 INTRODUCTION: STATISTICAL INFERENCE ......................................................................................................2
1.2 POINT ESTIMATORS ........................................................................................................................................2
1.3 ESTIMATION USING CONFIDENCE INTERVALS ................................................................................................3
1.4 SUMMARY TABLE ............................................................................................................................................9
2. HYPOTHESIS TESTING IN ONE POPULATION ..........................................................................................................10
2.1 INTRODUCTION, THE NULL AND ALTERNATIVE HYPOTHESES .......................................................................10
2.2 HYPOTHESIS TESTING PROCESS.....................................................................................................................10
2.3 TYPE I AND TYPE II ERRORS, POWER .............................................................................................................11
2.4 TEST STATISTIC, LEVEL OF SIGNIFICANCE AND REJECTION/ACCEPTANCE REGIONS IN UPPER-, LOWER- AND
TWO-TAIL TESTS ........................................................................................................................................................12
2.5 PROCEDUE FOR THE HYPOTHESIS TESTING...................................................................................................13
2.6 P-VALUE ........................................................................................................................................................15
2.7 TWO-TAIL TESTS AND CONFIDENCE INTERVALS ...........................................................................................17
2.8 EXAMPLES WITH VARIOUS PARAMETERS .....................................................................................................17
2.9 POWER AND SAMPLE SIZE CALCULATIONS ...................................................................................................17
3. COMPARING TWO POPULATIONS .........................................................................................................................19
3.1 HYPOTHESIS FOR THE DIFFERENCE BETWEEN TWO POPULATION MEANS: MATCHED PAIRS ......................19
3.2 HYPOTHESIS FOR THE DIFFERENCE BETWEEN TWO POPULATION MEANS: INDEPENDENT SAMPLES .........20
3.3 HYPOTHESIS FOR THE RATIO OF TWO POPULATION VARIANCES: INDEPENDENT SAMPLES ........................24
4. SIMPLE LINEAR REGRESSION .................................................................................................................................27
4.1 INTRODUCTION .............................................................................................................................................27
4.2 MEASURES OF LINEAR DEPENDENCE ............................................................................................................28
4.3 SIMPLE LINEAR REGRESSION MODEL ............................................................................................................28
4.4 STATISTICAL INFERENCE IN SIMPLE LINEAR REGRESSION MODEL ................................................................29
4.5 AVERAGE AND INDIVIDUAL PREDICTIONS ....................................................................................................32
4.6 SCALE CHANGES IN THE SIMPLE REGRESSION MODEL .................................................................................33
4.7 REGRESSION LINE: R-SQUARED AND VARIABILITY DECOMPOSITION ...........................................................33
5. COMPLEX LINEAR REGRESSION .............................................................................................................................34
5.1 REGRESSION DIAGNOSTICS & RESIDUAL ANALYSIS.......................................................................................34
5.2 ANOVA DECOMPOSITION..............................................................................................................................35
5.3 NONLINEAR RELATIONSHIPS & LINEARIZING TRANSFORMATION ................................................................36
5.4 MATRIX TREATMENT OF LINEAR REGRESSION .............................................................................................36
5.5 MULTIPLE LINEAR REGRESSION ....................................................................................................................37
STATISTICS II
,1. INFERENCE IN ONE POPULATION
1.1 INTRODUCTION: STATISTICAL INFERENCE
In the first part of the course, we undertake situations that require to estimate a population
parameter. We are going to discuss procedures to estimate mean and variance of a population,
and later we will introduce the concept of confidence intervals.
• Population: group of individuals or objects that share a common characteristic. In statistics,
a definition can be: “set of all items that interest an investigator” (Newbold, 2013).
Population size is commonly described with 𝑁 However, a Sample is an observed portion
of a population with sample 𝑛, and its used to make inferences about the population.
A Parameter is a numerical value that describes a characteristic of a population.
“An Estimator of a population parameter is a random variable that depends on the sample
information; its value provides approximations of this unknown parameter. A specific value of that
random variable is called an estimate” (Newbold, 2013).
Other relevant concepts are Confidence Interval, which is a range of values likely to contain the
true value of a population parameter with a certain confidence level, and Sampling distribution,
defined as the distribution of sample statistics that would be obtained if all possible samples of n
size were taken from a population.
1.2 POINT ESTIMATORS
Now we have learned the basics, we must be aware that the value of the population parameter will
always be unknown, and that is why our objective by sampling is to estimate its value.
“Any inference drawn about the population will be based on sample statistics. The choice of
appropriate statistics will depend on which population parameter is of interest.” (Newbold, 2013).
A point estimator of a population parameter is a function, of the sample information that yields a
single number, called point estimate.
POPULATION ESTIMATOR ESTIMATE
𝑻(𝑿𝒏 )
PARAMETER NOTATION NOTATION
#! $⋯$#"
Mean 𝝁𝑿 Sample mean &
𝑋' = 𝜇̂ # 𝑥̅
Proportion 𝒑𝑿 Sample proportion 𝑝̂ # 𝑝̂'
∑ ##$ *&(#,)$
Sample variance 𝜎1#. 𝜎1'.
&
Variance 𝝈𝟐𝑿
∑ ##$ *&(#,)$ &
Sample quasi variance = 𝜎1 . 𝑠#. 𝑠'.
&*/ &*/ #
… … … …
In general, 𝜽𝑿 … 𝜃5# 𝜃5'
STATISTICS II
, AN IMPORTANT REMARK ABOUT POINT ESTIMATORS IS THAT WE EVALUATE THEM BASED ON THREE
IMPORTANT ASPECTS : UNBIASEDNESS , EFFICIENCY , AND THE MEAN SQUARED ERROR .
A point estimator is an unbiased estimator of a population parameter if its expected value is equal
to that parameter it is estimating. Also, we can think of the expected value of 𝜃5 as the average of 𝜃$
values for all possible samples.
That means that, if the procedure of sampling is repeated infinite times, then, on average, the value
obtained for an unbiased estimator will be equal to the population parameter.
Thinking of bias in mathematical terms:
𝐵𝑖𝑎𝑠[𝜃$ 𝑋 ] = 𝐸[𝜃$ 𝑋 ] − 𝜃𝑋
Unbiasedness does not mean that a particular value of the sample estimator must be exactly the
correct value of the population estimate, but that “an unbiased estimator has the capability of
estimating the population parameter correctly on average”. (Newbold, 2013)
POPULATION ESTIMATOR MINIMUM VARIANCE
BIAS UNBIASED?
PARAMETER 𝑻(𝑿𝒏 ) UNBIASED ESTIMATOR?
Mean 𝝁𝑿 𝑋' 𝐸[𝑋'] − 𝜇' = 0 Yes Yes, if 𝑋 normal
Proportion 𝒑𝑿 𝑝̂# 𝐸[𝑝̂# ] − 𝑝' = 0 Yes Yes
𝜎1#. 𝐸[𝜎1#. ] − 𝜎#. ≠ 0 No No
Variance 𝝈𝟐𝑿
𝑠#. 𝐸[𝑠#. ] − 𝜎#. ≠ 0 Yes Yes, if 𝑋 normal
In general, 𝜽𝑿 <𝑿
𝜽 𝐸=𝜃5# > − 𝜃# Often Rarely
Efficiency is measured by the estimator´s variance. Estimators with smaller variance are more
efficient. Relative efficiency of two unbiased estimators is:
$ 𝑋,1 ]
𝑉𝑎𝑟[𝜃
𝑅𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑒𝑓𝑓𝑖𝑐𝑖𝑒𝑛𝑐𝑦 4𝜃$ 𝑋,1 ; 𝜃$ 𝑋,2 5 =
$ 𝑋,2 ]
𝑉𝑎𝑟[𝜃
A general criterion to select estimators is the Mean Squared Error, defined mathematically as:
𝑀𝑆𝐸[𝜃$ 𝑋 ] = 𝐸[(𝜃$ 𝑋 − 𝜃𝑋 )2 ] = 𝑉𝑎𝑟[𝜃$ 𝑋 ] + (𝐵𝑖𝑎𝑠[𝜃$ 𝑋 ])2
Important remarks
1. The mean squared error of an unbiased estimator equals its variance.
2. An estimator with smaller MSE is better
3. The minimum variance unbiased estimator has the smallest variance/MSE among all
1.3 ESTIMATION USING CONFIDENCE INTERVALS
So far, we have considered the point estimation of an unknown population parameter which,
assuming we had an SRS sample of 𝑛 observations from 𝑋, would produce an educated guess about
that unknown parameter.
Point estimates, however, do not consider the variability.
STATISTICS II
