Summary of class notes on Data Science Research Methods (JBM)

DATA SCIENCE
RESEARCH METHODS
LECTURE 1
Topics: course introduction, Scientific Method, Sample Size Determination, and ANOVA

p-value: highest significant value for which we accept H 0. If p<α → reject H 0.
Type I error α : reject H 0 when it is True.
Type II error β : accept H 0 when it is False.
The best combination of α and β is situation-specific.

One-Factor Design: Studies the impact of a single factor, Y =f ( X , ε ) for factor X on Y .
Replicated experiment: there is more than one data point at each level of the factor.
Replicates: number of rows, different values of Y .
Levels: number of columns, different levels of X .
Total outcomes: # replicates × # levels

Types of means:
 Column mean: Sum of all values in the column divided by the number of replicates.
 Grand mean: Sum of all data points divided by the total outcomes, RC ór sum of all column
means divided by the number of levels, C .

Least squares: optimal estimation that minimizes the sum of the squared differences.

Total Sum of Squares (TSS): sum of the squared difference between each data point and the grand mean.
Sum of Squares Between Columns (SSBc): sum of the squared difference between each column mean and
the grand mean, multiplied by R .
Sum of Squares Within Columns (SSW): sum of the squared difference between each data point in a
column and that column mean.

TSS=SS B c + SSW

If SSW ≈TSS → factor does not explain much.
If SSW /TSS ≈ 0 → factor has big influence.

MS=SSQ/df ; Mean square is the Sum of Squares divided by the degrees-of-freedom.
Unbiased estimate of population variance → use df instead of n .

E [ MSW ]=σ under constant variance assumption.
2


E [ MS Bc ]=σ 2 +V 2 with V col =[ R / ( C−1 ) ] ∙ ∑τ 2j .
E [ MS Bc ] ≠0 → true column means might not be equal, and sample error leads to difference in
column means.

F-statistic: way to find evidence of affects.
F calc >1 → evidence that V col ≠ 0 thus X affects Y . Evidence is not a final conclusion.

, F calc ≤1 → no evidence that X affects Y .
F calc ≫1 → reject H 0. Rejection means column means are different.
SLIDES LECTURE 1
Three goals of Data Science: Description, Prediction and Explanation.
Different types of Analytics:
1. Descriptive Analytics: insight into the past
2. Predictive Analytics: understanding the future
3. Prescriptive Analytics: advice on possible outcomes

Scientific Method: has an iterative nature.
Six Sigma: disciplined, data-driven methodology for process
improvement. Uses DMAIC cycle.

Key Insights:
 Identify the three data science goals.
 Scientific method is an iterative process.
 Not planning an experiment will not result in the wanted outcomes.
 Experiment can have more factors, that can have more than 2 levels.
 Six Sigma incorporated several aspects of the scientific method.

X−μ 0
Normal distribution gives test statistic T = if σ is known. Reject if |T |> z α / 2 or P H (|T|>|T 0|) <α .
σ /√ n 0




(
Confidence interval: reject if it doesn’t fall in the interval, x−z α /2
σ
√n
, x + z α /2
σ
√n ) .


Minimal sample sizes:
 Normal distribution:

( )
2
z α /2 σ
o ONE SAMPLE CASE: if σ is known → n ≥ with E maximal absolute error.
E
o ONE SAMPLE CASE: if σ is unknown → same as with known but use worst case σ .
 Round up to strictly satisfy the inequality.

( ) (σ + σ )
2
z α /2 2 2
o TWO SAMPLE CASE: equal sample sizes and variances known→ n ≥ 1 2
E

 Binomial distribution:

( )
2
zα/ 2
o ONE SAMPLE CASE: n ≥ ^p (1−^p )use worst case ^p or upper/lower bound
E
 Analyze p → p ( 1− p ) on [ 0,1 ]

( )
2
z
o TWO SAMPLE CASE: equal sample sizes n ≥ α/ 2 ( ^ p 1 ( 1− ^
p 1) + ^ p 2) )
p 2 ( 1− ^
E

Power analysis:
 Normal distribution H 0 : μ=μ0 :

o (
β=Φ z α / 2−
δ √n
σ ) (
−Φ −z α /2 −
δ √n
σ )
( )
2 2
δ √n (z +z ) σ
o Φ −z α/ 2− small compared to β → n ≈ α/ 2 β
σ δ
2