RMHI PSYC30013 Research Methods for Human Inquiry
Summary Research (everything you need to know) 2026 update
University of Melbourne
, Frequentist: coin has probability T-test one sample One way ANOVA Correlation
50% = 50% long run proportion of Comparing MORE THAN 2 MEANS Measure strength and direction (-1,1)
heads is 50% SS is high if difference between group means and global mean is Pearson assumes linearity
Probability is a long run frequency high and vice versa. The global mean indicated by X bar Spearman's rho assume monotonicity (always going up or down)
Probability is objective, only In R, have to specify: method="spearman"
applies to repeatable events, it is Regression
in the world Model acknowledges existence of random variation in data
Bayesian: coin 50% = 50% degree Regression minimises the sum squared deviation (least squares) between
of belief the coin is heads (Y-Y hat prediction) - same as ANOVA within groups sum of squares
Probability is a degree of belief - (residuals)
what would a rational agent think
Interaction: situation where relationship b/w one predictor and outcome
Probability is subjective, beliefs of
depends on the nature of the other predictor
rational agents, applies to
anything you can believe in T-stat and p-value are the same because underlying calculation when
Null Hypothesis testing a correlation for significance is nearly identical
Fisher: hypothesis testing is about Adding variables means the variance in different models are explained by
trying to falsify a single hypothesis the new ones unless the two had 0 relationship
P is the Type I error rate you are
willing to tolerate if you want to
reject H0
Neyman: hypothesis testing is
about choosing between two rival
hypotheses
P is the probability (if H0 is true) of
observing a test statistic at least
as extreme as the one that was
found (how likely you are to
observe that test stat given
everything else - if it is high then
we reject the null)
Both were frequentists and neither
would agree with method used F = MS (SSb corrected for df)/MSw (SSw corrected for df)
today
Research hypotheses: claims on
psychological construct e.g.,
people agree with each other
about whether there are food
shortages
Statistical hypotheses - null and
alternate: claims about population
parameters e.g., probability that
people agree = 0.5
Type I error: false positive (null is
true but is rejected - incorrectly
believe in H1 alternative
hypothesis)
Alpha/significance level is how
much type I error is accepted
Type II error: false negative (null is
false but is accepted)
Power = 1-Type II error
Lower alpha = lower power
(positively related) because alpha Higher df (from higher N) results in more normal t-dist
and type II error are inversely If null is true, t-stat should be close to 0 (aka if t-stat is larger is more likely
related to be significant - depends on sample size)
Chi-squared Goodness of Fit Effect size: Cohen's d - used for normality. 0.2=small, 0.5=medium,
Chi-squared test used for
0.8=large
categorical data, outcome variable
One sided test, H1: true mean is ABOVE 50 etc. only care about one side of
is nominal. Goodness of Fit
distirbution, if = use two sided test
compared observed frequency of
one variable against hypothesis of Independent t-test SS tot (total variability) = SSb+SSw, is NOT test statistic. It is the
true probabilities Comparing two groups - therefore t-stat based on difference between relationship between them - the ratio corrected for degrees of
Larger X^2 test statistic mean group means freedom
worse fit to the data Student t-test: assume both samples have same variance Reject null if F value really high
The X^2 is the difference between Welch t-test: not assume same variance
observed and expected squared Background assumptions: normal population distribution, independent
divided by expected ((O-E)^2/E) observations, homoscedasticity - groups have same sd
Karl Pearson: Chi-squared Assumption checking: need to check for normality for EACH GROUP Standardised coefficients required when variables are too different from
distribution: what you get when Below for student t-test: each other so we can compare raw regression coefficients.
you take normally distributed Unstandardised coefficients are useful for interpreting the slopes of the
data, square it and add it regression line.
Degrees of freedom: # of things
you're interested in - # of Linearity
constraints
If linear, residual plots should sit around 0
X^2 = k-1 df
Residual(). If p<.05 variable is not linear.
Regression assume residuals are normal (check by QQ or Shapiro Wilk)
Effect size means "x" explains z% of the variance in y
ANOVA ASSUMPTIONS AND CORRECTIONS
Running too many t-test is bad because you increase the
probability of having Type 1 error occurring within at least one Outliers
of your separate tests High leverage: observation with different values on predictors than others
To solve this run corrections on your p-value. Suggested to use (residual may still be small)
Holm and then Bonferroni if you can't use Holm. Important to Outlier: large residual from model (usually model fares well despite this)
make some correction. High influence: outlier + high leverage
Family-wise Type I error: probability of obtaining at least one
Type I error across multiple tests
Chi-squared test of
independence
Tests if two nominal variables are
related to each other, still
comparing observed vs expected,
you just do the calculations for R- In R it will use welch, unless you set var.equal=TRUE which indicates
rows, and C-columns student t-test
Observed - Expected (aka raw Paired samples t-test
residuals) Changes within individuals (not groups) - interest in difference scores. Two
Df=(rows-1)(columns-1) means within repeated tests (e.g., tests differences over time). Run a
This is still chi- down slope only, the df normal one sample t-test on the differences between two sets of data
makes it look like this shape. A lower df (e.g., CHANGE in the data). Exactly same as one-sample t-test but on
shifts the graph to the left. The df is the difference of scores with null hypothesis being mean difference = 0. (no
number of values we can move around change) Leverage can be quantified using hat value h which measures extent to
afterwards. A lower df means less numbers Assumption test: check for normality of DIFFERENCE VARIABLE which i-th observation "controls" the regression line
can be moved, its therefore less likely to Assumption TESTING
see your numbers All t-test: normal population distribution + independent data unless
Adjusted/standardised residuals / pearson paired. Student t-test: homogeneity of variance
residuals are raw residuals divided by root Normality: QQ plot - qqnorm() straight line imply normality
E. Normality: Shapiro-Wilk - W test statistic. Values W<1 imply deviations
If you have a signficant test and any from normality. Reject null: data not normal
adjusted residuals are extreme than +/- Problem: frequently significant if sample size >50 even if distribution is
1.96, those individual items are significant normal
as well Solve: look at QQ plot and histogram
Effect size – Cramer V Non-normal data - Wilcoxon test
Measures how "big" the difference Wilcoxon is a non-parametric test (avoids assumptions about distribution
between the data and null hypothesis shape) Not as powerful (higher Type II error). It counts the # of times a
predictions actually were score from group A is > score from group B. Half the possibilities should be
V=0~0.1 (negligible), 0.1~0.3 (weak larger if H0 is true.
association), 0.3~0.5 (moderate Interpret: 0.1-0.3 (small), 0.3-0.5 (medium), >0.5 (large)
association), 0.5~1 (high association)
Assumptions - Chi-squared
Large expected frequencies: there are Post-hoc test: test conducted after ANOVA for which you don’t
enough observations for the underlying have any particular hypotheses e.g., pairwise t-test run with no Collinearity
binomial distributions to be normal. particular plan in mind Predictors are highly correlated
Expected frequencies > 5 is safe. Multiple test correction: control overall Type I error rate e.g. Variance Inflation Factor (VIF) captures how badly correlation messes up
Fisher Exact test: calculates exact Bonferroni, Holm. When running post-hoc you apply multiple CI around coefficients
probability of obtaining a particular test correction
contingency table (p-value = sum Assumptions of ANOVA
contingency tables that are more Residuals (within group variance) are normally distributed check
In vote counting we see they are all insignificant for five studies of the
extreme/uneven than the observed one) with Shapiro-Wilk, if violated use Kruskal-Wallis
same results (but maybe different # of participants etc.)
Homogeneity of variance: check with Levene’s test, if violated
Summary Research (everything you need to know) 2026 update
University of Melbourne
, Frequentist: coin has probability T-test one sample One way ANOVA Correlation
50% = 50% long run proportion of Comparing MORE THAN 2 MEANS Measure strength and direction (-1,1)
heads is 50% SS is high if difference between group means and global mean is Pearson assumes linearity
Probability is a long run frequency high and vice versa. The global mean indicated by X bar Spearman's rho assume monotonicity (always going up or down)
Probability is objective, only In R, have to specify: method="spearman"
applies to repeatable events, it is Regression
in the world Model acknowledges existence of random variation in data
Bayesian: coin 50% = 50% degree Regression minimises the sum squared deviation (least squares) between
of belief the coin is heads (Y-Y hat prediction) - same as ANOVA within groups sum of squares
Probability is a degree of belief - (residuals)
what would a rational agent think
Interaction: situation where relationship b/w one predictor and outcome
Probability is subjective, beliefs of
depends on the nature of the other predictor
rational agents, applies to
anything you can believe in T-stat and p-value are the same because underlying calculation when
Null Hypothesis testing a correlation for significance is nearly identical
Fisher: hypothesis testing is about Adding variables means the variance in different models are explained by
trying to falsify a single hypothesis the new ones unless the two had 0 relationship
P is the Type I error rate you are
willing to tolerate if you want to
reject H0
Neyman: hypothesis testing is
about choosing between two rival
hypotheses
P is the probability (if H0 is true) of
observing a test statistic at least
as extreme as the one that was
found (how likely you are to
observe that test stat given
everything else - if it is high then
we reject the null)
Both were frequentists and neither
would agree with method used F = MS (SSb corrected for df)/MSw (SSw corrected for df)
today
Research hypotheses: claims on
psychological construct e.g.,
people agree with each other
about whether there are food
shortages
Statistical hypotheses - null and
alternate: claims about population
parameters e.g., probability that
people agree = 0.5
Type I error: false positive (null is
true but is rejected - incorrectly
believe in H1 alternative
hypothesis)
Alpha/significance level is how
much type I error is accepted
Type II error: false negative (null is
false but is accepted)
Power = 1-Type II error
Lower alpha = lower power
(positively related) because alpha Higher df (from higher N) results in more normal t-dist
and type II error are inversely If null is true, t-stat should be close to 0 (aka if t-stat is larger is more likely
related to be significant - depends on sample size)
Chi-squared Goodness of Fit Effect size: Cohen's d - used for normality. 0.2=small, 0.5=medium,
Chi-squared test used for
0.8=large
categorical data, outcome variable
One sided test, H1: true mean is ABOVE 50 etc. only care about one side of
is nominal. Goodness of Fit
distirbution, if = use two sided test
compared observed frequency of
one variable against hypothesis of Independent t-test SS tot (total variability) = SSb+SSw, is NOT test statistic. It is the
true probabilities Comparing two groups - therefore t-stat based on difference between relationship between them - the ratio corrected for degrees of
Larger X^2 test statistic mean group means freedom
worse fit to the data Student t-test: assume both samples have same variance Reject null if F value really high
The X^2 is the difference between Welch t-test: not assume same variance
observed and expected squared Background assumptions: normal population distribution, independent
divided by expected ((O-E)^2/E) observations, homoscedasticity - groups have same sd
Karl Pearson: Chi-squared Assumption checking: need to check for normality for EACH GROUP Standardised coefficients required when variables are too different from
distribution: what you get when Below for student t-test: each other so we can compare raw regression coefficients.
you take normally distributed Unstandardised coefficients are useful for interpreting the slopes of the
data, square it and add it regression line.
Degrees of freedom: # of things
you're interested in - # of Linearity
constraints
If linear, residual plots should sit around 0
X^2 = k-1 df
Residual(). If p<.05 variable is not linear.
Regression assume residuals are normal (check by QQ or Shapiro Wilk)
Effect size means "x" explains z% of the variance in y
ANOVA ASSUMPTIONS AND CORRECTIONS
Running too many t-test is bad because you increase the
probability of having Type 1 error occurring within at least one Outliers
of your separate tests High leverage: observation with different values on predictors than others
To solve this run corrections on your p-value. Suggested to use (residual may still be small)
Holm and then Bonferroni if you can't use Holm. Important to Outlier: large residual from model (usually model fares well despite this)
make some correction. High influence: outlier + high leverage
Family-wise Type I error: probability of obtaining at least one
Type I error across multiple tests
Chi-squared test of
independence
Tests if two nominal variables are
related to each other, still
comparing observed vs expected,
you just do the calculations for R- In R it will use welch, unless you set var.equal=TRUE which indicates
rows, and C-columns student t-test
Observed - Expected (aka raw Paired samples t-test
residuals) Changes within individuals (not groups) - interest in difference scores. Two
Df=(rows-1)(columns-1) means within repeated tests (e.g., tests differences over time). Run a
This is still chi- down slope only, the df normal one sample t-test on the differences between two sets of data
makes it look like this shape. A lower df (e.g., CHANGE in the data). Exactly same as one-sample t-test but on
shifts the graph to the left. The df is the difference of scores with null hypothesis being mean difference = 0. (no
number of values we can move around change) Leverage can be quantified using hat value h which measures extent to
afterwards. A lower df means less numbers Assumption test: check for normality of DIFFERENCE VARIABLE which i-th observation "controls" the regression line
can be moved, its therefore less likely to Assumption TESTING
see your numbers All t-test: normal population distribution + independent data unless
Adjusted/standardised residuals / pearson paired. Student t-test: homogeneity of variance
residuals are raw residuals divided by root Normality: QQ plot - qqnorm() straight line imply normality
E. Normality: Shapiro-Wilk - W test statistic. Values W<1 imply deviations
If you have a signficant test and any from normality. Reject null: data not normal
adjusted residuals are extreme than +/- Problem: frequently significant if sample size >50 even if distribution is
1.96, those individual items are significant normal
as well Solve: look at QQ plot and histogram
Effect size – Cramer V Non-normal data - Wilcoxon test
Measures how "big" the difference Wilcoxon is a non-parametric test (avoids assumptions about distribution
between the data and null hypothesis shape) Not as powerful (higher Type II error). It counts the # of times a
predictions actually were score from group A is > score from group B. Half the possibilities should be
V=0~0.1 (negligible), 0.1~0.3 (weak larger if H0 is true.
association), 0.3~0.5 (moderate Interpret: 0.1-0.3 (small), 0.3-0.5 (medium), >0.5 (large)
association), 0.5~1 (high association)
Assumptions - Chi-squared
Large expected frequencies: there are Post-hoc test: test conducted after ANOVA for which you don’t
enough observations for the underlying have any particular hypotheses e.g., pairwise t-test run with no Collinearity
binomial distributions to be normal. particular plan in mind Predictors are highly correlated
Expected frequencies > 5 is safe. Multiple test correction: control overall Type I error rate e.g. Variance Inflation Factor (VIF) captures how badly correlation messes up
Fisher Exact test: calculates exact Bonferroni, Holm. When running post-hoc you apply multiple CI around coefficients
probability of obtaining a particular test correction
contingency table (p-value = sum Assumptions of ANOVA
contingency tables that are more Residuals (within group variance) are normally distributed check
In vote counting we see they are all insignificant for five studies of the
extreme/uneven than the observed one) with Shapiro-Wilk, if violated use Kruskal-Wallis
same results (but maybe different # of participants etc.)
Homogeneity of variance: check with Levene’s test, if violated
