Research Methods pt2
Contents
Reliability & Validity
Maths content
Stats testing
Probability
Sections of research
Features of science
Peer review
,Reliability and validity
Validity
What is validity
Reliability can be used to test validity, a test cannot be valid but not reliable – if the test is
valid it will repeat the same result.
Internal validity if what is being measured is what is intended to be measured
What affects internal validity:
- Operationalisation of variables
- Confounding variables (DV affected by other variables)
- Demand characteristics
- Investigator bias
- Participant reactivity (evaluation apprehension / screw you effect)
Control improves internal validity; it removes confounding variables. Participant reactivity
also affects – the single-blind technique and deception can be used. Experimental bias can be
combated by the double-blind technique.
External validity how much results can be generalised outside the experimental setting
Types:
- Ecological
o More realistic experiment
o Balance between realism (external validity) and control (internal validity)
- Population
o Representative sample
o Ethical conflict – can’t force people to take part and so may always be biased
- Temporal
Assessing validity
1. Face validity
2. Concurrent validity
Face validity:
A simple form of validity where the researcher applies a subjective judgement of whether a
study or test measures what it claims to measure (logical validity).
It is based solely on the appearance of the measure and what it is supposed to measure – not
what the test actually measures.
It is the easiest form of validity to apply to research, it doesn’t include much in the way of
objective measurements – weakest form of validity.
Refers to how a variable has been operationalised. Researchers often ask an expert in the field
who reports their judgement of whether the test / study measures what it is meant to.
, Concurrent validity:
Amount of agreement between 2 different tests and measures.
A new test / measure is compared with an existing test / measure of the same behaviour and
nature to see if the measures produce similar results. If it does, and the scores form a strong,
positive correlation (+0.8), the new test is said to have concurrent validity
What is reliability?
Psychologists use scientific methods in their research investigations to try and design studies
that can be replicated or repeated – this is a feature of science.
Reliability can be improved by greater accuracy of measurement:
1. Pilot studies
2. Taking more than one measurement
3. Standardising measurement
Internal:
- Within a test - if both halves are consistent
External:
- Same results after a period of time, using same test / investigation
Assessing reliability
Split-half method (internal)
Test items divided into half and the scores on both halves are compared. Scores should be
similar, have a positive correlation if the test is reliable.
Inter-observer reliability (internal – observations)
Compare observations of different observations using a behavioural checklist to standardise
categories. Behaviour observed independently, observations compared and a correlational
analysis conducted.
Inter-rater reliability (internal)
Can be tested using statistical test Spearman’s Rho. This measures the strength of the
correlation between two variables – r = 1 is a perfect, positive correlation whilst r = -1 is a
perfect, negative correlation. The two observers would have their observations plotted on a
scatter graph.
Inter-rater reliability could also be established by having two judges judge the same test
independently and similar results would suggest inter-rater reliability, whilst different results
shows a lack of inter-rater reliability.
Test re-test (external)
Compare the results of the test at one time to a test taken a different time (w similar / the
same pts). Results should be similar if there is external reliability. Commonly applied when
assessing reliability of questionnaire, psychological tests (e.g., of personality / IQ) and
interviews. There must be sufficient time to be sure that pts is not simply recalling their
Contents
Reliability & Validity
Maths content
Stats testing
Probability
Sections of research
Features of science
Peer review
,Reliability and validity
Validity
What is validity
Reliability can be used to test validity, a test cannot be valid but not reliable – if the test is
valid it will repeat the same result.
Internal validity if what is being measured is what is intended to be measured
What affects internal validity:
- Operationalisation of variables
- Confounding variables (DV affected by other variables)
- Demand characteristics
- Investigator bias
- Participant reactivity (evaluation apprehension / screw you effect)
Control improves internal validity; it removes confounding variables. Participant reactivity
also affects – the single-blind technique and deception can be used. Experimental bias can be
combated by the double-blind technique.
External validity how much results can be generalised outside the experimental setting
Types:
- Ecological
o More realistic experiment
o Balance between realism (external validity) and control (internal validity)
- Population
o Representative sample
o Ethical conflict – can’t force people to take part and so may always be biased
- Temporal
Assessing validity
1. Face validity
2. Concurrent validity
Face validity:
A simple form of validity where the researcher applies a subjective judgement of whether a
study or test measures what it claims to measure (logical validity).
It is based solely on the appearance of the measure and what it is supposed to measure – not
what the test actually measures.
It is the easiest form of validity to apply to research, it doesn’t include much in the way of
objective measurements – weakest form of validity.
Refers to how a variable has been operationalised. Researchers often ask an expert in the field
who reports their judgement of whether the test / study measures what it is meant to.
, Concurrent validity:
Amount of agreement between 2 different tests and measures.
A new test / measure is compared with an existing test / measure of the same behaviour and
nature to see if the measures produce similar results. If it does, and the scores form a strong,
positive correlation (+0.8), the new test is said to have concurrent validity
What is reliability?
Psychologists use scientific methods in their research investigations to try and design studies
that can be replicated or repeated – this is a feature of science.
Reliability can be improved by greater accuracy of measurement:
1. Pilot studies
2. Taking more than one measurement
3. Standardising measurement
Internal:
- Within a test - if both halves are consistent
External:
- Same results after a period of time, using same test / investigation
Assessing reliability
Split-half method (internal)
Test items divided into half and the scores on both halves are compared. Scores should be
similar, have a positive correlation if the test is reliable.
Inter-observer reliability (internal – observations)
Compare observations of different observations using a behavioural checklist to standardise
categories. Behaviour observed independently, observations compared and a correlational
analysis conducted.
Inter-rater reliability (internal)
Can be tested using statistical test Spearman’s Rho. This measures the strength of the
correlation between two variables – r = 1 is a perfect, positive correlation whilst r = -1 is a
perfect, negative correlation. The two observers would have their observations plotted on a
scatter graph.
Inter-rater reliability could also be established by having two judges judge the same test
independently and similar results would suggest inter-rater reliability, whilst different results
shows a lack of inter-rater reliability.
Test re-test (external)
Compare the results of the test at one time to a test taken a different time (w similar / the
same pts). Results should be similar if there is external reliability. Commonly applied when
assessing reliability of questionnaire, psychological tests (e.g., of personality / IQ) and
interviews. There must be sufficient time to be sure that pts is not simply recalling their
