Improving reliability and validity
Reliability Measure of how consistent the findings from an investigation are.
Ways of assessing:
Internal reliability describes how consistent something is within itself.
- Split half method comparing results of one half of a test with the results from the
other half. These two forms of the same test are then done separately and should yield
the same score and have a correlation coefficient of more or equal to 0.8.
External reliability when consistent results are produced regardless of when the
investigation is used or who administers it.
Ways of assessing:
- Test-retest method researcher administers the same test on the same person on
different occasions (usually a week or 2 later). Results should be similar and lead a
correlation coefficient of more or equal to 0.8. must be sufficient time between test and
retest so ppts can't recall their answers but not too long as the persons attitude may
change.
- Inter observer reliability have 2 or more observers watching the same event or
sequence of events and record the data independently.
- Observers agree on interpretation of each behavioural category
- Tally charts compared to check for agreement
- extent to which there is agreement between two or more observers involved in
observing behaviour. Eliminating subjectivity bias and may either be carried out in a pilot
study or reported at the end of the study. (at least 0.8 correlation)
Measuring reliability
- Using a correlation analysis. In test retest and inter observer reliability, the 2 sets of
scores are correlated. Correlation coefficient should exceed .8 for reliability.
Improving reliability
Questionnaires if it produces low test – retest reliability it may require some of the items to
be ‘deselected’ or rewritten. One solution might be to replace some open questions where
there may be room for misinterpretation with closed, fixed choice alternatives to make them
less ambiguous.
Interviews use same interviewer each time, if this isn’t possible, all interviewers must be
properly trained. Should all be able to structure their interviews in a ‘certain manner’ which can
be followed by all to ensure everything is similar. More easily avoided in structured interviews
where interviewers’ behaviour is more controlled by fixed questions.
Reliability Measure of how consistent the findings from an investigation are.
Ways of assessing:
Internal reliability describes how consistent something is within itself.
- Split half method comparing results of one half of a test with the results from the
other half. These two forms of the same test are then done separately and should yield
the same score and have a correlation coefficient of more or equal to 0.8.
External reliability when consistent results are produced regardless of when the
investigation is used or who administers it.
Ways of assessing:
- Test-retest method researcher administers the same test on the same person on
different occasions (usually a week or 2 later). Results should be similar and lead a
correlation coefficient of more or equal to 0.8. must be sufficient time between test and
retest so ppts can't recall their answers but not too long as the persons attitude may
change.
- Inter observer reliability have 2 or more observers watching the same event or
sequence of events and record the data independently.
- Observers agree on interpretation of each behavioural category
- Tally charts compared to check for agreement
- extent to which there is agreement between two or more observers involved in
observing behaviour. Eliminating subjectivity bias and may either be carried out in a pilot
study or reported at the end of the study. (at least 0.8 correlation)
Measuring reliability
- Using a correlation analysis. In test retest and inter observer reliability, the 2 sets of
scores are correlated. Correlation coefficient should exceed .8 for reliability.
Improving reliability
Questionnaires if it produces low test – retest reliability it may require some of the items to
be ‘deselected’ or rewritten. One solution might be to replace some open questions where
there may be room for misinterpretation with closed, fixed choice alternatives to make them
less ambiguous.
Interviews use same interviewer each time, if this isn’t possible, all interviewers must be
properly trained. Should all be able to structure their interviews in a ‘certain manner’ which can
be followed by all to ensure everything is similar. More easily avoided in structured interviews
where interviewers’ behaviour is more controlled by fixed questions.
