SFL 290 MIDTERM 2 QUESTIONS WITH VERIFIED
ANSWERS
Reliability - Answers - consistency of a measure
Inter-Rater Reliabiltiy - Answers - Is there consistency from rater to rater? or observer to
observer?
o EX: TA's—"we grade on the same grading scale to make sure it is consistent
throughout and not one person is getting graded completely different from another
person
Statistical test of Inner-rater reliability - Answers - - % agreement
- Cohen's Kappa (K) is a statistic that is used to measure inter-rater reliability... Cohen's
Kappa is a statistic that is used to measure inter-rater reliability for qualitative
(categorical) items
- It is used to asses the agreement between two raters
Alternate/Parallel Form Reliability - Answers - o Is there consistency between forms of
test?
o EX: ACT/SAT---Everyone is given a different test, but really it is the exact same
material, it is just a different form. The score should be consistent across the board
Statistical test of - Alternate/Parallel Form Reliability - Answers - Statistical test:
correlation coefficient
Inter-Item Reliability - Answers - o Is there consistency between individual items and
total score?
o EX: Sexual Partners=== risky sexual behavior (variable A) and multiple sexual
partners (variable B) are likely to be related to eachother (high cronbach's alpha)
- You are basically wanting to make sure that things are consistent
Statistical test of Inter-Item Reliability - Answers - Cronbach's Alpha
Test-Retest Reliability - Answers - o When the measure is given on two separate
occasions, is there consistency between scores?
(GOAL: CHECK CONSISTENCY)
o Same test
o Participants are the same
o No treatment
o EX: If we gave you a test on Monday and gave you the same test on Friday without
any studying in between, you should have the same score
, - Same test
- You are the same
- You did not have treatment or studying
Stats test of Test-Retest reliability - Answers - Correlation Coefficient
Representative reliability - Answers - o Does the indicator deliver the same answer
when applied to different groups (age groups, cultural contexts)?
o EX: depression in older adults/teens
o EX: if we gave a test to the ppl in US and then we gave the same test to ppl in Africa..
both groups should be able to understand the test
Stats test for representative Reliability - Answers - No stats test
Veronica and Marnae are doing a research project looking at the different spiritual
messterm-0ages given in BYU devotionals. They go through every BYU devotional from
the past 20 years and discuss how they will define "spiritual messages". The conclude
that they will only code for spiritual messages when the speaker mentions or references
a scripture (i.e. James 1:5). This is an example of: - Answers - d. inter-rater reliability
You are studying the relationship between high school GPA and college GPA. You find
that those who had a higher GPA in high school had a higher GPA in college. This is an
example of:
a. predictive validity
b. criterion validity
c. representative validity
d. concurrent validity - Answers - a. predictive validity
validity is measured by what statistical test? - Answers - correlation coefficient
Validity - Answers - - accuracy or correctness
- How well do the conceptual and operational definitions mesh with each other?
- BASICALLY: does the measurement questions match with the dictionary definition??
And how accurate are your measurements
Face Validity - Answers - - think faces of experts
- Do items on the scale measure what they appear to measure?
- Determined by a panel of experts
- EX: Unemployment—ppl looking for work.
· Unemployment has a variety of measurements/definitions (looking for work, you have
a disability, you got layed off recently, you never have worked and you are not planning
on it) but when you fill out a questionnaire, the govt or panel of experts gives the
definition to what it means to be UNEMPLOYED in the survey. This way it is not
decided by randos but by the govt.
- EX: depression
ANSWERS
Reliability - Answers - consistency of a measure
Inter-Rater Reliabiltiy - Answers - Is there consistency from rater to rater? or observer to
observer?
o EX: TA's—"we grade on the same grading scale to make sure it is consistent
throughout and not one person is getting graded completely different from another
person
Statistical test of Inner-rater reliability - Answers - - % agreement
- Cohen's Kappa (K) is a statistic that is used to measure inter-rater reliability... Cohen's
Kappa is a statistic that is used to measure inter-rater reliability for qualitative
(categorical) items
- It is used to asses the agreement between two raters
Alternate/Parallel Form Reliability - Answers - o Is there consistency between forms of
test?
o EX: ACT/SAT---Everyone is given a different test, but really it is the exact same
material, it is just a different form. The score should be consistent across the board
Statistical test of - Alternate/Parallel Form Reliability - Answers - Statistical test:
correlation coefficient
Inter-Item Reliability - Answers - o Is there consistency between individual items and
total score?
o EX: Sexual Partners=== risky sexual behavior (variable A) and multiple sexual
partners (variable B) are likely to be related to eachother (high cronbach's alpha)
- You are basically wanting to make sure that things are consistent
Statistical test of Inter-Item Reliability - Answers - Cronbach's Alpha
Test-Retest Reliability - Answers - o When the measure is given on two separate
occasions, is there consistency between scores?
(GOAL: CHECK CONSISTENCY)
o Same test
o Participants are the same
o No treatment
o EX: If we gave you a test on Monday and gave you the same test on Friday without
any studying in between, you should have the same score
, - Same test
- You are the same
- You did not have treatment or studying
Stats test of Test-Retest reliability - Answers - Correlation Coefficient
Representative reliability - Answers - o Does the indicator deliver the same answer
when applied to different groups (age groups, cultural contexts)?
o EX: depression in older adults/teens
o EX: if we gave a test to the ppl in US and then we gave the same test to ppl in Africa..
both groups should be able to understand the test
Stats test for representative Reliability - Answers - No stats test
Veronica and Marnae are doing a research project looking at the different spiritual
messterm-0ages given in BYU devotionals. They go through every BYU devotional from
the past 20 years and discuss how they will define "spiritual messages". The conclude
that they will only code for spiritual messages when the speaker mentions or references
a scripture (i.e. James 1:5). This is an example of: - Answers - d. inter-rater reliability
You are studying the relationship between high school GPA and college GPA. You find
that those who had a higher GPA in high school had a higher GPA in college. This is an
example of:
a. predictive validity
b. criterion validity
c. representative validity
d. concurrent validity - Answers - a. predictive validity
validity is measured by what statistical test? - Answers - correlation coefficient
Validity - Answers - - accuracy or correctness
- How well do the conceptual and operational definitions mesh with each other?
- BASICALLY: does the measurement questions match with the dictionary definition??
And how accurate are your measurements
Face Validity - Answers - - think faces of experts
- Do items on the scale measure what they appear to measure?
- Determined by a panel of experts
- EX: Unemployment—ppl looking for work.
· Unemployment has a variety of measurements/definitions (looking for work, you have
a disability, you got layed off recently, you never have worked and you are not planning
on it) but when you fill out a questionnaire, the govt or panel of experts gives the
definition to what it means to be UNEMPLOYED in the survey. This way it is not
decided by randos but by the govt.
- EX: depression
