Progressive Assessment Test - Educational Assessment Review Examination
Progressive Assessment Test
Educational Assessment Comprehensive Review Examination
1. In Classical Test Theory, the observed score is conceptualized as which of the following?
A. The true score plus systematic error only
B. The true score plus random error only [CORRECT]
C. The true score plus both systematic and random error
D. The true score minus the error score
Correct Answer: B
Rationale: Correct because Classical Test Theory defines the observed score (X) as X = T + E, where T is the true score
and E is random error. This matches the foundational CTT equation and is the basis for all reliability estimation
methods.
2. A reading comprehension test yields a Cronbach alpha of 0.92. What does this value indicate about the test?
A. The test has poor internal consistency and should not be used
B. The test has excellent internal consistency, suggesting the items measure the same construct
[CORRECT]
C. The test is too difficult for the intended population
D. The test items are unrelated to each other
Correct Answer: B
Rationale: Correct because a Cronbach alpha of 0.92 indicates excellent internal consistency reliability, meaning the
items on the test are highly correlated and measure a single underlying construct. Values above 0.90 are considered
excellent in educational measurement.
3. An item has a difficulty index (p-value) of 0.15. What conclusion can be drawn about this item?
A. The item is moderately difficult and appropriate for most test-takers
B. The item is very easy and should be reviewed for ceiling effects
C. The item is very difficult, as only 15% of test-takers answered it correctly [CORRECT]
D. The item has no discriminative value and must be discarded
Correct Answer: C
Rationale: Correct because the p-value represents the proportion of test-takers who answered the item correctly. A
p-value of 0.15 means only 15% answered correctly, indicating a very difficult item that may need revision depending on
the test purpose.
4. A researcher correlates scores from a new intelligence test with academic GPA obtained two years later.
Which type of validity evidence is being collected?
A. Content validity evidence
B. Concurrent criterion-related validity evidence
C. Predictive criterion-related validity evidence [CORRECT]
D. Construct validity evidence
Correct Answer: C
Rationale: Correct because predictive criterion-related validity involves correlating test scores with a future criterion
measure. Using GPA obtained two years later as the criterion makes this predictive rather than concurrent, which would
use a criterion measured at the same time.
5. A standardized math test has a standard deviation of 10 and a reliability coefficient of 0.84. What is the
approximate standard error of measurement?
, Progressive Assessment Test - Educational Assessment Review Examination
A. 1.6
B. 4.0 [CORRECT]
C. 8.4
D. 2.0
Correct Answer: B
Rationale: Correct because the standard error of measurement is calculated as SEM = SD x the square root of (1 -
reliability). SEM = 10 x the square root of (1 - 0.84) = 10 x 0.4 = 4.0. This represents the expected variability around an
observed score.
6. Which type of test interpretation compares a test-taker performance to that of a specified group of peers?
A. Criterion-referenced interpretation
B. Norm-referenced interpretation [CORRECT]
C. Domain-referenced interpretation
D. Standards-based interpretation
Correct Answer: B
Rationale: Correct because norm-referenced interpretation involves comparing an individual score to the distribution of
scores from a clearly defined reference group, such as national percentile ranks. This matches the definition of
norm-referenced testing as described in measurement textbooks.
7. An item discrimination index (D) of -0.25 is calculated for a multiple-choice item. What does this value
suggest?
A. The item effectively differentiates between high and low achievers
B. The item is functioning well and should be retained
C. Low-scoring test-takers are more likely to answer correctly than high-scoring test-takers
[CORRECT]
D. The item has no relationship with total test performance
Correct Answer: C
Rationale: Correct because a negative discrimination index indicates that low-scoring test-takers are answering the item
correctly more often than high-scoring test-takers. This suggests a potential flaw in the item, such as a misleading
distractor that high achievers are selecting.
8. In Item Response Theory, which parameter describes how well an item discriminates between test-takers of
different ability levels?
A. The difficulty parameter (b)
B. The guessing parameter (c)
C. The discrimination parameter (a) [CORRECT]
D. The information function
Correct Answer: C
Rationale: Correct because the discrimination parameter (a) in the three-parameter logistic IRT model indicates how
steeply the item characteristic curve rises, reflecting how effectively the item differentiates between test-takers with
ability levels slightly above and below the item difficulty.
9. The Rasch model differs from the two-parameter and three-parameter IRT models in that it includes only
which parameters?
A. The difficulty parameter and the discrimination parameter
B. The difficulty parameter (b) and assumes equal discrimination across all items [CORRECT]
C. The guessing parameter and the difficulty parameter
D. The discrimination parameter and the guessing parameter
, Progressive Assessment Test - Educational Assessment Review Examination
Correct Answer: B
Rationale: Correct because the Rasch model is a one-parameter logistic model that estimates only item difficulty (b),
assuming all items have equal discriminating power. This specific objectivity property is a core assumption that
distinguishes it from multi-parameter IRT models.
10. In a computerized adaptive testing (CAT) system, how are subsequent items selected for a test-taker?
A. Items are selected randomly from the item pool
B. Items are selected based on the test-taker estimated ability after each response [CORRECT]
C. All test-takers receive the same fixed sequence of items
D. Items are selected by the test administrator before the test begins
Correct Answer: B
Rationale: Correct because CAT algorithms use the test-taker current ability estimate (typically via maximum likelihood
or Bayesian methods) to select the next item that provides the most information at that ability level. This matches the core
adaptive principle of CAT.
11. A student scores at the 72nd percentile on a standardized reading test. Which interpretation is most
accurate?
A. The student answered 72% of the items correctly
B. The student scored higher than 72% of the norming sample [CORRECT]
C. The student achieved a grade-equivalent score of 7.2
D. The student reading level is above average but below gifted
Correct Answer: B
Rationale: Correct because a percentile rank indicates the percentage of scores in the norming sample that fall below a
given score. A 72nd percentile rank means the student performed as well as or better than 72% of the reference group,
which is the standard interpretation.
12. A researcher developing a new creativity measure administers it along with established measures of
divergent thinking and originality, finding strong positive correlations. This is an example of what type of
validity evidence?
A. Convergent validity evidence [CORRECT]
B. Discriminant validity evidence
C. Content validity evidence
D. Face validity evidence
Correct Answer: A
Rationale: Correct because convergent validity evidence is demonstrated when a new measure correlates strongly with
existing measures of the same or theoretically related constructs. Correlating a creativity measure with divergent
thinking and originality measures provides convergent evidence.
13. Differential Item Functioning (DIF) analysis reveals that an item functions differently for two gender
groups after matching on ability. What is the most appropriate next step?
A. Immediately remove the item from the test
B. Investigate the source of the DIF and determine whether it reflects genuine group differences or item
bias [CORRECT]
C. Ignore the DIF findings as they are expected in most tests
D. Recalibrate the test using only one gender group as the reference
Correct Answer: B
Rationale: Correct because DIF indicates that item performance differs between groups at the same ability level, but DIF
alone does not confirm bias. Priority is to investigate the source of the differential functioning before making deletion
decisions, as DIF may reflect true group differences.
Progressive Assessment Test
Educational Assessment Comprehensive Review Examination
1. In Classical Test Theory, the observed score is conceptualized as which of the following?
A. The true score plus systematic error only
B. The true score plus random error only [CORRECT]
C. The true score plus both systematic and random error
D. The true score minus the error score
Correct Answer: B
Rationale: Correct because Classical Test Theory defines the observed score (X) as X = T + E, where T is the true score
and E is random error. This matches the foundational CTT equation and is the basis for all reliability estimation
methods.
2. A reading comprehension test yields a Cronbach alpha of 0.92. What does this value indicate about the test?
A. The test has poor internal consistency and should not be used
B. The test has excellent internal consistency, suggesting the items measure the same construct
[CORRECT]
C. The test is too difficult for the intended population
D. The test items are unrelated to each other
Correct Answer: B
Rationale: Correct because a Cronbach alpha of 0.92 indicates excellent internal consistency reliability, meaning the
items on the test are highly correlated and measure a single underlying construct. Values above 0.90 are considered
excellent in educational measurement.
3. An item has a difficulty index (p-value) of 0.15. What conclusion can be drawn about this item?
A. The item is moderately difficult and appropriate for most test-takers
B. The item is very easy and should be reviewed for ceiling effects
C. The item is very difficult, as only 15% of test-takers answered it correctly [CORRECT]
D. The item has no discriminative value and must be discarded
Correct Answer: C
Rationale: Correct because the p-value represents the proportion of test-takers who answered the item correctly. A
p-value of 0.15 means only 15% answered correctly, indicating a very difficult item that may need revision depending on
the test purpose.
4. A researcher correlates scores from a new intelligence test with academic GPA obtained two years later.
Which type of validity evidence is being collected?
A. Content validity evidence
B. Concurrent criterion-related validity evidence
C. Predictive criterion-related validity evidence [CORRECT]
D. Construct validity evidence
Correct Answer: C
Rationale: Correct because predictive criterion-related validity involves correlating test scores with a future criterion
measure. Using GPA obtained two years later as the criterion makes this predictive rather than concurrent, which would
use a criterion measured at the same time.
5. A standardized math test has a standard deviation of 10 and a reliability coefficient of 0.84. What is the
approximate standard error of measurement?
, Progressive Assessment Test - Educational Assessment Review Examination
A. 1.6
B. 4.0 [CORRECT]
C. 8.4
D. 2.0
Correct Answer: B
Rationale: Correct because the standard error of measurement is calculated as SEM = SD x the square root of (1 -
reliability). SEM = 10 x the square root of (1 - 0.84) = 10 x 0.4 = 4.0. This represents the expected variability around an
observed score.
6. Which type of test interpretation compares a test-taker performance to that of a specified group of peers?
A. Criterion-referenced interpretation
B. Norm-referenced interpretation [CORRECT]
C. Domain-referenced interpretation
D. Standards-based interpretation
Correct Answer: B
Rationale: Correct because norm-referenced interpretation involves comparing an individual score to the distribution of
scores from a clearly defined reference group, such as national percentile ranks. This matches the definition of
norm-referenced testing as described in measurement textbooks.
7. An item discrimination index (D) of -0.25 is calculated for a multiple-choice item. What does this value
suggest?
A. The item effectively differentiates between high and low achievers
B. The item is functioning well and should be retained
C. Low-scoring test-takers are more likely to answer correctly than high-scoring test-takers
[CORRECT]
D. The item has no relationship with total test performance
Correct Answer: C
Rationale: Correct because a negative discrimination index indicates that low-scoring test-takers are answering the item
correctly more often than high-scoring test-takers. This suggests a potential flaw in the item, such as a misleading
distractor that high achievers are selecting.
8. In Item Response Theory, which parameter describes how well an item discriminates between test-takers of
different ability levels?
A. The difficulty parameter (b)
B. The guessing parameter (c)
C. The discrimination parameter (a) [CORRECT]
D. The information function
Correct Answer: C
Rationale: Correct because the discrimination parameter (a) in the three-parameter logistic IRT model indicates how
steeply the item characteristic curve rises, reflecting how effectively the item differentiates between test-takers with
ability levels slightly above and below the item difficulty.
9. The Rasch model differs from the two-parameter and three-parameter IRT models in that it includes only
which parameters?
A. The difficulty parameter and the discrimination parameter
B. The difficulty parameter (b) and assumes equal discrimination across all items [CORRECT]
C. The guessing parameter and the difficulty parameter
D. The discrimination parameter and the guessing parameter
, Progressive Assessment Test - Educational Assessment Review Examination
Correct Answer: B
Rationale: Correct because the Rasch model is a one-parameter logistic model that estimates only item difficulty (b),
assuming all items have equal discriminating power. This specific objectivity property is a core assumption that
distinguishes it from multi-parameter IRT models.
10. In a computerized adaptive testing (CAT) system, how are subsequent items selected for a test-taker?
A. Items are selected randomly from the item pool
B. Items are selected based on the test-taker estimated ability after each response [CORRECT]
C. All test-takers receive the same fixed sequence of items
D. Items are selected by the test administrator before the test begins
Correct Answer: B
Rationale: Correct because CAT algorithms use the test-taker current ability estimate (typically via maximum likelihood
or Bayesian methods) to select the next item that provides the most information at that ability level. This matches the core
adaptive principle of CAT.
11. A student scores at the 72nd percentile on a standardized reading test. Which interpretation is most
accurate?
A. The student answered 72% of the items correctly
B. The student scored higher than 72% of the norming sample [CORRECT]
C. The student achieved a grade-equivalent score of 7.2
D. The student reading level is above average but below gifted
Correct Answer: B
Rationale: Correct because a percentile rank indicates the percentage of scores in the norming sample that fall below a
given score. A 72nd percentile rank means the student performed as well as or better than 72% of the reference group,
which is the standard interpretation.
12. A researcher developing a new creativity measure administers it along with established measures of
divergent thinking and originality, finding strong positive correlations. This is an example of what type of
validity evidence?
A. Convergent validity evidence [CORRECT]
B. Discriminant validity evidence
C. Content validity evidence
D. Face validity evidence
Correct Answer: A
Rationale: Correct because convergent validity evidence is demonstrated when a new measure correlates strongly with
existing measures of the same or theoretically related constructs. Correlating a creativity measure with divergent
thinking and originality measures provides convergent evidence.
13. Differential Item Functioning (DIF) analysis reveals that an item functions differently for two gender
groups after matching on ability. What is the most appropriate next step?
A. Immediately remove the item from the test
B. Investigate the source of the DIF and determine whether it reflects genuine group differences or item
bias [CORRECT]
C. Ignore the DIF findings as they are expected in most tests
D. Recalibrate the test using only one gender group as the reference
Correct Answer: B
Rationale: Correct because DIF indicates that item performance differs between groups at the same ability level, but DIF
alone does not confirm bias. Priority is to investigate the source of the differential functioning before making deletion
decisions, as DIF may reflect true group differences.
