Clinical Versus Actuarial Judgment
Methods of Judgment and Means of Comparison
• clinical method: the decision-maker combines or processes information in hos or her head
(has nothing to do with clinical stetting or clinical practitioner → psychiatry may use
clinical or actuarial method)
• actuarial or statistical method: human judge is eliminated and conclusions rest solely on
empirically established relations between data and the condition or event of interest →
interpretations must be both automatic and based on empirically established relations
• combination of both is also possible → proposals presume that the two judgment methods
work together harmoniously and overlook the many situations that require dichotomous
choices (when both methods agree there is no need to apply both)
• Meehl specified conditions for a fair comparison of the two methods
1. both methods should base judgments on the same data (number of cases/data can vary as
for the clinician's development of interpretive strategies depends on prior experience and
knowledge, whereas the actuarial strategies may thus be derived from separate or
overlapping data bases)
2. one must avoid conditions that can artificially inflate the accuracy of actuarial methods
→ derivation typically should be followed by cross-validation (application of decision
rule to new or fresh cases or by a standard statistical estimate of the probable outcome of
cross-validation)
Results of Comparative Studies
• Goldberg examined with a Minnesota Multiphasic Personality Inventory (MMPI) the
distinction between neurotic and psychotic, when experts had to judge the participants they
made on average about 62% correct decision and the best judge made 67% correct
decisions, however, the Goldberg Rule achieved 70% correct decisions (Goldberg derived
various decision rules through statistical analysis of scores on 11 MMPI scaled and
psychiatric patient's discharge diagnoses)
◦ when judges were allowed to practice still non of them equaled the Goldberg Rule’s 70%
accuracy
◦ in another study Goldberg constructed a mathematical (linear) models for each judge
that reproduced their decisions as closely as possible (analyzed relations between the
information available to the judge and the judge’s decisions) → found that in case of
disagreement, the models were more often correct than the judge (perfect reliability of
the models likely explains their superior performance in this and related studies)
• Leli & Filskov studied diagnosis of progressive brain dysfunction based on intellectual
testing
◦ decision rule identified 83% of new cases and clinicians only identified 63% and 58%
(when clinicians were given the results of the actuarial analysis both clinicians improved
but neither group matched the decision rule’s 83% accuracy) → clinicians improvement
appeared to depend on the extent to which they used the rule
• Einhorn studied prediction of survival time following internal diagnosis of Hodgkin’s
disease as established by biopsy → study reveals more than actuarial advantage, shows that
the pathologists ratings produced potentially useful information but that only the actuarial
method, which was based on these ratings, tapped their predictive value
→ 3 studies illustrate key features of a much larger literature in clinical vs. actuarial judgment:
1. studies met previously specified conditions for fair comparison
2. studies representative of general research outcomes (actuarial method has equal or
surpassed the clinical method)
• validity and generalization were criticized in past years but now established:
Methods of Judgment and Means of Comparison
• clinical method: the decision-maker combines or processes information in hos or her head
(has nothing to do with clinical stetting or clinical practitioner → psychiatry may use
clinical or actuarial method)
• actuarial or statistical method: human judge is eliminated and conclusions rest solely on
empirically established relations between data and the condition or event of interest →
interpretations must be both automatic and based on empirically established relations
• combination of both is also possible → proposals presume that the two judgment methods
work together harmoniously and overlook the many situations that require dichotomous
choices (when both methods agree there is no need to apply both)
• Meehl specified conditions for a fair comparison of the two methods
1. both methods should base judgments on the same data (number of cases/data can vary as
for the clinician's development of interpretive strategies depends on prior experience and
knowledge, whereas the actuarial strategies may thus be derived from separate or
overlapping data bases)
2. one must avoid conditions that can artificially inflate the accuracy of actuarial methods
→ derivation typically should be followed by cross-validation (application of decision
rule to new or fresh cases or by a standard statistical estimate of the probable outcome of
cross-validation)
Results of Comparative Studies
• Goldberg examined with a Minnesota Multiphasic Personality Inventory (MMPI) the
distinction between neurotic and psychotic, when experts had to judge the participants they
made on average about 62% correct decision and the best judge made 67% correct
decisions, however, the Goldberg Rule achieved 70% correct decisions (Goldberg derived
various decision rules through statistical analysis of scores on 11 MMPI scaled and
psychiatric patient's discharge diagnoses)
◦ when judges were allowed to practice still non of them equaled the Goldberg Rule’s 70%
accuracy
◦ in another study Goldberg constructed a mathematical (linear) models for each judge
that reproduced their decisions as closely as possible (analyzed relations between the
information available to the judge and the judge’s decisions) → found that in case of
disagreement, the models were more often correct than the judge (perfect reliability of
the models likely explains their superior performance in this and related studies)
• Leli & Filskov studied diagnosis of progressive brain dysfunction based on intellectual
testing
◦ decision rule identified 83% of new cases and clinicians only identified 63% and 58%
(when clinicians were given the results of the actuarial analysis both clinicians improved
but neither group matched the decision rule’s 83% accuracy) → clinicians improvement
appeared to depend on the extent to which they used the rule
• Einhorn studied prediction of survival time following internal diagnosis of Hodgkin’s
disease as established by biopsy → study reveals more than actuarial advantage, shows that
the pathologists ratings produced potentially useful information but that only the actuarial
method, which was based on these ratings, tapped their predictive value
→ 3 studies illustrate key features of a much larger literature in clinical vs. actuarial judgment:
1. studies met previously specified conditions for fair comparison
2. studies representative of general research outcomes (actuarial method has equal or
surpassed the clinical method)
• validity and generalization were criticized in past years but now established:
