Selection procedure in sports: Improving predictions of athletes‘ future
performance
• For athlete selection, two issues are essential:
(a) What kind of performance do we want to predict?
(b) What methods can best be used to predict that performance?
• Gap between literature and reality
• consider 2 important topics: 1. How to combine information in order to make predictions
and decisions?; 2. What kind of information to include in decisions to optimise predictions
Judging athletes
• selection of athletes often done by scouts or coaches → often base decision on overall
impression of athlete in their minds
• however, coaches and scouts are often biased (e.g. due to order effect, reputation of athlete,
body language, skin color, clothing)
• people, including experts such as scouts and coaches, are not good at consistently
integrating and weighting relevant sources of information to make judgments
• to increase the quality of the selection process, trainers or scouts can make their decision
rules explicit and possibly add a particular weight to certain athletic skills
◦ decision-making rules like “actuarial judgment” which results in better performance
prediction
◦ the consistent application of a decision rule alone already improves predictions and
decision-making, because it decreases the likelihood of biased judgments → accurate
performance predictions in sports when decisions are made based on pre-specified
decision rules (e.g. explicitly look for player who possesses dribbling (weighted 60%)
and passing (weighted 40%))
• decision rules can be used to combine information from expert ratings, but it is also possible
to use these rules when direct measures of sport performance are collected (e.g. continuous
position data) → such data often unavailable for players, then athletes may be asked to
participate in more simple assessments that provide information on the skills considered to
be important by the scout, coach, and/or club
Assessing athletic skills
• the way in which athletes’ sets of skills are assessed can differ considerably
◦ prevailing approach is that skills are tested in isolation in order to obtain a reliable
assessment of the skill in question (e.g. dribbling test)
→ signs approach: one attempts to measure distinct traits or skills that may predict later
criterion behavior
◦ samples approach: one attempts to sample behavior that mimics the criterion behavior as
closely as possible → closer to the performance behavior (criterion), it likely provides a
better prediction of future performance than the scores on signs that are assessed in
separate tests (especially for homogeneous populations as with elite (youth) athletes)
▪ when compared sign and sample test is NFL the sample test was able to predict
players performance across the following 4 NFL years
Combining selection psychology with theory and practice in sports
• best selection procedures are based on actuarial judgment, and that sample-based tests are
powerful tools to predict later criterion behavior, in particular in homogeneous athletic
populations → however, both is not mainstream is sports
◦ experts such as scouts or trainers in sports are considered to make the best and most
reliable judgments in their field → trusted that they are experts so not forced to use per-
specified decision rules
performance
• For athlete selection, two issues are essential:
(a) What kind of performance do we want to predict?
(b) What methods can best be used to predict that performance?
• Gap between literature and reality
• consider 2 important topics: 1. How to combine information in order to make predictions
and decisions?; 2. What kind of information to include in decisions to optimise predictions
Judging athletes
• selection of athletes often done by scouts or coaches → often base decision on overall
impression of athlete in their minds
• however, coaches and scouts are often biased (e.g. due to order effect, reputation of athlete,
body language, skin color, clothing)
• people, including experts such as scouts and coaches, are not good at consistently
integrating and weighting relevant sources of information to make judgments
• to increase the quality of the selection process, trainers or scouts can make their decision
rules explicit and possibly add a particular weight to certain athletic skills
◦ decision-making rules like “actuarial judgment” which results in better performance
prediction
◦ the consistent application of a decision rule alone already improves predictions and
decision-making, because it decreases the likelihood of biased judgments → accurate
performance predictions in sports when decisions are made based on pre-specified
decision rules (e.g. explicitly look for player who possesses dribbling (weighted 60%)
and passing (weighted 40%))
• decision rules can be used to combine information from expert ratings, but it is also possible
to use these rules when direct measures of sport performance are collected (e.g. continuous
position data) → such data often unavailable for players, then athletes may be asked to
participate in more simple assessments that provide information on the skills considered to
be important by the scout, coach, and/or club
Assessing athletic skills
• the way in which athletes’ sets of skills are assessed can differ considerably
◦ prevailing approach is that skills are tested in isolation in order to obtain a reliable
assessment of the skill in question (e.g. dribbling test)
→ signs approach: one attempts to measure distinct traits or skills that may predict later
criterion behavior
◦ samples approach: one attempts to sample behavior that mimics the criterion behavior as
closely as possible → closer to the performance behavior (criterion), it likely provides a
better prediction of future performance than the scores on signs that are assessed in
separate tests (especially for homogeneous populations as with elite (youth) athletes)
▪ when compared sign and sample test is NFL the sample test was able to predict
players performance across the following 4 NFL years
Combining selection psychology with theory and practice in sports
• best selection procedures are based on actuarial judgment, and that sample-based tests are
powerful tools to predict later criterion behavior, in particular in homogeneous athletic
populations → however, both is not mainstream is sports
◦ experts such as scouts or trainers in sports are considered to make the best and most
reliable judgments in their field → trusted that they are experts so not forced to use per-
specified decision rules
