Innovations Literature
February - April 2025
Clinical Neuropsychology
Leiden University
,Week 1 → Introduction
The Technology Crisis in Neuropsychology - Miller &
Barr (2017)
Overview of how little tech integration there is in NP, what are possibilities, benefits,
barriers
Introduction
● Neuropsychology has remained stagnant → tech has not been integrated into practice
● Outdated & labor intensive data collection methods → slow, inefficient, expensive, poor
data estimates of human behavior
● Field could be rapidly advanced if tech is integrated
● This review shows the advantages of integrating tech
Brief historical review & problem statement
● Tests for NP over the years have been used only to assess brain-behavior relationships
○ Originally meant for studying individual differences
● Boston Process Approach → inherent limitations in older tests led to modification &
extension of measures to capture qualitative test performance in a range of patients
● NP relies primarily on paper and pencil tasks
○ Depend on test publishers who earn money off of the sales of these tests
○ Publishers have made minor efforts to update the tests → minimal revision to
tests developed over 100 years ago
● In 1980s some believed computers would change the field → but people focused on the
concerns and problems instead of advantages
● Publishers did not adapt their tests for computers, just revised a new version for paper
● Because of low focus on computer tasks they have not been invested in and have
insufficient psychometric properties → makes clinicians hesitant to use them
● In studies it has been found that there is no sign of any evolution in the field over the
years → fails to keep up with other advancing fields, could risk losing integrity in eyes of
public if stays behind
Advantages of technological integration & how might we use it
● Laboratory testing
○ Most straightforward approach to incorporate tech into assessment & testing is
by putting them on a computer
■ Now tests have only been converted to computers for automatization of
scoring, but it can be used much more effectively
○ Drawing or writing tests can be done on tablets
■ Using computers can allow to record, review and replay responses
, ■ Many additional metrics can be extracted and quantified → speed &
consistency of output over time, accuracy, pencil pressure, number,
length, position of pauses, amplitude and frequency of tremulousness
differentiated between rest and while moving
■ Can be used for TMT or clock drawing for example
○ Non drawing tests can also be digital
■ Tower of Hanoi → easily administered on tablet, versions already exist for
entertainment , but standardization is necessary for clinical use
■ Boston naming test → incredibly simple to adapt, 2D pictures
■ More difficult 3D based tasks could be applied digitally, not able on paper
→ visuoconstructive (sensor integrated geometric block tests)
■ Verbal fluency → no longer manual recording is necessary, audio
recording & speech recognition technology can automatically record
latency, pauses, grammatical errors, consistency, evaluate patterns of
responding
■ Most important additional data that can be recorded is reaction time →
better range and degree of position to be recorded by computer than
human
● Measuring reaction time could eliminate the need for measuring
processing speed separately
■ Episodic memory → seems less useful to adapt to tech
● Presenting stimuli for learning trials could be done
● Free recall is harder → speech recognition could be used for
verbal responses, which can be recorded and played back
○ Limitations → accents, dysarthria, but the recognition
machine could be trained to avoid this, this technology
continues evolving
■ Nonverbal learning & memory → challenging to adapt
● Automated scoring would need fuzzy algorithms to avoid rigid
scoring, customized to patient populations
● Computer based memory tests could have multimodal paradigms
that integrate presentation of both verbal and nonverbal stimuli
which could disentangle verbal from nonverbal memory within the
same task
● Benefits: beyond scoring
○ Built in standardization → compliance with standardization of tests is ensured
■ Items can be repeated until tasks are fully understood
■ Increases standardization, reproducibility and reliability
■ Reduces administrative error potential, scoring & data errors
■ Facilitates real time evaluation of performance → increased efficiency
■ Manual data entry after test completion is error prone, time consuming,
inefficient
■ Patient access to services will be faster due to increased efficiency
○ Incorporation of principles of modern psychometric theory
, ■ Item response theory → many advantages over classical test theory
■ Computer adaptive testing leverages IRT principles to evaluate item level
performance in real time → requires computational speed & power of a
computer to be effective
■ Some analog settings have a somewhat adaptive format, but the starting
point is so low that unnecessary items are administered that give
unnecessary data
■ Computers allow for an iterative process → depending on the current
score, the items to administer are decided upon in real time (adjusting for
difficulty)
■ Requires significantly less items to determine ability levels → efficient
■ Difficult domains for IRT → memory, attention, WM
○ Functioning of adaptive measures
■ Item bank must be sufficiently large to draw from
■ Large pools of items have other benefits → alternate test forms (help
longitudinal monitoring & re-evaluations, increases flexibility of alternate
forms)
■ New “experimental” items can be regularly integrated and studied →
continual rotation of new items, removal of outdated items
■ Objective calibration and equating of test items → improves development
of tests in different languages for diverse populations
■ Reduces the need for large scale test revisions
● Points of consideration
○ Rigidity of a computer
■ Can be overly restrictive and impose constraints on testing
■ Limitations to adapt tasks to non-standardized versions when everything
is automated
■ Programs need to be sufficiently adaptive and provide reorienting cues
when applicable and be able to recognize when cues are necessary
■ Computers can make it harder to revise an answer once recorded
■ Review of completed items to avoid this is often inappropriate, inefficient
and impractical
○ Digital security
■ Security & confidentiality of patient data is important
■ Cloud based data storage is more commonplace and acceptable recently
■ There need to be plans in place in case of a data breach
○ Volume of data
■ Exponentially more data will be generated by tech tools than currently is
being generated
■ Data should be stored safely and securely, there are legal standards for
this → trustworthy, reliable, equivalent to paper records
○ Integration of technology into existing tests
February - April 2025
Clinical Neuropsychology
Leiden University
,Week 1 → Introduction
The Technology Crisis in Neuropsychology - Miller &
Barr (2017)
Overview of how little tech integration there is in NP, what are possibilities, benefits,
barriers
Introduction
● Neuropsychology has remained stagnant → tech has not been integrated into practice
● Outdated & labor intensive data collection methods → slow, inefficient, expensive, poor
data estimates of human behavior
● Field could be rapidly advanced if tech is integrated
● This review shows the advantages of integrating tech
Brief historical review & problem statement
● Tests for NP over the years have been used only to assess brain-behavior relationships
○ Originally meant for studying individual differences
● Boston Process Approach → inherent limitations in older tests led to modification &
extension of measures to capture qualitative test performance in a range of patients
● NP relies primarily on paper and pencil tasks
○ Depend on test publishers who earn money off of the sales of these tests
○ Publishers have made minor efforts to update the tests → minimal revision to
tests developed over 100 years ago
● In 1980s some believed computers would change the field → but people focused on the
concerns and problems instead of advantages
● Publishers did not adapt their tests for computers, just revised a new version for paper
● Because of low focus on computer tasks they have not been invested in and have
insufficient psychometric properties → makes clinicians hesitant to use them
● In studies it has been found that there is no sign of any evolution in the field over the
years → fails to keep up with other advancing fields, could risk losing integrity in eyes of
public if stays behind
Advantages of technological integration & how might we use it
● Laboratory testing
○ Most straightforward approach to incorporate tech into assessment & testing is
by putting them on a computer
■ Now tests have only been converted to computers for automatization of
scoring, but it can be used much more effectively
○ Drawing or writing tests can be done on tablets
■ Using computers can allow to record, review and replay responses
, ■ Many additional metrics can be extracted and quantified → speed &
consistency of output over time, accuracy, pencil pressure, number,
length, position of pauses, amplitude and frequency of tremulousness
differentiated between rest and while moving
■ Can be used for TMT or clock drawing for example
○ Non drawing tests can also be digital
■ Tower of Hanoi → easily administered on tablet, versions already exist for
entertainment , but standardization is necessary for clinical use
■ Boston naming test → incredibly simple to adapt, 2D pictures
■ More difficult 3D based tasks could be applied digitally, not able on paper
→ visuoconstructive (sensor integrated geometric block tests)
■ Verbal fluency → no longer manual recording is necessary, audio
recording & speech recognition technology can automatically record
latency, pauses, grammatical errors, consistency, evaluate patterns of
responding
■ Most important additional data that can be recorded is reaction time →
better range and degree of position to be recorded by computer than
human
● Measuring reaction time could eliminate the need for measuring
processing speed separately
■ Episodic memory → seems less useful to adapt to tech
● Presenting stimuli for learning trials could be done
● Free recall is harder → speech recognition could be used for
verbal responses, which can be recorded and played back
○ Limitations → accents, dysarthria, but the recognition
machine could be trained to avoid this, this technology
continues evolving
■ Nonverbal learning & memory → challenging to adapt
● Automated scoring would need fuzzy algorithms to avoid rigid
scoring, customized to patient populations
● Computer based memory tests could have multimodal paradigms
that integrate presentation of both verbal and nonverbal stimuli
which could disentangle verbal from nonverbal memory within the
same task
● Benefits: beyond scoring
○ Built in standardization → compliance with standardization of tests is ensured
■ Items can be repeated until tasks are fully understood
■ Increases standardization, reproducibility and reliability
■ Reduces administrative error potential, scoring & data errors
■ Facilitates real time evaluation of performance → increased efficiency
■ Manual data entry after test completion is error prone, time consuming,
inefficient
■ Patient access to services will be faster due to increased efficiency
○ Incorporation of principles of modern psychometric theory
, ■ Item response theory → many advantages over classical test theory
■ Computer adaptive testing leverages IRT principles to evaluate item level
performance in real time → requires computational speed & power of a
computer to be effective
■ Some analog settings have a somewhat adaptive format, but the starting
point is so low that unnecessary items are administered that give
unnecessary data
■ Computers allow for an iterative process → depending on the current
score, the items to administer are decided upon in real time (adjusting for
difficulty)
■ Requires significantly less items to determine ability levels → efficient
■ Difficult domains for IRT → memory, attention, WM
○ Functioning of adaptive measures
■ Item bank must be sufficiently large to draw from
■ Large pools of items have other benefits → alternate test forms (help
longitudinal monitoring & re-evaluations, increases flexibility of alternate
forms)
■ New “experimental” items can be regularly integrated and studied →
continual rotation of new items, removal of outdated items
■ Objective calibration and equating of test items → improves development
of tests in different languages for diverse populations
■ Reduces the need for large scale test revisions
● Points of consideration
○ Rigidity of a computer
■ Can be overly restrictive and impose constraints on testing
■ Limitations to adapt tasks to non-standardized versions when everything
is automated
■ Programs need to be sufficiently adaptive and provide reorienting cues
when applicable and be able to recognize when cues are necessary
■ Computers can make it harder to revise an answer once recorded
■ Review of completed items to avoid this is often inappropriate, inefficient
and impractical
○ Digital security
■ Security & confidentiality of patient data is important
■ Cloud based data storage is more commonplace and acceptable recently
■ There need to be plans in place in case of a data breach
○ Volume of data
■ Exponentially more data will be generated by tech tools than currently is
being generated
■ Data should be stored safely and securely, there are legal standards for
this → trustworthy, reliable, equivalent to paper records
○ Integration of technology into existing tests
