Development and Validation of the Cognition Test Battery for Spacefl ight

Development and Validation of the Cognition Test Battery for Spacefl ight Mathias Basner ; Adam Savitt ; Tyler M. Moore ; Allison M. Port ; Sarah McGuire ; Adrian J. Ecker ; Jad Nasrini ; Daniel J. Mollicone ; Christopher M. Mott ; Thom McCann ; David F. Dinges ; Ruben C. Gur BACKGROUND: Sustained high-level cognitive performance is of paramount importance for the success of space missions, which involve environmental, physiological, and psychological stressors that may aff ect brain functions. Despite subjective symptom reports of cognitive fl uctuations in spacefl ight, the nature of neurobehavioral functioning in space has not been clarifi ed. Delivered by Publishing Technology to: University of Pennsylvania Library IP: 165.123.243.167 On: Wed, 18 Nov 2015 15:22:07 Copyright: Aerospace Medical Association METHODS: We developed a computerized cognitive test battery ( Cognition ) that has sensitivity to multiple cognitive domains and was specifi cally designed for the high-performing astronaut population. Cognition consists of 15 unique forms of 10 neuropsychological tests that cover a range of cognitive domains, including emotion processing, spatial orientation, and risk decision making. Cognition is based on tests known to engage specifi c brain regions as evidenced by functional neuroimaging. Here we describe the fi rst normative and acute total sleep deprivation data on the Cognition test battery as well as several eff orts underway to establish the validity, sensitivity, feasibility, and acceptability of Cognition . RESULTS: Practice eff ects and test-retest variability diff ered substantially between the 10 Cognition tests, illustrating the impor tance of normative data that both refl ect practice eff ects and diff erences in stimulus set diffi culty in the population of interest. After one night without sleep, medium to large eff ect sizes were observed for 3 of the 10 tests addressing vigilant attention (Cohen ’ s d 5 1.00), cognitive throughput (d 5 0.68), and abstract reasoning (d 5 0.65). CONCLUSIONS: In addition to providing neuroimaging-based novel information on the eff ects of spacefl ight on a range of cognitive functions, Cognition will facilitate comparing the eff ects of ground-based analogues to spacefl ight, increase consistency across projects, and thus enable meta-analyses. KEYWORDS: neuropsychological test , cognitive test , cognition , space , spacefl ight , performance , astronaut , microgravity , stress , confi nement , isolation , sleep deprivation . Basner M, Savitt A, Moore TM, Port AM, McGuire S, Ecker AJ, Nasrini J, Mollicone DJ, Mott CM, McCann T, Dinges DF, Gur RC. Development and validation of the Cognition test battery for spacefl ight . Aerosp Med Hum Perform. 2015; 86(11): 942 – 952 . S uccessful human space exploration depends on the integ rity of a range of cognitive abilities for unprecedented durations. Errors and accidents may have debilitating or fatal consequences, lead to the loss of expensive equipment, and compromise mission success. In addition to the physiological eff ects of microgravity, the spacecraft setting can involve expo sure to a number of environmental toxicants and operational stressors that have the potential to degrade astronaut cognitive performance. Among these factors are radiation, noise, hyper capnia, hypoxia, decompression, dietary restrictions, fluid shift s, increased intracranial pressure, side eff ects of certain medications, and psychological factors related to isolation, con f i nement, and operational and interpersonal distress. Sleep of suffi cient length and quality is of paramount importance for 942 high levels of daytime performance, yet astronauts on Space Shuttle and International Space Station (ISS) missions have averaged less than 6.1 h sleep per 24 h. 2 Th is amount of sleep is comparable to that of chronic sleep restriction, which has been From the Division of Sleep and Chronobiology, Department of Psychiatry, and the Behavior Laboratory, Department of Psychiatry, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA; Pulsar Informatics, Inc., Philadelphia, PA; and Joggle Research, Seattle, WA. This manuscript was received for review in April 2015 . It was accepted for publication in August 2015 . Address correspondence to: Mathias Basner, M.D., Ph.D., M.Sc., Associate Professor of Sleep and Chronobiology in Psychiatry, Perelman School of Medicine at the University of Pennsylvania, 1019 Blockley Hall, 423 Guardian Drive, Philadelphia, PA ; . Reprint & Copyright © by the Aerospace Medical Association, Alexandria, VA. DOI: 10.3357/AMHP.4343.2015 AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 11 November 2015 COGNITION TEST & SPACEFLIGHT — Basner et al. shown to induce cognitive and neurobehavioral defi cits and negative health outcomes. 1 Although the reasons for reduced sleep durations in spacefl ight are unknown, factors that con tribute to sleep disturbance in spacefl ight include non-24 h light – dark cycles, acute operational shift s in sleep timing, high workload and physical stress. While astronauts have reported cognitive symptoms (oft en referred to as space fog or neurasthenia 23 ), especially aft er ini tial exposure to the spacecraft environment, the results of objective cognitive testing in spacefl ight have been inconclusive and most oft en fail to show statistically signifi cant changes in cognitive performance. 42 Th erefore, the extent, etiology, and persistence of these symptoms are still unknown. Several fac tors may contribute to the discrepancy between subjective symptom reports and objective assessment of cognitive func tions in space. Performance on most cognitive tests improves with repeated administration. Th is practice eff ect may confound (or mask) any cognitive defi cits induced by the spacecraft environment. Depending on the complexity of the cognitive tests and given pre-mission time constraints, it will oft en not be possible to achieve asymptotic performance levels prefl ight. Additionally, existing studies often lack adequate ground-based control groups, and ground-based normative data usually do not exist for the astronaut population. 42 Spacefl ight studies are oft en underpowered due to small sample sizes, and test batteries and cognitive domains they assess typically diff er among studies, complicating systematic meta-analyses that could increase statistical power. Further more, the tests used in the individual studies may simply lack sensitivity because they were designed for clinical populations or populations with lower aptitudes. Th ey may be sensitive enough to detect symptoms associated with manifest distur bances such as severe brain trauma, but fail to detect subclinical defi cits that can degrade optimal performance in high func tioning individuals such as astronauts. Although subclinical defi cits may not constitute an operational concern, they can be valuable in the early detection of environmental or psychologi cal stressors and thus in the prevention of manifest cognitive defi cits. predominantly probes working memory, but fails to assess cogni tive domains like spatial orientation, abstract reasoning, emo tion processing, stability of sustained attention, and risk decision making that are also important for space mission success. Th ere are other reasons why cognitive defi cits may go unde tected in spacefl ight. Well-educated, highly trained, motivated astronauts may be able to transiently compensate for defi cits in cognitive performance induced during spacefl ight by team work and other strategies. Countermeasures used by astronauts may reverse or mask a cognitive defi cit. Astronauts may not subjectively be aware of some cognitive defi cits that could be detected by a sensitive test battery. For example, sleep depriva tion studies indicate that subjective and objective assessments of performance may diff er substantially. 45 Although perfor mance capability is mostly overestimated during periods of sleep restriction, especially during the biological night, it is pos sible that the opposite can happen in spacefl ight. Th is under scores the need for brief and valid objective assays of cognitive performance in spacefl ight. Delivered by Publishing Technology to: University of Pennsylvania Library IP: 165.123.243.167 On: Wed, 18 Nov 2015 15:22:07 Copyright: Aerospace Medical Association Another limitation of available test batteries is that they oft en only probe a few cognitive domains (i.e., they are not comprehensive), despite having multiple cognitive evaluations. It is thus possible that defi cits in domains not covered by these batteries may have been overlooked. NASA currently uses the WinSCAT test battery operationally. 24 WinSCAT consists of a 5-test subset of the larger Automated Neuropsychological Assessment Metrics (ANAM) test system developed by the Department of Defense: 39 1. Mathematical Processing; 2. Run ning Memory Continuous Performance; 3. Delayed Matching to Sample; 4. Code Substitution; and 5. Code Substitution Delayed Recognition. Th e cognitive domains assessed by these tests are: 1. basic computational skills and working memory; 2. attention and working memory; 3. spatial processing and visuo spatial working memory; 4. complex scanning, visual track ing, and attention; and 5. memory. Th erefore, the WinSCAT Here, we describe the development of an improved neuro cognitive assessment tool, named Cognition , provide learning curves for each of its 10 cognitive tests, and show that each test diff ers in its sensitivity to acute total sleep deprivation. Cogni tion was specifi cally designed to assess cognitive functions in astronauts and address some of the more than 25 knowledge gaps and health risks that mention cognition in NASA ’ s Human Research Roadmap. 12 Cognition covers the main cognitive domains – executive, episodic memory, complex cognition, social cognition, and sensorimotor speed – and is based on tests known to engage specifi c brain systems during functional neu roimaging. 19 , 41 Th e latter may provide information on the neu rostructural origin of a cognitive defi cit, which is important as we currently have no neuroimaging capability in spacefl ight. Cognition is involved in several ongoing and soon-to-be com menced validation studies. Th e goal is to advance knowledge on the cognitive eff ects of spacefl ight by off ering a brief and well-validated battery of tests that are acceptable to the astro naut population, feasible in spacefl ight, and that provide crucial clinical feedback on neurobehavioral functions in space. Th is research tool will hopefully increase consistency across projects and facilitate meta-analyses. METHODS Subjects A normative study in astronauts/astronaut candidates ( N 5 8, mean age 44.1 yr, range 34-53 yr, 38% female) and mission con trollers ( N 5 11, mean age 28.0 yr, range 22-38 yr, 55% female) was completed in 2014 at Johnson Space Center, Houston, TX. T h is study was approved by NASA ’ s Institutional Review Board and subjects signed written informed consent prior to study participation. Th e Cognition battery was also performed by a total of 44 diff erent subjects (mean age 6 SD 34.1 6 8.7 yr, 50% male) in two sleep restriction protocols that included acute total sleep deprivation (i.e., one night without sleep). Th ese studies 943 AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 11 November 2015 COGNITION TEST & SPACEFLIGHT — Basner et al. were approved by the Institutional Review Board of the Univer sity of Pennsylvania, and subjects signed written informed con sent prior to study participation. Equipment Th e Cognition battery contains a subset of tests from a widely used and validated neurocognitive battery, the Penn Comput erized Neurocognitive Battery (CNB), 18 , 19 , 31 as well as a num ber of additional tests that have either been used extensively in spacefl ight (i.e., Psychomotor Vigilance Test, 28 Digit Symbol Substitution Test 43 ) or assess cognitive domains of particu lar interest in spacefl ight. The CNB is currently being used in cognitive assessment of military enlistees, cognitive devel opment in children, cognitive therapies for schizophrenia patients, and genomic research in populations with or at risk for schizophrenia. 19 Table I provides an overview of the cognitive domains assessed and brain regions primarily recruited by each of the 10 tests of the Cognition battery. It also shows average admin istration time for each test based on a study performed in astronauts, astronaut candidates, and mission controllers at Johnson Space Center. Screenshots of each of the 10 Cognition tests are shown in Fig. 1 . Th e tests are described in detail b e l o w : T h e Visual Object Learning Test (VOLT) . VOLT assesses par ticipant memory for complex fi gures. 14 Participants are asked to memorize 10 sequentially displayed three-dimensional fi g ures. Later, they are instructed to select those objects they memorized from a set of 20 such objects also sequentially pre sented, some of them from the learning set and some of them new. Such tasks have been shown to activate frontal and bilat eral anterior medial temporal lobe regions. 22 As the hippocam pus and medial temporal lobe are also adversely aff ected by chronic stress, 42 the VOLT off ers a validated tool for assessment of these temporo-limbic regions in operational settings. Delivered by Publishing Technology to: University of Pennsylvania Library IP: 165.123.243.167 On: Wed, 18 Nov 2015 15:22:07 Copyright: Aerospace Medical Association T h e Motor Praxis Task (MP) . Th e MP 17 is administered at the start of testing to ensure that participants have suffi cient com mand of the computer interface, and immediately thereaft er as a measure of sensorimotor speed. Participants are instructed to click on squares that appear randomly on the screen, each suc cessive square smaller and thus more diffi cult to track. Perfor mance is assessed by the speed with which participants click each square. Th e current implementation uses 20 consecutive stimuli. As a screener for computer skill, the MP has been included in every implementation of the CNB and validated for sensitivity to age eff ects, 18 sex-diff erences, 40 and associations with psychopathology. 33 T h e Fractal 2-Back (F2B) . Th e F2B 36 is a nonverbal variant of the Letter 2-Back, which is currently included in the core CNB. 19 N-back tasks have become standard probes of the work ing memory system and activate canonical working memory brain areas. Th e F2B consists of the sequential presentation of a set of fi gures (fractals), each potentially repeated multiple times. Participants have to respond when the current stimulus matches the stimulus displayed two fi gures ago. Th e current implemen tation uses 62 consecutive stimuli. Th e fractal version was cho sen for Cognition because of its increased diffi culty 36 and the availability of algorithms with which new items can be gener ated. Traditional letter N-back tasks are restricted to 26 English letters, which limits the ability to generate novel stimuli for repeat administrations. Th e Running Memory Continuous Per formance task implemented in WinSCAT is a 1-Back task that uses the numbers 0-9 as stimuli. The F2B implemented in Cognition is well-validated and shows robust activation of the dorsolateral prefrontal cortex. 36 Abstract Matching (AM) . Th e AM test 13 is a validated measure of the abstraction and fl exibility components of executive func tion, including an ability to discern general rules from specifi c instances. Th e test paradigm presents subjects with two pairs of objects at the bottom left and right of the screen, varied on perceptual dimensions (e.g., color and shape). Subjects are Table I. Overview of the Cognition Test Battery. TEST Motor Praxis (MP) Visual Object Learning (VOLT) Fractal 2-Back (F2B) Abstract Matching (AM) Line Orientation (LOT) Emotion Recognition (ERT) Matrix Reasoning (MRT) Digit Symbol Substitution (DSST) Balloon Analog Risk (BART) COGNITIVE DOMAINS ASSESSED Sensory-motor speed Spatial learning and memory Working memory Abstraction, concept formation Spatial orientation Emotion identifi cation Abstract reasoning BRAIN REGIONS PRIMARILY RECRUITED Sensorimotor cortex Medial temporal cortex, hippocampus Dorsolateral prefrontal cortex, cingulate, hippocampus Prefrontal cortex ADMINISTRATION TIME [MINUTES] * Median (Range) Right temporo-parietal cortex, visual cortex Cingulate, amygdala, hippocampus, fusiform face area Prefrontal cortex, parietal cortex, temporal cortex Complex scanning and visual tracking Temporal cortex, prefrontal cortex, motor cortex Risk decision making Psychomotor Vigilance (PVT) * Vigilant attention Orbital frontal and ventromedial prefrontal cortex, amygdala, hippocampus, anterior cingulate cortex, ventral striatum Prefrontal cortex, motor cortex, inferior parietal and some visual cortex 0.4 (0.3 – 2.3) 1.7 (1.4 – 8.2) 2.0 (1.7 – 16.5) 1.8 (1.3 - 7.9) 1.2 (0.8 – 2.4) 1.7 (1.2 – 3.1) 2.1 (0.6 – 3.9) 1.6 (1.6 – 2.6) 2.1 (1.7 – 4.1) 3.2 (3.1 – 4.5) Administration times based on N 5 15 administrations of the Cognition battery in each of N 5 19 astronauts, astronaut candidates, and mission controllers ( N 5 285 total administra tions; see text for details). Administration times include the time needed to input comments and any pause taken by the subject before proceeding to the next test. 944 AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 11 November 2015