Why is this important to me?
If you have MS, you may experience cognitive fatigue, which is reduced cognitive performance over time when performing a mental task that requires sustained activity. Researchers do not know if cognitive fatigue is related to generalized physical fatigue, which is common if you have MS. A standardized way to measure cognitive fatigue has not been established.
What is the objective of this study?
The authors compared healthy controls and people with MS who performed a test called the Paced Auditory Serial Addition Test (PASAT). In the PASAT, you will hear single-digit numbers presented 3 seconds apart, and you are asked to add the last two numbers you heard. For example, if you hear 3, 7, the correct answer is 10 (3 + 7 = 10). If the next number you hear is 5, the correct answer is 12 (7 + 5 = 12). If the next number is 3, the answer is 8 (5 + 3 = 8), and so on. This test measures attention as well as processing speed, which is the time it takes to perform a mental task. The authors tested the idea that if MS patients had cognitive fatigue, they would provide fewer correct answers at the end of the test.
The authors tested 130 healthy controls (92 women, 38 men, average age 37 years) and 100 people with MS (80 women, 20 men, average age 39 years) who had taken the PASAT. They compared the number of correct answers in the first third of the test with the number of correct answers in the last third of the test in both groups. Results showed:
- Healthy controls showed improved performance in the last third of the test and answered around two more correct answers in the last third compared to the first third of the test.
- In contrast, people with MS performed worse in the last third of the test and provided two or three fewer correct answers in the last third compared to the first third of the test.
- Of the 100 people with MS, 46 showed evidence of cognitive fatigue.
This study is the first to provide a standardized way to measure and diagnose cognitive fatigue if you have MS. The authors believe that cognitive fatigue is distinct from generalized physical fatigue, meaning that fatigue in terms of your ability to perform mental tasks is different from fatigue you may experience in terms of your ability to perform activities of daily living. This method will help physicians plan ways to manage cognitive fatigue and will allow your doctor to follow your cognitive fatigue over time. In addition, this method can be used to monitor the effectiveness of treatments for cognitive fatigue.
How did the authors study this issue?
The authors determined the performance on the PASAT in the first third of the test compared to the last third of the test in healthy controls and people with MS and who lived in Ontario, Canada. They accessed the PASAT scores that were stored in a database for 100 people with MS who had previously taken the PASAT as part of a neuropsychological assessment. One hundred thirty healthy controls were recruited and newly tested on the PASAT.
Diagnosis and Quantification of Cognitive Fatigue in Multiple Sclerosis
Sarah A. Morrow, MD, MS, FRCPC; Heather Rosehart, BScH; Andrew M. Johnson, PhDz
Cognitive and Behavioral Neurology
Objective: To standardize a method to measure cognitive fatigue in patients with multiple sclerosis (MS).
Background: Many patients with MS complain of cognitive fatigue, defined as a decline in cognitive performance during a task requiring sustained activity. Until now there has not been a standardized way to detect cognitive fatigue or determine its severity.
Methods: We administered the Paced Auditory Serial Addition Test (PASAT) to 130 normal controls and 100 randomly selected patients with MS, and compared the number of correct responses between the first third and last third of the test.
Results: The controls averaged 2 more correct responses in the last third of the PASAT than in the first third. The patients with MS averaged 2 to 3 fewer correct responses in the last third than the first third.
Conclusions: Our study showed that comparing responses between the first and last thirds of the PASAT is a reliable method to measure cognitive fatigue in patients with MS. We also present normative data to be used to determine whether patients with MS have cognitive fatigue.
BDI-FS=Beck Depression Inventory–Fast Screen ; NAART=North American Adult Reading Test ; PASAT=Paced Auditory Serial Addition Test ; SC= subject characteristic.
Cognitive impairment is a common and debilitating consequence of multiple sclerosis (MS), affecting about 43% to 60% of patients (Benedict et al, 2006; Rao et al, 1991) and can take a serious toll on quality of life (Benedict et al, 2005). Studies using objective measures of cognitive function have consistently shown impairment, with the most commonly affected domains being speed of information processing, sustained attention, and episodic memory (DeLuca et al, 1993, 1994, 1998, 2004; Demaree et al, 1999; Peyser et al, 1990; Rao et al, 1984, 1989a, 1989b).
Patients with MS also complain of cognitive fatigue. Cognitive fatigue has been defined as a decline in cognitive performance during a task that requires sustained cognitive activity. In other words, people perform worse on a cognitive task the longer they continue it (Schwid et al, 2003).
Cognitive fatigue may or may not be related to generalized fatigue, another common symptom in patients with MS. Generalized fatigue is a patient’s subjective sense of fatigue, most often graded by self-report questionnaires. While generalized fatigue has been shown to affect patients’ subjective assessment of their cognitive performance, it does not affect their performance on objective cognitive measures (Kinsinger et al, 2010; Morrow et al, 2009). In contrast, cognitive fatigue is a subjective sense of worsening performance over time, and can be demonstrated on objective neuropsychological tests (Krupp and Elkins, 2000; Kujala et al, 1995; Schwid et al, 2003).
Until now, a standardized measure of cognitive fatigue had not been developed. A few previous studies lay the groundwork by giving patients measures of processing speed, with worsening performance over time indicating cognitive fatigue. For example, several studies used the Paced Auditory Serial Addition Test (PASAT) (Rao, 1991), comparing the number of correct responses earlier in the test to the number later in the test (Bryant et al, 2004; Kujala et al, 1995; Walker et al, 2012). Schwid et al (2003) showed worsening performance on the PASAT by comparing the number of correct responses on the first third of the test to those on the last third. The PASAT is 1 of 2 tests of processing speed included in 2 widely used tools for monitoring cognitive function in MS: the Minimal Assessment of Cognitive Function in Multiple Sclerosis battery (Benedict et al, 2002, 2006), which our clinic uses, and Rao’s Brief Repeatable Battery of Neuropsychological Tests (Rao, 1991).
Based on this previous work, our goals in this study were, first, to administer the PASAT to a group of “normal” control volunteers who represented the healthy general population, so as to generate normative cognitive fatigue data that could be used to detect impairment in patients with MS, and, second, to compare the normative data to test results from a random sample of patients with MS.
We recruited 130 control volunteers through posters in medical clinics, online advertisements, and social networking sites. We recruited from October 2011 through December 2011. We excluded individuals who had a history of drug or alcohol abuse, psychiatric disorder, attention deficit hyperactivity disorder, learning disability, major medical illness, or any neurologic condition that could cause cognitive impairment, eg, mild cognitive impairment, Alzheimer disease, or traumatic head injury. We required that the controls be between 18 and 55 years of age, have normal vision (≥20/70 corrected) and normal hearing, and be fluent in the English language. (We had used the same control group and similar procedures for a previous study [Morrow, 2013] reporting normative data for the Stroop Color Test in a North American population with MS and showing that Stroop scores, which measure selective attention, are independent of PASAT scores, which measure processing speed.)
We performed a retrospective chart review of 100 patients with MS, ≤55 years of age, whom we had randomly selected from the database of our MS Cognitive Clinic. The Clinic is part of the London (Ontario, Canada) MS Clinic, located at a tertiary care center providing care for >5000 patients with MS in the catchment area. Patients are referred to the MS Cognitive Clinic for baseline testing after a new MS diagnosis; for cognitive assessment because of cognitive complaints noted by the patient, a family member, or a clinician; or as part of a research study. See the Procedures and Measures section below for a description of how we evaluate the patients who are included in the database.
We excluded patients from the current study if they had not taken the PASAT or if they met any of the same exclusion criteria that we used for the controls. We also excluded patients who were taking daily antipsychotic medications, narcotics, amphetamine stimulants, or marijuana, and those taking benzodiazepines other than at night. Finally, we excluded patients if they had scored ≥13 (severe depression) on the Beck Depression Inventory–Fast Screen (BDI-FS) (Beck et al, 2000; Benedict et al, 2003), or >7.5 (essentially restricted to bed or chair) on the Expanded Disability Status Scale (Kurtzke, 1983).
After removing the patients who had been excluded, we randomly selected 100 of the qualified patients using SPSS version 21 Select Cases and Random Selection of Cases commands (SPSS Inc, Chicago, Illinois).
The study was approved by the Research Ethics Board at the University of Western Ontario, London, Ontario, Canada. All of the controls gave written informed consent.
Procedures and Measures
Our MS Cognitive Clinic procedures follow the recommendations of a consensus statement on neuropsychological evaluation of patients with MS (Benedict et al, 2002). Benedict et al advised administering the Minimal Assessment of Cognitive Function in MS battery (Benedict et al, 2002, 2006) as well as unspecified measures of premorbid intelligence, generalized fatigue, and depression. All of the testing should be done during 1 session.
In accordance with these recommendations, our clinic evaluates patients with the Minimal Assessment battery (which includes the PASAT as a measure of processing speed) and these additional tests: (1) The North American Adult Reading Test (NAART) (Blair and Spreen, 1989) is a measure of general intelligence that is considered a good indicator of premorbid ability; the NAART is reported as a standard score with a mean of 100. (2) The Fatigue Severity Scale (Krupp et al, 1989) measures generalized fatigue; this self-reported scale ranges from 0.0 (no fatigue) to 7.0 (severe fatigue). (3) The BDI-FS, validated for the MS population (Beck et al, 2000; Benedict et al, 2003), is used to seek out unreported depressive symptoms. The BDI-FS is a self-reported questionnaire with a composite score of 0 to 21, categorized as no depression (0 to 3), mild depression (4 to 7), moderate depression (8 to 12), and severe depression (≥13). We perform all these tests at the Clinic in a single session that takes about 90 minutes to complete.
In addition to the neuropsychological evaluation, we calculate each patient’s Expanded Disability Status Scale score based on the neurologic examination. The Expanded Disability Status Scale is an ordinal scale of physical disability ranging from 0.0 (no disability) to 10.0 (death from MS).
After we complete the evaluation, we enter the patients’ information into the Clinic database. For the purposes of this study, we obtained from the database our selected patients’ demographic information and their results on the PASAT, NAART, Fatigue Severity Scale, BDI-FS, and Expanded Disability Status Scale.
For the control group, we collected information on age, sex, years of education completed, and ethnicity, as well as past medical history, current medical conditions, and use of medications, both prescribed and over the counter. We administered the NAART, BDI-FS, and PASAT to the controls during a single 20-minute session at the Clinic.
The PASAT is an auditory test of attention and processing speed in which participants listen to a total of 60 single-digit numbers, 1 number voiced every 3 seconds. Participants respond with the sum of the last 2 digits presented. As participants hear each new number, they must disregard their previous response, recall the last 2 digits, and respond with the new sum. The test is scored by the number of correct responses in the 60 trials (Benedict et al, 2006; Rao, 1991).
As with previous studies, we hypothesized that if patients had cognitive fatigue, they would give fewer correct responses toward the end of the trial (Bryant et al, 2004; Schwid et al, 2003; Walker et al, 2012). Thus, to measure cognitive fatigue with the PASAT, we compared the number of correct responses given during the first third of the test to the number given during the last third.
We used the Kolmogorov-Smirnov test to assess our results for normal distribution. To examine the influence of demographic variables as potential confounders on PASAT scores, we used the Pearson correlation for age and years of education, and 1-way analysis of variance for noncontinuous variables. If we found any of the demographic or subject characteristics (SC) to be significantly related to, or potentially confounding, the cognitive fatigue scores, we derived a standardized slope coefficient (beta) from a linear regression analysis and used the resulting value to correct the cognitive fatigue scores with Equation 1:
where xCF is an individual’s score on the cognitive fatigue measure, xSC is an individual’s score on the potential confounding characteristic, and μSC is the mean score for the confounding characteristic (estimated from this study’s control participants). The asterisk indicates that the formula is “corrected” for any detected potential confounder.
We used the cognitive fatigue scores derived in this fashion to calculate a normalized score (zscore) for the patients with MS via Equation 2:
where xCF is an individual’s score on the cognitive fatigue measure, μCF is the population mean for the cognitive fatigue variable (estimated from this study’s controls), and σCF is the population standard deviation for the cognitive fatigue variable (also estimated from this study’s controls). We used this standardized score to estimate where an individual with MS fell on the distribution of scores within the “normal” population.
We tested this norming method within our group of patients with MS by generating corrected cognitive fatigue scores when necessary and calculating cognitive fatigue z scores for the PASAT. We considered a z score <−1.5 (ie, worse than 1.5 standard deviations below the mean) to be abnormal, according to the definition in the Minimal Assessment of Cognitive Function in Multiple Sclerosis battery (Benedict et al, 2006). We report descriptive statistics for our MS group to illustrate how this method assessed their prevalence of cognitive fatigue.
We used SPSS version 21 for all analyses. We set significance at P<0.05.
Table 1 summarizes the study results. The 130 controls comprised 92 women and 38 men. The mean age was 37.2 years. The controls had an average of 14.3 years of education and a mean NAART score of 107.7. The great majority of the controls were white (92.3%) and had been born in Canada (90.0%). Pearson correlation showed that education correlated significantly but weakly with NAART scores (r=0.251, P=0.004). One-way analysis of variance showed that the controls’ BDI-FS scores were significantly higher in the men than the women (mean 1.32 versus 0.75, P=0.027), but fell within the normal range for both men and women (range 0 to 6 for both groups). We did not find other relationships among the demographic variables.
Results of the Kolmogorov-Smirnov test showed that the controls’ PASAT scores were not normally distributed (P=0.003), an unsurprising finding given the test’s ceiling effect with a maximum score of 60 correct responses. The Kolmogorov-Smirnov test did, however, suggest that the PASAT cognitive fatigue scores were normally distributed (P=0.13) and had low skew (0.224) and kurtosis (−0.057), indicating symmetry around the mean and modest rather than extreme differences from the mean.
The controls’ PASAT cognitive fatigue score was 1.9 (standard deviation=2.97), indicating that they improved their performance between the first third and last third of the test. On average, the controls had 2 more correct responses in the last third than the first third. The PASAT cognitive fatigue score correlated significantly but weakly with NAART scores (r=−0.29, P=0.001) and years of education (r=−0.24, P=0.006). The controls’ PASAT cognitive fatigue score did not correlate with BDI-FS scores, disease duration in the patients with MS, or any other potential confounders. Linear regression indicated that the NAART contributed 7.7% (β=−0.293, P<0.001) of the variance. To calculate the PASAT cognitive fatigue z score adjusted for the NAART, we applied values to Equation 1 as follows:
where beta is −0.3 (rounded to 1 decimal place from −0.293) and the mean NAART score is 107.7.
Patients with MS
Our random sample of 100 patients with MS comprised 80 women and 20 men (Table 1). Their mean age was 39.2. The great majority had relapsing-remitting MS and on average had been diagnosed 6.6 years earlier. We found no significant differences in demographic variables between the MS and control groups.
As with the controls, the patients’ NAART scores correlated with their years of education (r=0.462, P<0.001). Unlike the controls, the MS group did worse on the last third of the PASAT than the first third. Their PASAT cognitive fatigue score was −2.3 (standard deviation=3.7), indicating on average 2 or 3 fewer correct responses in the last third of the test. According to the z scores for the PASAT cognitive fatigue score calculated based on the controls, 54 of the patients with MS did not show any cognitive fatigue, while 46 patients were ranked as impaired, falling>1.5 standard deviations below the mean. The patients’ PASAT cognitive fatigue score correlated significantly but weakly with their Fatigue Severity Scale (r=0.11, P=0.029).
Cognitive fatigue can be a disabling symptom but is difficult to detect in a standardized and objective way. To our knowledge, this is the first study that has tried to determine the degree of cognitive fatigue that would be considered diagnostic in MS, based on comparisons with a well-matched healthy control group. This approach promises to be useful both clinically and in further studying cognitive fatigue in patients with MS and other disorders.
Several studies of cognitive fatigue in MS have not shown a drop in performance over time on objective neuropsychological tests. In a study of patients with fatiguing illnesses, Johnson et al (1997) administered the PASAT 4 times over 3 hours, at 30-minute intervals. The authors hypothesized that patients with MS and other conditions causing significant fatigue would not show a practice effect, but instead found that the PASAT scores improved over time with each administration. Similarly, Paul et al (1998) compared patients with MS and controls for learning and vigilance, and tested the participants’ grip strength after 30 minutes of cognitive work. Although the MS group reported an increase in physical and cognitive fatigue, they and the control group did not differ on objective assessment.
In contrast, several studies have noted cognitive fatigue in patients with MS. Kujala et al (1995) gave a task of sustained attention to patients with MS, with and without mild cognitive impairment. Both MS groups’ scores declined over time, while controls’ did not. Krupp and Elkins (2000)administered a verbal learning and memory task twice, separated by a continuous cognitively effortful task of processing speed and attention. As expected, controls recalled 2.6 more words on the second test, presumably the result of a practice effect, while patients with MS recalled 1.8 fewer words. This difference was found even though both groups subjectively reported an increase in mental fatigue over the testing period.
As mentioned, Schwid et al (2003) compared PASAT performance by patients with MS and controls. The MS group had a 5.3% to 5.8% drop in correct responses between the first and last thirds of the test, but the control group did not show a significant decline. In a group of patients with MS, Bryant et al (2004) found consistently lower scores on the second half of the PASAT than the first half.
Unlike the studies that did not show cognitive fatigue over time, those that did show cognitive fatigue used either time or a physical task to induce the fatigue. As we did here, those studies compared participants’ accuracy at the beginning and end of a continuous task of processing speed.
Similarly, Walker et al (2012) also administered the PASAT to patients with MS and controls, comparing accuracy on the first 30 responses to the second 30. The comparison showed no significant effect of cognitive fatigue. The authors then compared the number of correct dyads (2 correct responses in a row) in the first and second halves of the test. Correct dyads are thought to reflect the proportion of time when participants are performing a task as intended. Walker et al’s (2012) patients had significantly fewer correct dyads in the second half of the test, and this worsening correlated with worse scores on the “cognitive dimension” of the Fatigue Impact Scale (Fisk et al, 1994). The authors concluded that the PASAT was a sensitive measure of cognitive fatigue.
Thus, cognitive fatigue has been demonstrated by a worsening in PASAT accuracy over time or by comparing scores on the last third of the test to the first third. However, no previous study has determined a specific threshold or a z score that could be used for clinical care or future research.
In our study, the controls had more correct PASAT responses as the test progressed, as would be expected from a practice effect. By contrast, the patients with MS had fewer correct responses in the last third than in the first third. The patients’ slowing was not always pathological, however; just fewer than half of the patients fell within the impaired range on the PASAT cognitive fatigue z score. Although NAART scores correlated with cognitive fatigue on the PASAT, the beta value was quite small (−0.3), and thus the cognitive fatigue score without its correction would, in all likelihood, still be valid.
Fatigue scores on the Fatigue Severity Scale correlated significantly but only very weakly with cognitive fatigue on the PASAT. This weak correlation was not unexpected. We do not think it is significant in terms of the results of our study, for these reasons:
- Previous studies have not found any relationship between subjective fatigue and decline in performance on the PASAT (Bryant et al, 2004).
- Subjective fatigue measures have not been found to correlate with cognitive impairment in working memory (Bailey et al, 2007; Johnson et al, 1997; Morrow et al, 2009), short-term memory (Johnson et al, 1998; Morrow et al, 2009), executive function (Krupp and Elkins, 2000; Morrow et al, 2009), or vigilance (Paul et al, 1998).
- The Fatigue Severity Scale is a measure of physical fatigue, focusing on how fatigue affects the ability of patients with MS to perform their activities of daily living. This construct is different from cognitive fatigue as we defined it in this study.
Our study had several limitations:
- Our MS group may have included a disproportionate number of patients with cognitive impairment, as many members of the group had been referred for a cognitive assessment.
- We administered the PASAT to the control group along with just a few other tests that took a total of only 20 minutes. However, the patients took the PASAT as part of a large 90-minute test battery. The long testing session may have put the patients at higher risk for cognitive fatigue. We did not control specifically for this possibility.
- Although the PASAT is known to cause a notable practice effect (Barker-Collo, 2005; Bever et al, 1995), we did not determine whether the patients with MS had had previous exposure to the test.
- Some researchers have postulated that examining dyads on the PASAT would discriminate differences in working memory more sensitively between patients with MS and controls than would the number of correct responses (Fisk and Archibald, 2001; Gonzalez et al, 2006a, 2006b). We chose not to analyze dyads in this study, as we were focusing on the change over time with sustained attention. It is not known whether working memory is a component of cognitive fatigue; this would be an interesting question to address in future research.
- Because of the small proportion of non-white participants in both the control and MS groups, we cannot draw any conclusions about whether ethnicity influences cognitive fatigue.
In summary, this is the first study to standardize the measurement and diagnosis of cognitive fatigue in patients with MS. Our data, and the z scores created, can be used to diagnose cognitive fatigue clinically, allowing validation of the patient’s subjective symptoms and aiding in management planning. Being able to quantify the severity of cognitive fatigue also enables physicians to follow changes in this common symptom over a patient’s course. Finally, these scores can be used in research protocols to evaluate the effectiveness of potential treatments for cognitive fatigue, whether pharmacological, physiological, or behavioral.
Bailey A, Channon S, Beaumont JG. 2007. The relationship between subjective fatigue and cognitive fatigue in advanced multiple sclerosis. Mult Scler. 13:73–80.
Barker-Collo SL. 2005. Within session practice effects on the PASAT in clients with multiple sclerosis. Arch Clin Neuropsychol. 20:145–152.
Beck AT, Steer RA, Brown GK. 2000. BDIs-Fast Screen for Medical Patients: Manual. San Antonio, Texas: Psychological Corporation.
Benedict RH, Cookfair D, Gavett R, et al. 2006. Validity of the minimal assessment of cognitive function in multiple sclerosis (MACFIMS). J Int Neuropsychol Soc. 12:549–558.
Benedict RH, Fischer JS, Archibald CJ, et al. 2002. Minimal neuropsychological assessment of MS patients: a consensus approach. Clin Neuropsychol. 16:381–397.
Benedict RH, Fishman I, McClellan MM, et al. 2003. Validity of the Beck Depression Inventory–Fast Screen in multiple sclerosis. Mult Scler. 9:393–396.
Benedict RH, Wahlig E, Bakshi R, et al. 2005. Predicting quality of life in multiple sclerosis: accounting for physical disability, fatigue, cognition, mood disorder, personality, and behavior change. J Neurol Sci. 231:29–34.
Bever CT Jr, Grattan L, Panitch HS, et al. 1995. The Brief Repeatable Battery of Neuropsychological Tests for Multiple Sclerosis: a preliminary serial study. Mult Scler. 1:165–169.
Blair JR, Spreen O. 1989. Predicting premorbid IQ: a revision of the National Adult Reading Test. Clin Neuropsychol. 3:129–136.
Bryant D, Chiaravalloti N, DeLuca J. 2004. Objective measurement of cognitive fatigue in multiple sclerosis. Rehabil Psychol. 49:114–122.
DeLuca J, Barbieri-Berger S, Johnson SK. 1994. The nature of memory impairments in multiple sclerosis: aquisition versus retrieval. J Clin Exp Neuropsychol. 16:183–189.
DeLuca J, Chelune GJ, Tulsky DS, et al. 2004. Is speed of processing or working memory the primary information processing deficit in multiple sclerosis? J Clin Exp Neuropsychol. 26:550–562.
DeLuca J, Gaudino EA, Diamond BJ, et al. 1998. Acquisition and storage deficits in multiple sclerosis. J Clin Exp Neuropsychol. 20:376–390.
DeLuca J, Johnson SK, Natelson BH. 1993. Information processing efficiency in chronic fatigue syndrome and multiple sclerosis. Arch Neurol. 50:301–304.
Demaree HA, DeLuca J, Gaudino EA, et al. 1999. Speed of information processing as a key deficit in multiple sclerosis: implications for rehabilitation. J Neurol Neurosurg Psychiatry. 67:661–663.
Fisk JD, Archibald CJ. 2001. Limitations of the Paced Auditory Serial Addition Test as a measure of working memory in patients with multiple sclerosis. J Int Neuropsychol Soc. 7:363–372.
Fisk JD, Ritvo PG, Ross L, et al. 1994. Measuring the functional impact of fatigue: initial validation of the Fatigue Impact Scale. Clin Infect Dis. 18:S79–S83.
Gonzalez R, Grant I, Miller SW, et al. 2006a. Demographically adjusted normative standards for new indices of performance on the Paced Auditory Serial Addition Task (PASAT). Clin Neuropsychol. 20:396–413.
Gonzalez R, Miller SW, Carey CL, et al. 2006b. Association between dyads and correct responses on the Paced Auditory Serial Addition Task (PASAT). Assessment. 13:381–384.
Johnson SK, DeLuca J, Diamond BJ. 1998. Memory dysfunction in fatiguing illness: examining interference and distraction in shortterm memory. Cogn Neuropsychiatry. 3:269–285.
Johnson SK, Lange G, DeLuca J, et al. 1997. The effects of fatigue on neuropsychological performance in patients with chronic fatigue syndrome, multiple sclerosis and depression. Appl Neuropsychol. 4:145–153.
Kinsinger SW, Lattie E, Mohr DC. 2010. Relationship between depression, fatigue, subjective cognitive impairment, and objective neuropsychological functioning in patients with multiple sclerosis. Neuropsychology. 24:573–580.
Krupp LB, Elkins LE. 2000. Fatigue and declines in cognitive functioning in multiple sclerosis. Neurology. 55:934–939.
Krupp LB, LaRocca NG, Muir-Nash J, et al. 1989. The fatigue severity scale: application to patients with multiple sclerosis and systemic lupus erythematosus. Arch Neurol. 46:1121–1123.
Kujala P, Portin R, Revonsuo A, et al. 1995. Attention related performance in two cognitively different subgroups of patients with multiple sclerosis. J Neurol Neurosurg Psychiatry. 59:77–82.
Kurtzke JF. 1983. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology. 33: 1444–1452.
Morrow SA. 2013. Normative data for the Stroop Color Word Test for a North American population. Can J Neurol Sci. 40:842–847.
Morrow SA, Weinstock-Guttman B, Munschauer FE, et al. 2009. Subjective fatigue is not associated with cognitive impairment in multiple sclerosis: cross-sectional and longitudinal analysis. Mult Scler. 15:998–1005.
Paul RH, Beatty WW, Schneider R, et al. 1998. Cognitive and physical fatigue in multiple sclerosis: relations between self-report and objective performance. Appl Neuropsychol. 5:143–148.
Peyser JM, Rao SM, LaRocca NG, et al. 1990. Guidelines for neuropsychological research in multiple sclerosis. Arch Neurol. 47:94–97.
Rao SM. 1991. A Manual for the Brief Repeatable Battery of Neuropsychological Tests in Multiple Sclerosis. New York, New York: National Multiple Sclerosis Society.
Rao SM, Hammeke TA, McQuillen MP, et al. 1984. Memory disturbance in chronic progressive multiple sclerosis. Arch Neurol. 41:625–631.
Rao SM, Leo GJ, Bernardin L, et al. 1991. Cognitive dysfunction in multiple sclerosis: I. frequency, patterns, and prediction. Neurology. 41:685–691.
Rao SM, Leo GJ, St Aubin-Faubert P. 1989a. On the nature of memory disturbance in multiple sclerosis. J Clin Exp Neuropsychol. 11:699–712.
Rao SM, St Aubin-Faubert P, Leo GJ. 1989b. Information processing speed in patients with multiple sclerosis. J Clin Exp Neuropsychol. 11:471–477.
Schwid SR, Tyler CM, Scheid EA, et al. 2003. Cognitive fatigue during a test requiring sustained attention: a pilot study. Mult Scler. 9:503–508.
Walker LA, Berard JA, Berrigan LI, et al. 2012. Detecting cognitive fatigue in multiple sclerosis: method matters. J Neurol Sci. 316:86–92.