Practice effects due to serial cognitive assessment: Implications for preclinical Alzheimer's disease randomized controlled trials

Terry E Goldberg, Philip D Harvey, Keith A Wesnes, Peter J Snyder, Lon S Schneider, Terry E Goldberg, Philip D Harvey, Keith A Wesnes, Peter J Snyder, Lon S Schneider

Abstract

Introduction: Practice effects are characteristic of nearly all standard cognitive tasks when repeated during serial assessments and are frequently important confounders in clinical trials.

Methods: We summarize evidence that gains in neuropsychological test performance scores associated with practice effects occur as artifactual changes associated with serial testing within clinical trials. We identify and emphasize such gains in older, non-cognitively impaired individuals and estimate an effect size of 0.25 for composite cognitive measures in older populations assessed three times in a 6- to 12-month period.

Results: We identified three complementary approaches that can be used to attenuate practice effects: (1) massed practice in a prebaseline period to reduce task familiarity effects; (2) tests designed to reduce practice-related gains so that item-specific driven improvements are minimized by using tasks that minimize strategy and/or maximize interitem interference; and (3) well-matched alternate forms.

Discussion: We have drawn attention to and increased awareness of practice effect-related gains that could result in type 1 or type 2 errors in trials. Successfully managing practice effects will eliminate a large source of error and reduce the likelihood of misinterpretation of clinical trials outcomes.

Keywords: Alzheimer's disease; Clinical trials; Cognition; Neuropsychology; Practice effects; Preclinical Alzheimer's disease; Serial assessment.

Figures

Fig. 1
Fig. 1
Change from year 0 data in the ADNI studies for the Rey Auditory Verbal Learning Test. Ascending values reflect improvements and descending impairments. The two parallel forms (A and B) are clearly not of equivalent difficulty. Abbreviations: ADNI, Alzheimer's Disease Neuroimaging Initiative; AVLT, Auditory Verbal Learning Test; LS, Least Square.

References

    1. Goldberg T.E., Goldman R.S., Burdick K.E., Malhotra A.K., Lencz T., Patel R.C. Cognitive improvement after treatment with second-generation antipsychotic medications in first-episode schizophrenia: Is it a practice effect? Arch Gen Psychiatry. 2007;64:1115–1122.
    1. Goldberg T.E., Keefe R.S., Goldman R.S., Robinson D.G., Harvey P.D. Circumstances under which practice does not make perfect: a review of the practice effect literature in schizophrenia and its relevance to clinical treatment studies. Neuropsychopharmacology. 2010;35:1053–1062.
    1. Pietrzak R.H., Snyder P.J., Maruff P. Amphetamine-related improvement in executive function in patients with chronic schizophrenia is modulated by practice effects. Schizophr Res. 2010;124:176–182.
    1. Sperling R.A., Aisen P.S., Beckett L.A., Bennett D.A., Craft S., Fagan A.M. Toward defining the preclinical stages of Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease. Alzheimers Dement. 2011;7:280–292.
    1. Kozauer N., Katz R. Regulatory innovation and drug development for early-stage Alzheimer's disease. N Engl J Med. 2013;368:1169–1171.
    1. Budson A.E., Price B.H. Memory dysfunction. N Engl J Med. 2005;352:692–699.
    1. Wesnes K., Pincock C. Practice effects on cognitive tasks: a major problem? Lancet Neurol. 2002;1:473.
    1. Thorndike E.L., Woodworth R.S. The influence of movement in one mental function upon the efficiency of other functions. Psychol Rev. 1901;8:247–261.
    1. Kelly A.M., Garavan H. Human functional neuroimaging of brain changes associated with practice. Cereb Cortex. 2005;15:1089–1102.
    1. Duff K., Lyketsos C.G., Beglinger L.J., Chelune G., Moser D.J., Arndt S. Practice effects predict cognitive outcome in amnestic mild cognitive impairment. Am J Geriatr Psychiatry. 2011;19:932–939.
    1. Dodge H.H., Wang C.N., Chang C.C., Ganguli M. Terminal decline and practice effects in older adults without dementia: the MoVIES project. Neurology. 2011;77:722–730.
    1. Meinert C.L., Breitner J.C. Chronic disease long-term drug prevention trials: lessons from the Alzheimer's Disease Anti-inflammatory Prevention Trial (ADAPT) Alzheimers Dement. 2008;4(1 Suppl 1):S7–S14.
    1. Calamia M., Markon K., Tranel D. Scoring higher the second time around: meta-analyses of practice effects in neuropsychological assessment. Clin Neuropsychol. 2012;26:543–570.
    1. Eagger S., Morant N., Levy R., Sahakian B. Tacrine in Alzheimer's disease. Time course of changes in cognitive function and practice effects. Br J Psychiatry. 1992;160:36–40.
    1. Galasko D., Abramson I., Corey-Bloom J., Thal L.J. Repeated exposure to the Mini-Mental State Examination and the Information-Memory-Concentration Test results in a practice effect in Alzheimer's disease. Neurology. 1993;43:1559–1563.
    1. Rogers S.L., Farlow M.R., Doody R.S., Mohs R., Friedhoff L.T. A 24-week, double-blind, placebo-controlled trial of donepezil in patients with Alzheimer's disease. Donepezil Study Group. Neurology. 1998;50:136–145.
    1. Birks J., Harvey R.J. Donepezil for dementia due to Alzheimer's disease. Cochrane Database Syst Rev. 2006:CD001190.
    1. Rogers S.L., Doody R.S., Mohs R.C., Friedhoff L.T. Donepezil improves cognition and global function in Alzheimer disease: a 15-week, double-blind, placebo-controlled study. Donepezil Study Group. Arch Intern Med. 1998;158:1021–1031.
    1. Petersen R.C., Thomas R.G., Grundman M., Bennett D., Doody R., Ferris S. Vitamin E and donepezil for the treatment of mild cognitive impairment. N Engl J Med. 2005;352:2379–2388.
    1. Bartels C., Wegrzyn M., Wiedl A., Ackermann V., Ehrenreich H. Practice effects in healthy adults: a longitudinal study on frequent repetitive cognitive testing. BMC Neurosci. 2010;11:118.
    1. Rabbitt P., Diggle P., Smith D., Holland F., Mc Innes L. Identifying and separating the effects of practice and of cognitive ageing during a large longitudinal study of elderly community residents. Neuropsychologia. 2001;39:532–543.
    1. Krenk L., Rasmussen L.S., Siersma V.D., Kehlet H. Short-term practice effects and variability in cognitive testing in a healthy elderly population. Exp Gerontol. 2012;47:432–436.
    1. Frank R., Wiederholt W.C., Kritz-Silverstein D.K., Salmon D.P., Barrett-Connor E. Effects of sequential neuropsychological testing of an elderly community-based sample. Neuroepidemiology. 1996;15:257–268.
    1. Wesnes K.A., Schneider L.S. Are neuropsychological tests such as those used in ADNI suitable for long-term trials of cognition enhancers for preclinical Alzheimer's disease? J Nutr Health Aging. 2012;16:810.
    1. Salthouse T.A., Tucker-Drob E.M. Implications of short-term retest effects for the interpretation of longitudinal change. Neuropsychology. 2008;22:800–811.
    1. Machulda M.M., Pankratz V.S., Christianson T.J., Ivnik R.J., Mielke M.M., Roberts R.O. [Formula: see text]Practice effects and longitudinal cognitive change in normal aging vs. incident mild cognitive impairment and dementia in the Mayo Clinic Study of Aging. Clin Neuropsychol. 2013;27:1247–1264.
    1. Harvey P.D., Reichenberg A., Bowie C.R., Patterson T.L., Heaton R.K. The course of neuropsychological performance and functional capacity in older patients with schizophrenia: Influences of previous history of long-term institutional stay. Biol Psychiatry. 2010;67:933–939.
    1. Keefe R.S., Bilder R.M., Davis S.M., Harvey P.D., Palmer B.W., Gold J.M. Neurocognitive effects of antipsychotic medications in patients with chronic schizophrenia in the CATIE trial. Arch Gen Psychiatry. 2007;64:633–647.
    1. Davidson M., Galderisi S., Weiser M., Werbeloff N., Fleischhacker W.W., Keefe R.S. Cognitive effects of antipsychotic drugs in first-episode schizophrenia and schizophreniform disorder: a randomized, open-label clinical trial (EUFEST) Am J Psychiatry. 2009;166:675–682.
    1. Beglinger L.J., Gaydos B., Tangphao-Daniels O., Duff K., Kareken D.A., Crawford J. Practice effects and the use of alternate forms in serial neuropsychological testing. Arch Clin Neuropsychol. 2005;20:517–529.
    1. Harvey P.D., Palmer B.W., Heaton R.K., Mohamed S., Kennedy J., Brickman A. Stability of cognitive performance in older patients with schizophrenia: an 8-week test-retest study. Am J Psychiatry. 2005;162:110–117.
    1. Temkin N.R., Heaton R.K., Grant I., Dikmen S.S. Detecting significant change in neuropsychological test performance: a comparison of four models. J Int Neuropsychol Soc. 1999;5:357–369.
    1. McClelland G.R. The effects of practice on measures of performance. Hum Psychopharmacol. 1987;2:109–118.
    1. Duff K., Westervelt H.J., McCaffrey R.J., Haase R.F. Practice effects, test-retest stability, and dual baseline assessments with the California Verbal Learning Test in an HIV sample. Arch Clin Neuropsychol. 2001;16:461–476.
    1. Collie A., Maruff P., Darby D.G., McStephen M. The effects of practice on the cognitive test performance of neurologically normal individuals assessed at brief test-retest intervals. J Int Neuropsychol Soc. 2003;9:419–428.
    1. Falleti M.G., Maruff P., Collie A., Darby D.G. Practice effects associated with the repeated assessment of cognitive function using the CogState battery at 10-minute, one week and one month test-retest intervals. J Clin Exp Neuropsychol. 2006;28:1095–1112.
    1. Beglinger L.J., Ahmed S., Derby M.A., Siemers E., Fastenau P.S., Crawford-Miller J. Neuropsychological practice effects and change detection in people with schizophrenia. Schizophr Res. 2003;62:191–194.
    1. Vinogradov S., Fisher M., de Villers-Sidani E. Cognitive training for impaired neural systems in neuropsychiatric illness. Neuropsychopharmacology. 2012;37:43–76.
    1. Pietrzak R.H., Maruff P., Snyder P.J. Methodological improvements in quantifying cognitive change in clinical trials: an example with single-dose administration of donepezil. J Nutr Health Aging. 2009;13:268–273.
    1. Olanow C.W., Rascol O., Hauser R., Feigin P.D., Jankovic J., Lang A. A double-blind, delayed-start trial of rasagiline in Parkinson's disease. N Engl J Med. 2009;361:1268–1278.
    1. Karantzoulis S., Novitski J., Gold M., Randolph C. The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): Utility in detection and characterization of mild cognitive impairment due to Alzheimer's disease. Arch Clin Neuropsychol. 2013;28:837–844.
    1. Pietrzak R.H., Snyder P.J., Jackson C.E., Olver J., Norman T., Piskulic D. Stability of cognitive impairment in chronic schizophrenia over brief and intermediate re-test intervals. Hum Psychopharmacol. 2009;24:113–121.
    1. Pietrzak R.H., Olver J., Norman T., Piskulic D., Maruff P., Snyder P.J. A comparison of the CogState Schizophrenia Battery and the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) Battery in assessing cognitive impairment in chronic schizophrenia. J Clin Exp Neuropsychol. 2009;31:848–859.
    1. Maruff P., Thomas E., Cysique L., Brew B., Collie A., Snyder P. Validity of the CogState brief battery: relationship to standardized tests and sensitivity to cognitive impairment in mild traumatic brain injury, schizophrenia, and AIDS dementia complex. Arch Clin Neuropsychol. 2009;24:165–178.
    1. Allain H., Neuman E., Malbezin M., Salzman V., Guez D., Wesnes K. Bridging study of S12024 in 53 in-patients with Alzheimer's disease. J Am Geriatr Soc. 1997;45:125–126.
    1. Vellas B., Cunha L., Gertz H.J., De Deyn P.P., Wesnes K., Hammond G. Early onset effects of galantamine treatment on attention in patients with Alzheimer's disease. Curr Med Res Opin. 2005;21:1423–1429.
    1. Galvin J.E., Cornblatt B., Newhouse P., Ancoli-Israel S., Wesnes K., Williamson D. Effects of galantamine on measures of attention: results from 2 clinical trials in Alzheimer disease patients with comparisons to donepezil. Alzheimer Dis Assoc Disord. 2008;22:30–38.
    1. Wesnes K., Edgar C., Andreasen N., Annas P., Basun H., Lannfelt L. Computerized cognition assessment during acetylcholinesterase inhibitor treatment in Alzheimer's disease. Acta Neurol Scand. 2010;122:270–277.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe