UCancellation: A new mobile measure of selective attention and concentration

Anja Pahor, Randy E Mester, Audrey A Carrillo, Eunice Ghil, Jason F Reimer, Susanne M Jaeggi, Aaron R Seitz, Anja Pahor, Randy E Mester, Audrey A Carrillo, Eunice Ghil, Jason F Reimer, Susanne M Jaeggi, Aaron R Seitz

Abstract

Measuring selective attention in a speeded task can provide valuable insight into the concentration ability of an individual, and can inform neuropsychological assessment of attention in aging, traumatic brain injury, and in various psychiatric disorders. There are only a few tools to measure selective attention that are freely available, psychometrically validated, and can be used flexibly both for in-person and remote assessment. To address this gap, we developed a self-administrable, mobile-based test called "UCancellation" (University of California Cancellation), which was designed to assess selective attention and concentration and has two stimulus sets: Letters and Pictures. UCancellation takes less than 7 minutes to complete, is automatically scored, has multiple forms to allow repeated testing, and is compatible with a variety of iOS and Android devices. Here we report the results of a study that examined parallel-test reliability and convergent validity of UCancellation in a sample of 104 college students. UCancellation Letters and Pictures showed adequate parallel test reliability (r = .71-.83, p < 0.01) and internal consistency (ɑ = .73-.91). It also showed convergent validity with another widely used cancellation task, d2 Test of Attention (r = .43-.59, p < 0.01), and predicted performance on a cognitive control composite (r = .34-.41, p < 0.05). These results suggest that UCancellation is a valid test of selective attention and inhibitory control, which warrants further data collection to establish norms.

Keywords: Cancellation; Inhibitory control; Selective attention; Software; Validation.

© 2022. The Psychonomic Society, Inc.

Figures

Fig. 1
Fig. 1
Screenshots of UCancellation Letters (top) and Pictures (bottom). A countdown timer is present in the top left corner of the screen and a green arrow can be pressed to advance to the next line (if row time has not been exceeded)
Fig. 2
Fig. 2
Violin plot ( Hoffmann 2021) of Concentration Performance in UCancellation Letters and UCancellation Pictures groups in session 1 (S1) and session 2 (S2). Wider parts of the violin plot indicate a higher probability for a data point to occur in a certain section
Fig. 3
Fig. 3
Scatter plot of Concentration Performance in UCancellation Pictures (N = 53) and UCancellation Letters (N = 48) across two sessions. Two outliers were removed from the UCancellation Letters group based on performance in the second session
Fig. 4
Fig. 4
Scatter plots of Concentration Performance in UCancellation Pictures (left column) and UCancellation Letters (right column) in relation to d2 (row 1), the EXAMINER Cognitive Control composite (row 2) and Countermanding RT measures (rows 3–5)

References

    1. Bates ME, Lemay EP. The d2 Test of attention: construct validity and extensions in scoring techniques. Journal of the International Neuropsychological Society. 2004;10(3):392–400. doi: 10.1017/S135561770410307X.
    1. Benjamins JS, Dalmaijer ES, Ten Brink AF, Nijboer TCW, Van der Stigchel S. Multi-target visual search organisation across the lifespan: cancellation task performance in a large and demographically stratified sample of healthy adults. Neuropsychology, Development, and Cognition. Section B, Aging, Neuropsychology and Cognition. 2019;26(5):731–748. doi: 10.1080/13825585.2018.1521508.
    1. Bignardi G, Dalmaijer ES, Anwyl-Irvine A, Astle DE. Collecting big data with small screens: Group tests of children’s cognition with touchscreen tablets are reliable and valid. Behavior Research Methods. 2021;53(4):1515–1529. doi: 10.3758/s13428-020-01503-3.
    1. Blatter K, Cajochen C. Circadian rhythms in cognitive performance: methodological constraints, protocols, theoretical underpinnings. Physiology & Behavior. 2007;90(2–3):196–208. doi: 10.1016/j.physbeh.2006.09.009.
    1. Blotenberg, I. and Schmidt-Atzert, L. (2019). On the locus of the practice effect in sustained attention tests. Journal of Intelligence, 7(2), 12. 10.3390/jintelligence7020012
    1. Bond CF, Titus LJ. Social facilitation: A meta-analysis of 241 studies. Psychological Bulletin. 1983;94(2):265–292. doi: 10.1037/0033-2909.94.2.265.
    1. Brickenkamp R, Zillmer E. Test d2: Concentration-Endurance Test. CJ Hogrefe; 1998.
    1. Broadbent DE, Cooper PF, FitzGerald P, Parkes KR. The Cognitive Failures Questionnaire (CFQ) and its correlates. The British Journal of Clinical Psychology. 1982;21(Pt 1):1–16. doi: 10.1111/j.2044-8260.1982.tb01421.x.
    1. Brodeur DA, Pond M. The development of selective attention in children with attention deficit hyperactivity disorder. Journal of Abnormal Child Psychology. 2001;29(3):229–239. doi: 10.1023/A:1010381731658.
    1. Bruggemans EF, Van de Vijver FJ, Huysmans HA. Assessment of cognitive deterioration in individual patients following cardiac surgery: correcting for measurement error and practice effects. Journal of Clinical and Experimental Neuropsychology. 1997;19(4):543–559. doi: 10.1080/01688639708403743.
    1. Bühner M, Ziegler M, Bohnes B, Lauterbach K. Übungseffekte in den TAP Untertests Test Go/Nogo und Geteilte Aufmerksamkeit sowie dem Aufmerksamkeits-Belastungstest (d2) Zeitschrift für Neuropsychologie. 2006;17(3):191–199. doi: 10.1024/1016-264X.17.3.191.
    1. Carver CS, White TL. Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The BIS/BAS Scales. Journal of Personality and Social Psychology. 1994;67(2):319–333. doi: 10.1037/0022-3514.67.2.319.
    1. Commodari E, Guarnera M. Attention and aging. Aging Clinical and Experimental Research. 2008;20(6):578–584. doi: 10.1007/BF03324887.
    1. Dalmaijer ES, Van der Stigchel S, Nijboer TCW, Cornelissen THW, Husain M. CancellationTools: All-in-one software for administration and analysis of cancellation tasks. Behavior Research Methods. 2015;47(4):1065–1075. doi: 10.3758/s13428-014-0522-7.
    1. Davidson MC, Amso D, Anderson LC, Diamond A. Development of cognitive control and executive functions from 4 to 13 years: evidence from manipulations of memory, inhibition, and task switching. Neuropsychologia. 2006;44(11):2037–2078. doi: 10.1016/j.neuropsychologia.2006.02.006.
    1. Della Sala S, Laiacona M, Spinnler H, Ubezio C. A cancellation test: its reliability in assessing attentional deficits in Alzheimer’s disease. Psychological Medicine. 1992;22(4):885–901. doi: 10.1017/S0033291700038460.
    1. Duchek JM, Balota DA, Tse C-S, Holtzman DM, Fagan AM, Goate AM. The utility of intraindividual variability in selective attention tasks as an early marker for Alzheimer’s disease. Neuropsychology. 2009;23(6):746–758. doi: 10.1037/a0016583.
    1. Dye MWG, Green CS, Bavelier D. Increasing Speed of Processing With Action Video Games. Current Directions in Psychological Science. 2009;18(6):321–326. doi: 10.1111/j.1467-8721.2009.01660.x.
    1. Ferber S, Karnath HO. How to assess spatial neglect--line bisection or cancellation tasks? Journal of Clinical and Experimental Neuropsychology. 2001;23(5):599–607. doi: 10.1076/jcen.23.5.599.1243.
    1. Fernández-Castillo A, Caurcel MJ. State test-anxiety, selective attention and concentration in university students. International Journal of Psychology : Journal International de Psychologie. 2015;50(4):265–271. doi: 10.1002/ijop.12092.
    1. Flehmig, H. C., Steinborn, M., Langner, R., Scholz, A., and Westhoff, K. (2007). Assessing intraindividual variability in sustained attention: Reliability, relation to speed and accuracy, and practice effects. Psychology Science, 49(2), 132.
    1. Gauthier, L., Dehaut, F., and Joanette, Y. (1989). The bells test: a quantitative and qualitative test for visual neglect. International Journal of Clinical Neuropsychology, 11(2), 49–54.
    1. Green CS, Bavelier D. Action video game modifies visual selective attention. Nature. 2003;423(6939):534–537. doi: 10.1038/nature01647.
    1. Green CS, Kattner F, Eichenbaum A, et al. Playing some video games but not others is related to cognitive abilities: A critique of unsworth et al. (2015) Psychological Science. 2017;28(5):679–682. doi: 10.1177/0956797616644837.
    1. Hagemeister C. How useful is the power law of practice for recognizing practice in concentration tests? European Journal of Psychological Assessment. 2007;23(3):157–165. doi: 10.1027/1015-5759.23.3.157.
    1. Halligan, P. W., Marshall, J. C., and Wade, D. T. (1989). Visuospatial neglect: underlying factors and test sensitivity. The Lancet, 334(8668), 908–911.
    1. Hoffmann, H. (2022). Violin Plot (), MATLAB Central File Exchange. Retrieved March 10, 2021.
    1. Huang H-C, Wang T-Y. Stimulus effects on cancellation task performance in children with and without dyslexia. Behavior Research Methods. 2009;41(2):539–545. doi: 10.3758/BRM.41.2.539.
    1. Hutchinson CV, Barrett DJK, Nitka A, Raynes K. Action video game training reduces the Simon Effect. Psychonomic Bulletin & Review. 2016;23(2):587–592. doi: 10.3758/s13423-015-0912-6.
    1. Huygelier H, Gillebert CR. Quantifying egocentric spatial neglect with cancellation tasks: A theoretical validation. Journal of Neuropsychology. 2020;14(1):1–19. doi: 10.1111/jnp.12177.
    1. Jonkman LM. Selective Attention Deficits in Children with Attention Deficit Hyperactivity Disorder. In: Gozal D, Molfese DL, editors. Attention deficit hyperactivity disorder. Humana Press; 2005. pp. 255–275.
    1. Kramer, J. H. (2010). Executive abilities: measures and instruments for neurobehavioral evaluation and research (EXAMINER) User Manual 3.6. . Available at: . Accessed 20 October 2021.
    1. Kramer JH, Mungas D, Possin KL, et al. NIH EXAMINER: conceptualization and development of an executive function battery. Journal of the International Neuropsychological Society. 2014;20(1):11–19. doi: 10.1017/S1355617713001094.
    1. Lee P, Lu W-S, Liu C-H, Lin H-Y, Hsieh C-L. Test-Retest Reliability and Minimal Detectable Change of the D2 Test of Attention in Patients with Schizophrenia. Archives of Clinical Neuropsychology. 2018;33(8):1060–1068. doi: 10.1093/arclin/acx123.
    1. Levinoff EJ, Li KZH, Murtha S, Chertkow H. Selective attention impairments in Alzheimer’s disease: evidence for dissociable components. Neuropsychology. 2004;18(3):580–588. doi: 10.1037/0894-4105.18.3.580.
    1. Maassen GH, Bossema E, Brand N. Reliable change and practice effects: outcomes of various indices compared. Journal of Clinical and Experimental Neuropsychology. 2009;31(3):339–352. doi: 10.1080/13803390802169059.
    1. Maddox WT, Filoteo JV, Delis DC, Salmon DP. Visual selective attention deficits in patients with Parkinson’s disease: A quantitative model-based approach. Neuropsychology. 1996;10(2):197–218. doi: 10.1037/0894-4105.10.2.197.
    1. McDowd JM, Filion DL. Aging, selective attention, and inhibitory processes: A psychophysiological approach. Psychology and Aging. 1992;7(1):65–71. doi: 10.1037/0882-7974.7.1.65.
    1. Miller J, Ulrich R. Mental chronometry and individual differences: modeling reliabilities and correlations of reaction time means and effect sizes. Psychonomic Bulletin & Review. 2013;20(5):819–858. doi: 10.3758/s13423-013-0404-5.
    1. Noyes JM, Garland KJ. Computer- vs. paper-based tasks: are they equivalent? Ergonomics. 2008;51(9):1352–1375. doi: 10.1080/00140130802170387.
    1. Paap KR, Sawi O. The role of test-retest reliability in measuring individual and group differences in executive functioning. Journal of Neuroscience Methods. 2016;274:81–93. doi: 10.1016/j.jneumeth.2016.10.002.
    1. Parks S, Bartlett A, Wickham A, Myors B. Developing a computerized test of perceptual/clerical speed. Computers in Human Behavior. 2001;17(1):111–124. doi: 10.1016/S0747-5632(00)00031-5.
    1. Prabhakaran R, Kraemer DJM, Thompson-Schill SL. Approach, avoidance, and inhibition: personality traits predict cognitive control abilities. Personality and Individual Differences. 2011;51(4):439–444. doi: 10.1016/j.paid.2011.04.009.
    1. Ramani GB, Daubert EN, Lin GC, Kamarsu S, Wodzinski A, Jaeggi SM. Racing dragons and remembering aliens: Benefits of playing number and working memory games on kindergartners’ numerical knowledge. Developmental Science. 2020;23(4):e12908. doi: 10.1111/desc.12908.
    1. Rivera D, Salinas C, Ramos-Usuga D, et al. Concentration Endurance Test (d2): Normative data for Spanish-speaking pediatric population. NeuroRehabilitation. 2017;41(3):661–671. doi: 10.3233/NRE-172248.
    1. Robertson IH, Manly T, Andrade J, Baddeley BT, Yiend J. “Oops!”: performance correlates of everyday attentional failures in traumatic brain injured and normal subjects. Neuropsychologia. 1997;35(6):747–758. doi: 10.1016/S0028-3932(97)00015-8.
    1. Rorden C, Karnath H-O. A simple measure of neglect severity. Neuropsychologia. 2010;48(9):2758–2763. doi: 10.1016/j.neuropsychologia.2010.04.018.
    1. Saucier G. Mini-markers: a brief version of Goldberg’s unipolar big-five markers. Journal of Personality Assessment. 1994;63(3):506–516. doi: 10.1207/s15327752jpa6303_8.
    1. Spaccavento S, Marinelli CV, Nardulli R, et al. Attention Deficits in Stroke Patients: The Role of Lesion Characteristics, Time from Stroke, and Concomitant Neuropsychological Deficits. Behavioural neurology. 2019;2019:7835710. doi: 10.1155/2019/7835710.
    1. Steinborn MB, Huestegge L. Socially alerted cognition evoked by a confederate’s mere presence: analysis of reaction-time distributions and delta plots. Psychological Research. 2020;84(5):1424–1439. doi: 10.1007/s00426-019-01143-z.
    1. Steinborn MB, Langner R, Flehmig HC, Huestegge L. Methodology of performance scoring in the d2 sustained-attention test: Cumulative-reliability functions and practical guidelines. Psychological Assessment. 2018;30(3):339–357. doi: 10.1037/pas0000482.
    1. Stevens C, Bavelier D. The role of selective attention on academic foundations: a cognitive neuroscience perspective. Developmental cognitive neuroscience. 2012;2(Suppl 1):S30–48. doi: 10.1016/j.dcn.2011.11.001.
    1. Uttl B, Pilkenton-Taylor C. Letter cancellation performance across the adult life span. The Clinical Neuropsychologist. 2001;15(4):521–530. doi: 10.1076/clin.15.4.521.1881.
    1. van Steenbergen H, Band GPH, Hommel B. Reward counteracts conflict adaptation. Evidence for a role of affect in executive control. Psychological Science. 2009;20(12):1473–1477. doi: 10.1111/j.1467-9280.2009.02470.x.
    1. Walker MP, Brakefield T, Morgan A, Hobson JA, Stickgold R. Practice with sleep makes perfect: sleep-dependent motor skill learning. Neuron. 2002;35(1):205–211. doi: 10.1016/S0896-6273(02)00746-8.
    1. Wang T-Y, Huang H-C, Huang H-S. Design and implementation of cancellation tasks for visual search strategies and visual attention in school children. Computers & Education. 2006;47(1):1–16. doi: 10.1016/j.compedu.2004.08.012.
    1. Wassenberg R, Hendriksen JGM, Hurks PPM, et al. Development of inattention, impulsivity, and processing speed as measured by the d2 Test: results of a large cross-sectional study in children aged 7–13. Child Neuropsychology. 2008;14(3):195–210. doi: 10.1080/09297040601187940.
    1. Weintraub, S. and Mesulam M.-M. 1985. Mental state assessment of young and elderly adults in behavioral neurology. In: Edited by M.-M. Mesulam, Principles of behavioral neurology. Philadelphia: F. A. Davis, pp. 71–123
    1. Wühr, P., and Ansorge, U. (2020). Do left-handers outperform right-handers in paper-and-pencil tests of attention?. Psychological Research, 84(8), 2262–2272.
    1. Zanto TP, Gazzaley A. Selective attention and inhibitory control in the aging brain. In: Cabeza R, Nyberg L, Park DC, editors. Cognitive neuroscience of aging: Linking cognitive and cerebral aging. Oxford University Press; 2004. pp. 207–234.
    1. Ziino C, Ponsford J. Selective attention deficits and subjective fatigue following traumatic brain injury. Neuropsychology. 2006;20(3):383–390. doi: 10.1037/0894-4105.20.3.383.

Source: PubMed

Sponzoři a spolupracovníci

Zdravotní podmínky

Drogové intervence

3
Předplatit