Test-retest reliability and validity of a frustration paradigm and irritability measures

Abstract

Background: Data on the reliability and validity of assessments for irritability, particularly behavioral paradigms, are limited. This study examined the test-retest reliability and validity of a frustration paradigm (the Affective Posner 2 task) and two irritability measures [the Affective Reactivity Index (ARI) and Child Behavior Checklist (CBCL) irritability].

Methods: Participants were 109 youth from a general population sample of twins (aged 9-14 years). Participants completed two visits that were 2-4 weeks apart. At both visits, participants completed the Affective Posner 2 task and self-reported their irritability using the ARI. Parents reported their child's irritability using the ARI and completed the CBCL.

Results: The Affective Posner 2 task demonstrated good test-retest reliability, with intraclass correlations (ICCs) ranging from .44 to .78. The task effectively evoked negative affect (frustration and unhappiness) at both test and retest, demonstrating its construct validity. Moreover, self-rated frustration and unhappiness during the frustration components of the task correlated positively with self-reported but not parent-reported irritability, providing modest support for convergent validity. Parent- and child-reports of the ARI and parent-reports of the CBCL irritability measure showed excellent test-retest reliability, with ICCs ranging from .88 to .90.

Limitations: The sample consists of mostly twins aged 9-14 years from the communities. Thus, results may not generalize to non-twin samples or clinical samples outside of this age range.

Conclusions: The Affective Posner 2 paradigm and the ARI and CBCL irritability scales may be useful tools for longitudinal or treatment research on irritability.

Keywords: Frustration; Irritability; Parent-child agreement; Test-retest reliability; Validity.

Conflict of interest statement

Financial Disclosures

The authors disclose no conflicts of interest related to this work.

Copyright © 2017 Elsevier B.V. All rights reserved.

Figures

Figure 1a

Trial Structure during the Frustration Condition (Game 3)

Figure 1b

Task Structure Note. (1a) ITI = inter-trial interval; ISI = inter-stimulus interval. In the frustration condition, 40% of correct responses were followed by positive feedback (“YOU WIN”), and 60% of correct responses were followed by negative feedback (“TOO SLOW”). All incorrect responses were followed by negative feedback (“WRONG”).

Figure 2a

Happy-Unhappy Ratings across Task by Visits

Figure 2b

Frustration Ratings across Task by Visits Note. NF = Non-Frustration; F = Frustration. * p < .05; ** p < .01; *** p < .001. (a) All the pairwise comparisons between runs were significant at p < .05, except for the one between F Run 1 and F Run 2 at Visit 2; (b) All the pairwise comparisons between runs were significant at p < .05, except for the ones between NF Run 1 and NF Run 2 at both Visit 1 and Visit 2.