Negative Effects of Psychological Treatments: An Exploratory Factor Analysis of the Negative Effects Questionnaire for Monitoring and Reporting Adverse and Unwanted Events

Alexander Rozental, Anders Kottorp, Johanna Boettcher, Gerhard Andersson, Per Carlbring, Alexander Rozental, Anders Kottorp, Johanna Boettcher, Gerhard Andersson, Per Carlbring

Abstract

Research conducted during the last decades has provided increasing evidence for the use of psychological treatments for a number of psychiatric disorders and somatic complaints. However, by focusing only on the positive outcomes, less attention has been given to the potential of negative effects. Despite indications of deterioration and other adverse and unwanted events during treatment, little is known about their occurrence and characteristics. Hence, in order to facilitate research of negative effects, a new instrument for monitoring and reporting their incidence and impact was developed using a consensus among researchers, self-reports by patients, and a literature review: the Negative Effects Questionnaire. Participants were recruited via a smartphone-delivered self-help treatment for social anxiety disorder and through the media (N = 653). An exploratory factor analysis was performed, resulting in a six-factor solution with 32 items, accounting for 57.64% of the variance. The derived factors were: symptoms, quality, dependency, stigma, hopelessness, and failure. Items related to unpleasant memories, stress, and anxiety were experienced by more than one-third of the participants. Further, increased or novel symptoms, as well as lack of quality in the treatment and therapeutic relationship rendered the highest self-reported negative impact. In addition, the findings were discussed in relation to prior research and other similar instruments of adverse and unwanted events, giving credence to the items that are included. The instrument is presently available in eleven different languages and can be freely downloaded and used from www.neqscale.com.

Conflict of interest statement

Competing Interests: The authors have declared that no competing interests exist.

Figures

Fig 1. Parallel analysis of the factor…
Fig 1. Parallel analysis of the factor solution.

References

    1. Hofmann SG, Asnaani A, Vonk IJJ, Sawyer AT, Fang A. The efficacy of Cognitive Behavioral Therapy: A review of meta-analyses. Cognitive Therapy and Research. 2012;36(5):427–40. 10.1007/s10608-012-9476-1
    1. Cuijpers P, Berking M, Andersson G, Quigley L, Kleiboer A, Dobson KS. A Meta-Analysis of Cognitive-Behavioural Therapy for Adult Depression, Alone and in Comparison With Other Treatments. Can J Psychiat. 2013;58(7):376–85. .
    1. Tolin DF. Is cognitive–behavioral therapy more effective than other therapies?: A meta-analytic review. Clin Psychol Rev. 2010;30(6):710–20. 10.1016/j.cpr.2010.05.003
    1. McHugh RK, Barlow DH. The dissemination and implementation of evidence-based psychological treatments: A review of current efforts. Am Psychol. 2010;65(2):73–84. Epub 2010/02/10. 10.1037/a0018121 .
    1. Shafran R, Clark DM, Fairburn CG, Arntz A, Barlow DH, Ehlers A, et al. Mind the gap: Improving the dissemination of CBT. Behav Res Ther. 2009;47(11):902–9. 10.1016/j.brat.2009.07.003 .
    1. Clark DM. Implementing NICE guidelines for the psychological treatment of depression and anxiety disorders: The IAPT experience. Int Rev Psychiatr. 2011;23(4):318–27. 10.3109/09540261.2011.606803 .
    1. Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: A systematic review and meta-analysis. World Psychiatry. 2014;13(3):288–95. 10.1002/wps.20151
    1. Cuijpers P, Donker T, van Straten A, Li J, Andersson G. Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med. 2010;40(12):1943–57. 10.1017/S0033291710000772 .
    1. Ly KH, Truschel A, Jarl L, Magnusson S, Windahl T, Johansson R, et al. Behavioural activation versus mindfulness-based guided self-help treatment administered through a smartphone application: a randomised controlled trial. BMJ Open. 2014;4(1):e003440–e. 10.1136/bmjopen-2013-003440
    1. Parry GD, Crawford MJ, Duggan C. Iatrogenic harm from psychological therapies—Time to move on. The British Journal of Psychiatry. 2016;208:210–2. 10.1192/bjp.bp.115.163618
    1. Lilienfeld SO. Psychological treatments that cause harm. Perspect Psychol Sci. 2007;2(1):53–70. 10.1111/J.1745-6916.2007.00029.X .
    1. Castonguay LG, Boswell JF, Constantino MJ, Goldfried MR, Hill CE. Training implications of harmful effects of psychological treatments. Am Psychol. 2010;65(1):34–49. 10.1037/A0017330 .
    1. Berk M, Parker G. The elephant on the couch: side-effects of psychotherapy. Aust Nz J Psychiat. 2009;43(9):787–94. 10.1080/00048670903107559 .
    1. Vaughan B, Goldstein MH, Alikakos M, Cohen LJ, Serby MJ. Frequency of reporting of adverse events in randomized controlled trials of psychotherapy vs. psychopharmacotherapy. Comprehensive Psychiatry. 2014;55(4):849–55. Epub 2014/03/19. 10.1016/j.comppsych.2014.01.001
    1. Jonsson U, Alaie I, Parling T, Arnberg FK. Reporting of harms in randomized controlled trials of psychological interventions for mental and behavioral disorders: A review of current practice. Contemp Clin Trials. 2014;38(1):1–8. Epub 2014/03/13. 10.1016/j.cct.2014.02.005 .
    1. Barlow DH. Negative effects from psychological treatments: A perspective. Am Psychol. 2010;65(1):13–20. 10.1037/A0015643 .
    1. Powers E, Witmer HL. An experiment in the prevention of delinquency: the Cambridge Somerville youth study New York: Columbia University Press; 1951. 649 s. p.
    1. Bergin AE. The Empirical Emphasis in Psychotherapy: A Symposium—The Effects of Psychotherapy: Negative Results Revisited. Journal of Counseling Psychology. 1963;10(3):244–50. 10.1037/H0043353 .
    1. May PRA. For better or worse? Psychotherapy and variance change: A critical review of the literature. The Journal of Nervous and Mental Disease. 1971;152(3):184–92. 10.1097/00005053-197103000-00004
    1. Lambert MJ, Whipple JL, Hawkins EJ, Vermeersch DA, Nielsen SL, Smart DW. Is It Time for Clinicians to Routinely Track Patient Outcome? A Meta-Analysis. Clinical Psychology: Science and Practice. 2003;10(3):288–301. 10.1093/clipsy.bpg025
    1. Whipple JL, Lambert MJ, Vermeersch DA, Smart DW, Nielsen SL, Hawkins EJ. Improving the effects of psychotherapy: The use of early identification of treatment failure and problem-solving strategies in routine practice. Journal of Counseling Psychology. 2003;50(1):59–68. 10.1037//0022-0167.50.1.59 .
    1. Bergin AE. The deterioration effect: A reply to Braucht. J Abnorm Psychol. 1970;75(3):300–2. 10.1037/h0029341
    1. Boswell JF, Kraus DR, Miller SD, Lambert MJ. Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions (vol 25, pg 6, 2015). Psychother Res. 2015;25(1):Iii–Iii. 10.1080/10503307.2014.942531 .
    1. Hadley SW, Strupp HH. Contemporary Views of Negative Effects in Psychotherapy—Integrated Account. Arch Gen Psychiat. 1976;33(11):1291–302. 10.1001/archpsyc.1976.01770110019001 .
    1. Mays DT, Franks CM. Negative Outcome in Psychotherapy and What to do About It New York: Springer Publishing Company, Inc.; 1985.
    1. Crown S. Contraindicators and dangers of psychotherapy. The British Journal of Psychiatry. 1983;143:436–41. 10.1192/bjp.143.5.436
    1. Mays DT, Franks CM. Getting worse: Psychotherapy or no treatment—The jury should still be out. Prof Psychol. 1980;11(1):78–92. 10.1037//0735-7028.11.1.78
    1. Dimidjian S, Hollon SD. How would we know if psychotherapy were harmful? Am Psychol. 2010;65(1):21–33. 10.1037/A0017299 .
    1. Strupp HH, Hadley SW. A tripartite model of mental health and therapeutic outcomes. With special reference to negative effects in psychotherapy. The American psychologist. 1977;32(3):187–96. Epub 1977/03/01. 10.1037/0003-066X.32.3.187 .
    1. Linden M, Schermuly-Haupt M-L. Definition, assessment and rate of psychotherapy side effects. World Psychiatry. 2014;13(3):306–9. 10.1002/wps.20153
    1. Peterson AL, Roache JD, Raj J, Young-McCaughan S. The need for expanded monitoring of adverse events in behavioral health clinical trials. Contemp Clin Trials. 2013;34(1):152–4. 10.1016/j.cct.2012.10.009
    1. Rozental A, Andersson G, Boettcher J, Ebert DD, Cuijpers P, Knaevelsrud C, et al. Consensus statement on defining and measuring negative effects of Internet interventions. Internet Interventions. 2014;1(1):12–9. 10.1016/j.invent.2014.02.001
    1. Mohr DC, Beutler LE, Engle D, Shohamsalomon V, Bergan J, Kaszniak AW, et al. Identification of patients at risk for nonresponse and negative outcome in psychotherapy. J Consult Clin Psych. 1990;58(5):622–8. 10.1037/0022-006x.58.5.622 .
    1. Suh CS, Strupp HH, O'Malley SS. The Vanderbilt process measures: The Psychotherapy Process Scale (VPPS) and the Negative Indicators Scale (VNIS) In: Greenberg LS, editor. The psychotherapeutic process: A research handbook. New York, NY, US: Guilford Press; 1986. p. 285–323.
    1. Binder JL, Strupp HH. ''Negative process'': A recurrently discovered and underestimated facet of therapeutic process and outcome in the individual psychotherapy of adults. Clin Psychol-Sci Pr. 1997;4(2):121–39. .
    1. Sachs JS. Negative Factors in Brief Psychotherapy—an Empirical-Assessment. J Consult Clin Psych. 1983;51(4):557–64. 10.1037//0022-006x.51.4.557 .
    1. Crits-Christoph P, Cooper A, Luborsky L. The Accuracy of Therapists Interpretations and the Outcome of Dynamic Psychotherapy. J Consult Clin Psych. 1988;56(4):490–5. 10.1037/0022-006x.56.4.490 .
    1. Eaton TT, Abeles N, Gutfreund JM. negative indicators, therapeutic alliance, and therapy outcome. Psychother Res. 1993;3(2):115–23.
    1. Parker G, Fletcher K, Berk M, Paterson A. Development of a measure quantifying adverse psychotherapeutic ingredients: The Experiences of Therapy Questionnaire (ETQ). Psychiat Res. 2013;206(2–3):293–301. 10.1016/J.Psychres.2012.11.026 .
    1. Parker G, Paterson A, Fletcher K, McClure G, Berk M. Construct validity of the Experiences of Therapy Questionnaire (ETQ). BMC psychiatry. 2014;14 Artn 369 10.1186/S12888-014-0369-6 .
    1. Linden M. How to define, find and classify side effects in psychotherapy: From unwanted events to adverse treatment reactions. Clin Psychol Psychot. 2013;20(4):286–96. 10.1002/Cpp.1765 .
    1. Boettcher J, Rozental A, Andersson G, Carlbring P. Side effects in Internet-based interventions for Social Anxiety Disorder. Internet Interventions. 2014;1(1):3–11. 10.1016/j.invent.2014.02.002
    1. Ladwig I, Rief W, Nestoriuc Y. What are the Risks and Side Effects to Psychotherapy?—Development of an Inventory for the Assessment of Negative Effects of Psychotherapy (INEP). Verhaltenstherapie. 2014;24(4):252–63. 10.1159/000367928 .
    1. Rozental A, Boettcher J, Andersson G, Schmidt B, Carlbring P. Negative effects of Internet interventions: A qualitative content analysis of patients' experiences with treatments delivered online. Cognitive Behaviour Therapy. 2015;44(3):223–36. 10.1080/16506073.2015.1008033
    1. Cronbach LL, Meehl PE. Construct validity in psychological test. Psychol Bull. 1955;52:281–302.
    1. Clark LA, Watson D. Constructing validity: Basic issues in objective scale development. Psychol Assessment. 1995;7(3):309–19. 10.1037//1040-3590.7.3.309 .
    1. Drennan J. Cognitive interviewing: verbal data in the design and pretesting of questionnaires. J Adv Nurs. 2003;42(1):57–63. 10.1046/j.1365-2648.2003.02579.x .
    1. Krosnick JA. Survey research. Annu Rev Psychol. 1999;50(1):537–67. 10.1146/annurev.psych.50.1.537
    1. Hinkin T. A review of scale development practices in the study of organizations. Journal of Management. 1995;21(5):967–88. 10.1016/0149-2063(95)90050-0
    1. Miloff A, Marklund A, Carlbring P. The Challenger app for social anxiety disorder: New advances in mobile psychological treatment. Internet Interventions. 2015;2(4):382–91. 10.1016/j.invent.2015.08.001
    1. Liebowitz MR. Social phobia. Modern Problems of Pharmacopsychiatry. 1987;22:141–73.
    1. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (MINI): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiat. 1998;59:22–33. .
    1. Fabrigar LR, Wegener DT, MacCallum RC, Strahan EJ. Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods. 1999;4(3):272–99. 10.1037//1082-989x.4.3.272 .
    1. Floyd FJ, Widaman KF. Factor analysis in the development and refinement of clinical assessment instruments. Psychol Assessment. 1995;7(3):286–99. 10.1037//1040-3590.7.3.286 .
    1. Loehlin JC. Component Analysis Versus Common Factor-Analysis—a Case of Disputed Authorship. Multivar Behav Res. 1990;25(1):29–31. 10.1207/s15327906mbr2501_2 .
    1. Browne MW. An overview of analytic rotation in exploratory factor analysis. Multivar Behav Res. 2001;36(1):111–50. 10.1207/S15327906mbr3601_05 .
    1. Zwick WR, Velicer WF. Comparison of five rules for determining the number of components to retain. Psychol Bull. 1986;99(3):432–42. 10.1037/0033-2909.99.3.432
    1. Cattell RB. The Scree Test For The Number Of Factors. Multivariate Behav Res. 1966;1(2):245–76. 10.1207/s15327906mbr0102_10 .
    1. Kaiser HF. The Application of Electronic-Computers to Factor-Analysis. Educational and Psychological Measurement. 1960;20(1):141–51. 10.1177/001316446002000116 .
    1. Horn JL. A Rationale and Test for the Number of Factors in Factor-Analysis. Psychometrika. 1965;30(2):179–85. 10.1007/Bf02289447 .
    1. O'Connor BP. SPSS and SAS programs for determining the number of components using parallel analysis and Velicer's MAP test. Behav Res Meth Ins C. 2000;32(3):396–402. 10.3758/Bf03200807 .
    1. Comrey AL. Common methodological problems in factor analyic studies. J Consult Clin Psych. 1978;46:648–59.
    1. Lambert M. Outcome in psychotherapy: The past and important advances. Psychotherapy. 2013;50(1):42–51. 10.1037/a0030682
    1. Moritz S, Fieker M, Hottenrott B, Seeralan T, Cludius B, Kolbeck K, et al. No pain, no gain? Adverse effects of psychotherapy in obsessive-compulsive disorder and its relationship to treatment gains. J Obsess-Compuls Rel. 2015;5:61–6. 10.1016/j.jocrd.2015.02.002 .
    1. Lambert MJ, Barley DE. Research summary on the therapeutic relationship and psychotherapy outcome. Psychotherapy: Theory, Research, Practice, Training. 2001;38(4):357–61.
    1. Saxon D, Barkham M. Patterns of Therapist Variability: Therapist Effects and the Contribution of Patient Severity and Risk. J Consult Clin Psych. 2012;80(4):535–46. 10.1037/a0028898 .
    1. Mohr DC, Burns MN, Schueller SM, Clarke G, Klinkman M. Behavioral Intervention Technologies: Evidence review and recommendations for future research in mental health. Gen Hosp Psychiat. 2013;35(4):332–8. 10.1016/J.Genhosppsych.2013.03.008 .
    1. Leitner A, Märtens M, Koschier A, Gerlich K, Liegl G, Hinterwallner H, et al. Patients’ Perceptions of Risky Developments During Psychotherapy. Journal of Contemporary Psychotherapy. 2013;43(2):95–105.
    1. Nutt DJ, Sharpe M. Uncritical positive regard? Issues in the efficacy and safety of psychotherapy. J Psychopharmacol. 2008;22(1):3–6. 10.1177/0269881107086283 .
    1. Wahl OF. Mental health consumers' experience of stigma. Schizophrenia Bull. 1999;25(3):467–78. .
    1. Link BG, Struening EL, Neese-Todd S, Asmussen S, Phelan JC. Stigma as a barrier to recovery: The consequences of stigma for the self-esteem of people with mental illnesses. Psychiatric Services. 2001;52(12):1621–6.
    1. Corrigan P. How stigma interferes with mental health care. Am Psychol. 2004;59(7):614–25. 10.1037/0003-066x.59.7.614 .
    1. Stewart JW, Mercier MA, Quitkin FM, McGrath PJ, Nunes E, Young J, et al. Demoralization predicts nonresponse to cognitive therapy in depressed outpatients. Journal of Cognitive Psychotherapy. 1993;7(2):105–16.
    1. Beck AT, Brown G, Berchick RJ, Stewart BL, Steer RA. Relationship between hopelessness and ultimate suicide: a replication with psychiatric outpatients. Am J Psychiat. 1990;147(2):190–5.
    1. Beck AT, Erbaugh J, Ward CH, Mock J, Mendelsohn M. An Inventory for Measuring Depression. Arch Gen Psychiat. 1961;4(6):561–&. .
    1. Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol. 2000;55(1):68–78.
    1. Klein DC, Fencii-Morse E, Seligman ME. Learned helplessness, depression, and the attribution of failure. Journal of Personality and Social Psychology. 1976;33(5):508–16.
    1. Abela JRZ, Seligman MEP. The hopelessness theory of depression: A test of the diathesis-stress component in the interpersonal and achievement domains. Cognitive Therapy and Research. 2000;24(4):361–78.
    1. Bystedt S, Rozental A, Andersson G, Boettcher J, Carlbring P. Clinicians' perspectives on negative effects of psychological treatments. Cogn Behav Ther. 2014;43(4):319–31. 10.1080/16506073.2014.939593
    1. Schwarz N. Self-reports: How the questions shape the answers. Am Psychol. 1999;54(2):93–105. 10.1037//0003-066x.54.2.93
    1. Wright B. Solving measurement problems with the Rasch model. Journal of Educational Measurement. 1977;14(2):97–116.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe