Digital Content-Free Speech Analysis Tool to Measure Affective Distress in Mental Health: Evaluation Study

Peter Tonn, Lea Seule, Yoav Degani, Shani Herzinger, Amit Klein, Nina Schulze, Peter Tonn, Lea Seule, Yoav Degani, Shani Herzinger, Amit Klein, Nina Schulze

Abstract

Background: Mood disorders and depression are pervasive and significant problems worldwide. These represent severe health and emotional impairments for individuals and a considerable economic and social burden. Therefore, fast and reliable diagnosis and appropriate treatment are of great importance. Verbal communication can clarify the speaker's mental state-regardless of the content, via speech melody, intonation, and so on. In both everyday life and clinical conditions, a listener with appropriate previous knowledge or a trained specialist can grasp helpful knowledge about the speaker's psychological state. Using automated speech analysis for the assessment and tracking of patients with mental health issues opens up the possibility of remote, automatic, and ongoing evaluation when used with patients' smartphones, as part of the current trends toward the increasing use of digital and mobile health tools.

Objective: The primary aim of this study is to evaluate the measurements of the presence or absence of depressive mood in participants by comparing the analysis of noncontentual speech parameters with the results of the Patient Health Questionnaire-9.

Methods: This proof-of-concept study included participants in different affective phases (with and without depression). The inclusion criteria included a neurological or psychiatric diagnosis made by a specialist and fluent use of the German language. The measuring instrument was the VoiceSense digital voice analysis tool, which enables the analysis of 200 specific speech parameters based on machine learning and the assessment of the findings using Patient Health Questionnaire-9.

Results: A total of 292 psychiatric and voice assessments were performed with 163 participants (males: n=47, 28.8%) aged 15 to 82 years. Of the 163 participants, 87 (53.3%) were not depressed at the time of assessment, and 88 (53.9%) participants had clinically mild to moderate depressive phases. Of the 163 participants, 98 (32.5%) showed subsyndromal symptoms, and 19 (11.7%) participants were severely depressed. In the speech analysis, a clear differentiation between the individual depressive levels, as seen in the Patient Health Questionnaire-9, was also shown, especially the clear differentiation between nondepressed and depressed participants. The study showed a Pearson correlation of 0.41 between clinical assessment and noncontentual speech analysis (P<.001).

Conclusions: The use of speech analysis shows a high level of accuracy, not only in terms of the general recognition of a clinically relevant depressive state in the participants. Instead, there is a high degree of agreement regarding the extent of depressive impairment with the assessment of experienced clinical practitioners. From our point of view, the application of the noncontentual analysis system in everyday clinical practice makes sense, especially with the idea of a quick and unproblematic assessment of the state of mind, which can even be carried out without personal contact.

Trial registration: ClinicalTrials.gov NCT03700008; https://ichgcp.net/clinical-trials-registry/NCT03700008.

Keywords: assessment; depression; diagnosis; distress; evaluation; mHealth; measurement; mental health; mobile health; mobile phone; mood; questionnaire; speech; speech analysis; tool; voice analysis.

Conflict of interest statement

Conflicts of Interest: YD works in a company called VoiceSense. VoiceSense has developed the vocal analysis system used in this study and it also funded this study.

©Peter Tonn, Lea Seule, Yoav Degani, Shani Herzinger, Amit Klein, Nina Schulze. Originally published in JMIR Formative Research (https://formative.jmir.org), 30.08.2022.

Figures

Figure 1
Figure 1
Patient Health Questionnaire-9 (PHQ-9) and vocal depression scores by age group.
Figure 2
Figure 2
Patient Health Questionnaire-9 (PHQ-9) and vocal depression scores by educational level.
Figure 3
Figure 3
Patient Health Questionnaire-9 (PHQ-9) and vocal depression scores by psychological treatment status.

References

    1. Cosgrove L, Mills C, Amsterdam J, Heath I, Mehta A, Kalathil J, Shaughnessy A. Global mental health. Lancet. 2019 Jul 13;394(10193):117–8. doi: 10.1016/S0140-6736(19)30945-6.S0140-6736(19)30945-6
    1. Liu Q, He H, Yang J, Feng X, Zhao F, Lyu J. Changes in the global burden of depression from 1990 to 2017: findings from the Global Burden of Disease study. J Psychiatr Res. 2020 Jul;126:134–40. doi: 10.1016/j.jpsychires.2019.08.002. S0022-3956(19)30738-1
    1. Kessler RC. The costs of depression. Psychiatr Clin North Am. 2012 Mar;35(1):1–14. doi: 10.1016/j.psc.2011.11.005.S0193-953X(11)00113-4
    1. Furukawa TA. Assessment of mood: guides for clinicians. J Psychosom Res. 2010 Jun;68(6):581–9. doi: 10.1016/j.jpsychores.2009.05.003.S0022-3999(09)00182-2
    1. Fekadu A, Wooderson SC, Markopoulo K, Donaldson C, Papadopoulos A, Cleare AJ. What happens to patients with treatment-resistant depression? A systematic review of medium to long term outcome studies. J Affect Disord. 2009 Jul;116(1-2):4–11. doi: 10.1016/j.jad.2008.10.014.S0165-0327(08)00424-2
    1. Hershenberg R, McDonald WM, Crowell A, Riva-Posse P, Craighead WE, Mayberg HS, Dunlop BW. Concordance between clinician-rated and patient reported outcome measures of depressive symptoms in treatment resistant depression. J Affect Disord. 2020 Apr 01;266:22–9. doi: 10.1016/j.jad.2020.01.108.S0165-0327(19)32418-8
    1. Hall M, Scherner PV, Kreidel Y, Rubel JA. A systematic review of momentary assessment designs for mood and anxiety symptoms. Front Psychol. 2021 May 17;12:642044. doi: 10.3389/fpsyg.2021.642044. doi: 10.3389/fpsyg.2021.642044.
    1. Ebner-Priemer UW, Trull TJ. Ecological momentary assessment of mood disorders and mood dysregulation. Psychol Assess. 2009 Dec;21(4):463–75. doi: 10.1037/a0017075.2009-22537-002
    1. Burke LE, Shiffman S, Music E, Styn MA, Kriska A, Smailagic A, Siewiorek D, Ewing LJ, Chasens E, French B, Mancino J, Mendez D, Strollo P, Rathbun SL. Ecological momentary assessment in behavioral research: addressing technological and human participant challenges. J Med Internet Res. 2017 Mar 15;19(3):e77. doi: 10.2196/jmir.7138. v19i3e77
    1. Goessl VC, Curtiss JE, Hofmann SG. The effect of heart rate variability biofeedback training on stress and anxiety: a meta-analysis. Psychol Med. 2017 Nov;47(15):2578–86. doi: 10.1017/S0033291717001003.S0033291717001003
    1. van Ballegooijen W, Ruwaard J, Karyotaki E, Ebert DD, Smit JH, Riper H. Reactivity to smartphone-based ecological momentary assessment of depressive symptoms (MoodMonitor): protocol of a randomised controlled trial. BMC Psychiatry. 2016 Oct 21;16(1):359. doi: 10.1186/s12888-016-1065-5. 10.1186/s12888-016-1065-5
    1. van Genugten CR, Schuurmans J, Lamers F, Riese H, Penninx BW, Schoevers RA, Riper HM, Smit JH. Experienced burden of and adherence to smartphone-based ecological momentary assessment in persons with affective disorders. J Clin Med. 2020 Jan 23;9(2):322. doi: 10.3390/jcm9020322. jcm9020322
    1. Snyder HR. Major depressive disorder is associated with broad impairments on neuropsychological measures of executive function: a meta-analysis and review. Psychol Bull. 2013 Jan;139(1):81–132. doi: 10.1037/a0028727.2012-13786-001
    1. Miller WR. Psychological deficit in depression. Psychol Bull. 1975 Mar;82(2):238–60. doi: 10.1037/h0076367.
    1. Stassen HH, Bomben G, Günther E. Speech characteristics in depression. Psychopathology. 1991;24(2):88–105. doi: 10.1159/000284700.
    1. Hoermann S, McCabe KL, Milne DN, Calvo RA. Application of synchronous text-based dialogue systems in mental health interventions: systematic review. J Med Internet Res. 2017 Jul 21;19(8):e267. doi: 10.2196/jmir.7023. v19i8e267
    1. Mundt JC, Vogel AP, Feltner DE, Lenderking WR. Vocal acoustic biomarkers of depression severity and treatment response. Biol Psychiatry. 2012 Oct 01;72(7):580–7. doi: 10.1016/j.biopsych.2012.03.015.S0006-3223(12)00263-6
    1. Mitsuyoshi S, Nakamura M, Omiya Y, Shinohara S, Hagiwara N, Tokuno S. Mental status assessment of disaster relief personnel by vocal affect display based on voice emotion recognition. Disaster Mil Med. 2017 Apr 8;3:4. doi: 10.1186/s40696-017-0032-0. 32
    1. Harati S, Crowell A, Mayberg H, Nemati S. Depression severity classification from speech emotion. Annu Int Conf IEEE Eng Med Biol Soc. 2018 Jul;2018:5763–6. doi: 10.1109/EMBC.2018.8513610.
    1. McCarthy-Jones S, Fernyhough C. The varieties of inner speech: links between quality of inner speech and psychopathological variables in a sample of young adults. Conscious Cogn. 2011 Dec;20(4):1586–93. doi: 10.1016/j.concog.2011.08.005.S1053-8100(11)00195-4
    1. Luyten P, Blatt SJ. Psychodynamic treatment of depression. Psychiatr Clin North Am. 2012 Mar;35(1):111–29. doi: 10.1016/j.psc.2012.01.001.S0193-953X(12)00002-0
    1. Hofmann SG, Gómez AF. Mindfulness-based interventions for anxiety and depression. Psychiatr Clin North Am. 2017 Dec;40(4):739–49. doi: 10.1016/j.psc.2017.08.008.S0193-953X(17)30076-X
    1. Twohig MP, Levin ME. Acceptance and commitment therapy as a treatment for anxiety and depression: a review. Psychiatr Clin North Am. 2017 Dec;40(4):751–70. doi: 10.1016/j.psc.2017.08.009.S0193-953X(17)30077-1
    1. Miralles I, Granell C, Díaz-Sanahuja L, Van Woensel W, Bretón-López J, Mira A, Castilla D, Casteleyn S. Smartphone apps for the treatment of mental disorders: systematic review. JMIR Mhealth Uhealth. 2020 Apr 02;8(4):e14897. doi: 10.2196/14897. v8i4e14897
    1. Zargaran A, Ash J, Kerry G, Rasasingam D, Gokani S, Mittal A, Zargaran D. Ethics of smartphone usage for medical image sharing. Indian J Surg. 2018 Jun;80(3):300–1. doi: 10.1007/s12262-017-1710-3.1710
    1. Hansen L, Zhang YP, Wolf D, Sechidis K, Ladegaard N, Fusaroli R. A generalizable speech emotion recognition model reveals depression and remission. Acta Psychiatr Scand. 2022 Feb;145(2):186–99. doi: 10.1111/acps.13388.
    1. Petrizzo D, Popolo PS. Smartphone use in clinical voice recording and acoustic analysis: a literature review. J Voice. 2021 May;35(3):499.e23–8. doi: 10.1016/j.jvoice.2019.10.006.S0892-1997(19)30284-X
    1. Flanagan O, Chan A, Roop P, Sundram F. Using acoustic speech patterns from smartphones to investigate mood disorders: scoping review. JMIR Mhealth Uhealth. 2021 Sep 17;9(9):e24352. doi: 10.2196/24352. v9i9e24352
    1. Valeriani D, Simonyan K. The dynamic connectome of speech control. Philos Trans R Soc Lond B Biol Sci. 2021 Oct 25;376(1836):20200256. doi: 10.1098/rstb.2020.0256.
    1. Low DM, Bentley KH, Ghosh SS. Automated assessment of psychiatric disorders using speech: a systematic review. Laryngoscope Investig Otolaryngol. 2020 Feb;5(1):96–116. doi: 10.1002/lio2.354.LIO2354
    1. Chekroud AM, Zotti RJ, Shehzad Z, Gueorguieva R, Johnson MK, Trivedi MH, Cannon TD, Krystal JH, Corlett PR. Cross-trial prediction of treatment outcome in depression: a machine learning approach. Lancet Psychiatry. 2016 Mar;3(3):243–50. doi: 10.1016/S2215-0366(15)00471-X.S2215-0366(15)00471-X
    1. Rathbone AL, Clarry L, Prescott J. Assessing the efficacy of mobile health apps using the basic principles of cognitive behavioral therapy: systematic review. J Med Internet Res. 2017 Nov 28;19(11):e399. doi: 10.2196/jmir.8598. v19i11e399
    1. Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. JAMA. 1999 Nov 10;282(18):1737–44. doi: 10.1001/jama.282.18.1737.joc90770
    1. Ma S, Yang J, Yang B, Kang L, Wang P, Zhang N, Wang W, Zong X, Wang Y, Bai H, Guo Q, Yao L, Fang L, Liu Z. The Patient Health Questionnaire-9 vs. the Hamilton Rating Scale for depression in assessing major depressive disorder. Front Psychiatry. 2021 Nov 4;12:747139. doi: 10.3389/fpsyt.2021.747139. doi: 10.3389/fpsyt.2021.747139.
    1. Rykov Y, Thach TQ, Bojic I, Christopoulos G, Car J. Digital biomarkers for depression screening with wearable devices: cross-sectional study with machine learning modeling. JMIR Mhealth Uhealth. 2021 Oct 25;9(10):e24872. doi: 10.2196/24872. v9i10e24872
    1. Albuquerque L, Valente AR, Teixeira A, Figueiredo D, Sa-Couto P, Oliveira C. Association between acoustic speech features and non-severe levels of anxiety and depression symptoms across lifespan. PLoS One. 2021 Apr 8;16(4):e0248842. doi: 10.1371/journal.pone.0248842. PONE-D-20-20587
    1. Pampouchidou A, Simantiraki O, Vazakopoulou CM, Chatzaki C, Pediaditis M, Maridaki A, Marias K, Simos P, Yang F, Meriaudeau F, Tsiknakis M. Facial geometry and speech analysis for depression detection. Annu Int Conf IEEE Eng Med Biol Soc. 2017 Jul;2017:1433–6. doi: 10.1109/EMBC.2017.8037103.
    1. He L, Cao C. Automated depression analysis using convolutional neural networks from speech. J Biomed Inform. 2018 Jul;83:103–11. doi: 10.1016/j.jbi.2018.05.007. S1532-0464(18)30090-X
    1. Little B, Alshabrawy O, Stow D, Ferrier IN, McNaney R, Jackson DG, Ladha K, Ladha C, Ploetz T, Bacardit J, Olivier P, Gallagher P, O'Brien JT. Deep learning-based automated speech detection as a marker of social functioning in late-life depression. Psychol Med. 2021 Jul;51(9):1441–50. doi: 10.1017/S0033291719003994.S0033291719003994
    1. Chokshi S, Senathirajah Y, Yadav V, Winsberg M, O'Callaghan E, Sullivan S, Verma A, Kachnowski S. A comparative evaluation of measurement-based psychiatric care delivered via specialized telemental health platform versus treatment as usual: a retrospective analysis. Cureus. 2022 Jan;14(1):e21219. doi: 10.7759/cureus.21219.

Source: PubMed

Patrocinadores e Colaboradores

Condições médicas

Intervenções de drogas

3
Se inscrever