The Use of Automated Machine Translation to Translate Figurative Language in a Clinical Setting: Analysis of a Convenience Sample of Patients Drawn From a Randomized Controlled Trial

Hailee Tougas, Steven Chan, Tara Shahrvini, Alvaro Gonzalez, Ruth Chun Reyes, Michelle Burke Parish, Peter Yellowlees, Hailee Tougas, Steven Chan, Tara Shahrvini, Alvaro Gonzalez, Ruth Chun Reyes, Michelle Burke Parish, Peter Yellowlees

Abstract

Background: Patients with limited English proficiency frequently receive substandard health care. Asynchronous telepsychiatry (ATP) has been established as a clinically valid method for psychiatric assessments. The addition of automated speech recognition (ASR) and automated machine translation (AMT) technologies to asynchronous telepsychiatry may be a viable artificial intelligence (AI)-language interpretation option.

Objective: This project measures the frequency and accuracy of the translation of figurative language devices (FLDs) and patient word count per minute, in a subset of psychiatric interviews from a larger trial, as an approximation to patient speech complexity and quantity in clinical encounters that require interpretation.

Methods: A total of 6 patients were selected from the original trial, where they had undergone 2 assessments, once by an English-speaking psychiatrist through a Spanish-speaking human interpreter and once in Spanish by a trained mental health interviewer-researcher with AI interpretation. 3 (50%) of the 6 selected patients were interviewed via videoconferencing because of the COVID-19 pandemic. Interview transcripts were created by automated speech recognition with manual corrections for transcriptional accuracy and assessment for translational accuracy of FLDs.

Results: AI-interpreted interviews were found to have a significant increase in the use of FLDs and patient word count per minute. Both human and AI-interpreted FLDs were frequently translated inaccurately, however FLD translation may be more accurate on videoconferencing.

Conclusions: AI interpretation is currently not sufficiently accurate for use in clinical settings. However, this study suggests that alternatives to human interpretation are needed to circumvent modifications to patients' speech. While AI interpretation technologies are being further developed, using videoconferencing for human interpreting may be more accurate than in-person interpreting.

Trial registration: ClinicalTrials.gov NCT03538860; https://ichgcp.net/clinical-trials-registry/NCT03538860.

Keywords: AI; AI interpretation; AMT; ASR; ATP; FLD; LEP; artificial intelligence; assessment; asynchronous telepsychiatry; automated; automated machine translation; automated speech recognition; automated translation; figurative language device; language barriers; language concordant; language discordant; limited English proficiency; psychiatry; speech recognition; telepsychiatry; translation.

Conflict of interest statement

Conflicts of Interest: PY receives book royalties from the American Psychiatric Association. In 2022, SC has performed contract consulting for University of California, Davis, and owns <1% of stock in Orbit Health Telepsychiatry and Doximity. From 2017-2022, SC has taught and is financially compensated by North American Center for Continuing Medical Education, LLC.

©Hailee Tougas, Steven Chan, Tara Shahrvini, Alvaro Gonzalez, Ruth Chun Reyes, Michelle Burke Parish, Peter Yellowlees. Originally published in JMIR Mental Health (https://mental.jmir.org), 06.09.2022.

Figures

Figure 1
Figure 1
Frequency of figurative language devices, patient word count per minute, and percentage of accurate translation per method and patient. FLDs: figurative language devices.

References

    1. Batalova J, Zong J. Language diversity and English proficiency in the United States. Migration Policy Institute. 2016. [2022-08-19]. .
    1. Kim G, Aguado Loi CX, Chiriboga DA, Jang Y, Parmelee P, Allen RS. Limited English proficiency as a barrier to mental health service use: a study of Latino and Asian immigrants with psychiatric disorders. J Psychiatr Res. 2011 Jan;45(1):104–110. doi: 10.1016/j.jpsychires.2010.04.031.S0022-3956(10)00141-X
    1. Limited English Proficiency (LEP) . [2021-01-15]. .
    1. Section 1557: Ensuring meaningful access for individuals with limited English proficiency. . [2021-01-15]. .
    1. Ryan C. Language Use in the United States: 2011. . 2013. Aug, [2021-01-15]. .
    1. Karliner LS, Jacobs EA, Chen AH, Mutha S. Do professional interpreters improve clinical care for patients with limited English proficiency? A systematic review of the literature. Health Serv Res. 2007 Apr;42(2):727–754. doi: 10.1111/j.1475-6773.2006.00629.x. HESR629
    1. Hsieh E. Not just "getting by": factors influencing providers' choice of interpreters. J Gen Intern Med. 2015 Jan 23;30(1):75–82. doi: 10.1007/s11606-014-3066-8.
    1. Kirchhoff K, Turner AM, Axelrod A, Saavedra F. Application of statistical machine translation to public health information: a feasibility study. J Am Med Inform Assoc. 2011 Jul 01;18(4):473–478. doi: 10.1136/amiajnl-2011-000176. amiajnl-2011-000176
    1. Soto X, Perez-de-Viñaspre O, Labaka G, Oronoz M. Neural machine translation of clinical texts between long distance languages. J Am Med Inform Assoc. 2019 Dec 01;26(12):1478–1487. doi: 10.1093/jamia/ocz110. 5537182
    1. Juckett G, Unger K. Appropriate use of medical interpreters. Am Fam Physician. 2014 Oct 01;90(7):476–480.d11499
    1. Crystal D. A Dictionary of Linguistics and Phonetics. Cambridge, MA, US: Basil Blackwell Ltd; 1991.
    1. Saygin A. Processing figurative language in a multilingual task: Translation, transfer and metaphor. Corpus Linguistics. 2001:1–7.
    1. Yellowlees P, Nakagawa K, Pakyurek M, Hanson A, Elder J, Kales HC. Rapid Conversion of an Outpatient Psychiatric Clinic to a 100% Virtual Telepsychiatry Clinic in Response to COVID-19. Psychiatr Serv. 2020 Jul 01;71(7):749–752. doi: 10.1176/appi.ps.202000230.
    1. University of California, Davis. Agency for Healthcare Research and Quality (AHRQ) Validation of an Automated Online Language Interpreting Tool - Phase Two. . 2017. [2020-11-19]. .
    1. O’Keefe M, White K, Jennings JC. Asynchronous telepsychiatry: A systematic review. J Telemed Telecare. 2019 Jul 29;27(3):137–145. doi: 10.1177/1357633x19867189.
    1. Bauer AM, Alegría M. Impact of patient language proficiency and interpreter service use on the quality of psychiatric care: a systematic review. Psychiatr Serv. 2010 Aug;61(8):765–773. doi: 10.1176/ps.2010.61.8.765. 61/8/765
    1. Green AR, Ngo-Metzger Q, Legedza ATR, Massagli MP, Phillips RS, Iezzoni LI. Interpreter services, language concordance, and health care quality. Experiences of Asian Americans with limited English proficiency. J Gen Intern Med. 2005 Nov;20(11):1050–1056. doi: 10.1111/j.1525-1497.2005.0223.x. JGI223
    1. Chang DF, Hsieh E, Somerville WB, Dimond J, Thomas M, Nicasio A, Boiler M, Lewis-Fernández R. Rethinking Interpreter Functions in Mental Health Services. Psychiatr Serv. 2021 Mar 01;72(3):353–357. doi: 10.1176/appi.ps.202000085.
    1. Ngo-Metzger Q, Sorkin DH, Phillips RS, Greenfield S, Massagli MP, Clarridge B, Kaplan SH. Providing high-quality care for limited English proficient patients: the importance of language concordance and interpreter use. J Gen Intern Med. 2007 Nov 24;22 Suppl 2(S2):324–330. doi: 10.1007/s11606-007-0340-z.
    1. Wang H, Wu H, He Z, Huang L, Ward Church K. Progress in Machine Translation. Engineering. doi: 10.1016/j.eng.2021.03.023. Preprint posted online July 14, 2021.

Source: PubMed

Szponzorok és közreműködők

Egészségi állapot

Kábítószer-beavatkozások

3
Iratkozz fel