INTRODUCTION
Due to communication barriers, limited English-proficient (LEP) patients face challenges in the healthcare system. The LEP population accounts for 8.5% of the US population.1 LEP patients experience multiple gaps in care.2 If inclusively designed, technology has the potential to serve as a tool to address these disparities.3
Close to 90% of organizations provide a patient portal.4 As a main portal function, secure messaging extends care beyond the visit. To meet the needs of LEP patients, portals and messaging must be multilingual. This introduces a challenge for language discordant patient-care team relationships. Workflows involving translators place a strain on limited resources. Google Translate (GT) presents an innovative solution. A recent work supports the cautious use of GT for discharge instructions.5 This study evaluates clinician-written text but does not address the practical use of GT to translate patient messages.
Given the language divide, we sought to assess the use of GT in the clinical setting and compare its efficacy to traditional human translation of patient messages.
METHODS
We performed a blinded evaluation of clinical staff, testing their comprehension of translated portal messages. We collected 7 portal messages from patients in Portuguese for this study. The messages had a range of complexity (Flesch-Kincaid grade level ranging 2–8 based on human translated text). We chose Portuguese since it is a prevalent language in our community. We translated the messages to English using a professional translator and GT. We created a series of clinical comprehension questions based on the message content (e.g., “How long has the patient had symptoms?”).
The study was conducted at 2 academic, safety-net institutions in Boston and Cambridge, MA. Both sites use the Epic MyChart (Epic Systems, Verona, WI) portal available in English. Clinicians were emailed an evaluation link and randomized to read either GT or human translated messages. We captured participant experience and language information. Participants answered a total of 25 comprehension questions. Participants rated their confidence in responding to the messages using a 4-point Likert scale by answering the question: “Based on my comprehension, I would feel confident responding to this message.”
For each question, we calculated the percent correct response. We used the chi-squared test to examine the relationship between translation modality and comprehension. This study was approved by the Institutional Review Board at both institutions. All analyses were performed using R software (version 3.4.3).
RESULTS
A total of 179 clinical staff completed the evaluation, ninety randomized to GT translations and 89 to human translations. The participants were primarily physicians (62% in the translator arm, 47% in the GT arm (Table 1)).
Our comprehension testing revealed GT translation was non-inferior to human translation, except for one question (Table 2). For this question, the human translation had a higher proportion of correct responses (97.8% vs. 79.8%, p < 0.01). Participants also reported a significantly higher confidence in responding with the human translation (p = 0.005) for this question.
DISCUSSION
To our knowledge, this is the first study to evaluate clinician comprehension of machine-translated patient portal messages. We demonstrated that clinicians showed similar comprehension and confidence in messages regardless of translation modality. Our study provides support for further exploration of GT use in facilitating secure messaging. Despite the creation of multilingual portals, there are no standards for language-discordant secure messaging. Collaboration between healthcare organizations and machine translation companies offers an opportunity to integrate machine translation as an adjunct to a human translator.
Our study has several limitations. First, it relies on a convenience sample of clinicians who responded to our email. Second, we translated only Portuguese messages, which may affect generalizability to other languages.
Despite concerns about GT, the challenge of communicating with LEP patients represents an unmet need. Clinicians may view GT as an accessible solution. However, the appropriate use of GT translation in the clinical setting has not been established. The Massachusetts Board of Registration in Medicine discourages GT use, revealing a discrepancy between policy and clinician use of GT.6 The use of machine translation introduces a digital form of getting by. Machine translation is not a panacea, but our findings reveal the need to critically evaluate its use.
References
American Community Survey 5-Year Estimates 2013-2017. Selected Social Characteristics in the United States. https://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?pid=ACS_17_5YR_DP02&prodType=table. Accessed 21 June 2019.
Flores G. Language barriers to health care in the United States. N Engl J Med. 2006;355(3):229–231.
López L, Green AR, Tan-McGrory A, King R, Betancourt JR. Bridging the digital divide in health care: the role of health information technology in addressing racial and ethnic disparities. Jt Comm J Qual Patient Saf. 2011;37(10):437–445.
U.S. Government Accountability Office. Health Information Technology: HHS Should Assess the Effectiveness of Its Efforts to Enhance Patient Access to and Use of Electronic Health Information. https://www.gao.gov/products/GAO-17-305. Published March 15, 2017. Accessed June 21, 2019.
Khoong EC, Steinbrook E, Brown C, Fernandez A. Assessing the Use of Google Translate for Spanish and Chinese Translations of Emergency Department Discharge Instructions. JAMA Intern Med. 2019. doi: https://doi.org/10.1001/jamainternmed.2018.7653
Clinical Translation Advisory. https://www.mass.gov/files/documents/2016/07/xa/cta.pdf. Published July 2016. Accessed June 2019.
Acknowledgments
The authors would like to thank Fernando Gargano for his support in reviewing and translating text.
Funding
J.A.R. received support from a career development award from the Office of Diversity, Inclusion and Career Advancement at Beth Israel Deaconess Medical Center. The study was partially funded under grant number R01 HS021495-01A1 from the Agency for Healthcare Research and Quality (AHRQ), US Department of Health and Human Services, support from Harvard Catalyst, the Harvard Clinical and Translational Science Center (National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health Award UL1 TR001102), and financial contributions from Harvard University and its affiliated academic healthcare centers.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
This study was approved by the Institutional Review Board at both institutions.
Conflict of Interest
The authors declare that they do not have a conflict of interest.
Disclaimer
The content is solely the responsibility of the authors and does not necessarily represent the official views of Harvard Catalyst, Harvard University, and its affiliated academic healthcare centers, or the National Institutes of Health.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Prior Presentation
This work was previously presented as a poster at the American Medical Informatics Association Symposium in November 2018.
Rights and permissions
About this article
Cite this article
Rodriguez, J.A., Fossa, A., Mishuris, R. et al. Bridging the Language Gap in Patient Portals: An Evaluation of Google Translate. J GEN INTERN MED 36, 567–569 (2021). https://doi.org/10.1007/s11606-020-05719-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-020-05719-z