Common errors in students’ translations using google translate with post editing a case study of translated texts in the advanced translation exam
INTRODUCTION
Statement of research problem & rationale for the study
In Industrial Revolution 4.0, recent scientific and technological breakthroughs have enabled the development and advancement of machine translation (MT), particularly Google Translate This March, Google Translate announced its one billion installs on Google Play Store (Pitman, 2021) Up till now, this MT service has taken a long stride toward providing a translation of higher grammatical and lexical accuracy in 108 languages for hundreds of millions of users Google Translate (GT) is also a potentially effective tool for language learning (Groves & Mundt, 2015) and translator training (Korosec,
2020) Recent research has suggested that GT can be utilized as a time-saving solution or a reference to inspiration and terminology Yet, it might use improper or incomprehensible structures to the target language, resulting in comparatively restricted output in terms of translation quality (Rossi & Chevrot, 2019)
Nevertheless, MT output has been considerably enhanced in recent years, thereby spurring an inevitable trend for Machine Translation Post-Editing (MTPE), which produces a hybrid translation of the machine translation engines’ speed and the trained translators’ skills It has been reported that MT not only increases the translators’ productivity (Guerberof, 2009; Plitt & Masselot, 2010) but also lowers production costs (Turner et al., 2014) According to a survey in
2020 conducted by the European Commission, 78% of language service providers (LSPs) aimed to start or increase the use of MTPE The severity of the COVID-19 pandemic also heightens the price pressure in the translation industry; as a result, MTPE is gaining in popularity for its lower costs and becoming a novel option for translation buyers
The world has changed drastically since the COVID-19 pandemic, and likewise, in Vietnam, there is no immediate prospect of public gatherings shortly Additionally, the nationwide lockdown extensions pose numerous challenges to the educational system, disrupting teaching and learning On account of the fourth COVID-19 wave surge in May 2021, Hanoi enforced a city-wide lockdown to curb community transmission The government implemented social distancing and pandemic containment measures to minimize the adverse impacts on public health; thus, the Ministry of Education and Training directed schools and universities to switch from face-to-face to online learning (Le, 2021) In the University of Languages and International Studies – Vietnam National University (VNU-ULIS), a leading educational institution for translator training, an open-book exam was adopted as an alternative in the Advanced Translation final exam for junior students of Translation and Interpreting major at the Faculty of English Language Teacher Education (FELTE) A quick survey among 18E20 members revealed that GT was used by the majority during their translation process The exam results, however, were low despite their attempts at post- editing (PE) Only 35 out of 133 students (26.3%) scored above 8 Hence, it is imperative to investigate common errors in students’ translations that were post- edited using GT in the final exam and propose recommendations for higher quality translation
Previous research on MT, specifically MTPE, is relatively limited, albeit increasing in the past years (De Almeida, 2013) Post-editors have become a separate profession from translators, and LSPs are prone to adopt MTPE to cut costs and enhance productivity Also, there has been scant research on MTPE or common errors of MT post-editor in the English-Vietnamese and Vietnamese- English language pairs; thereby urging more researchers to investigate the case and suggest strategies for translation quality improvement Hence, the study
“Common errors in students’ translations using Google Translate with post- editing: A case study of translated texts in the Advanced Translation Exam” is believed to contribute, more or less, to the knowledge of this field
Research objectives and research questions
The study aims to address the limitations of MTPE in the translator-training context, particularly by analyzing ULIS students’ translations in the Advanced Translation final exam The primary objective is to explore the common translation errors committed by students after post-editing the GT output, thereby suggesting possible solutions to enhance the quality of their translation from the perspective of ULIS lecturers In brief, two research questions to be addressed in the study are as follows:
- What are the common errors in students’ post-editing translation using Google Translate?
- What are the recommendations for post-editing Google Translate output, as perceived by translation trainers?
Significance of the study
Although it was conducted on a small scale, the study was expected to offer valuable insights into the most frequent errors in MTPE, particularly students’ translations using GT with PE, of students at FELTE, ULIS, VNU Hence, students would be fully aware of which errors to be avoided Recommendations of the professionals were also included with a view to facilitating improvement in students’ translation quality Moreover, this paper would be of great importance for lecturers to adjust translator training courses by constructing MTPE skills set and integrating it into existing translation activities The results of this research could serve as a reference source for future studies, especially for other researchers interested in MTPE-related topics.
Scope of the study
This research paper investigated students of Translation and Interpreting major at FELTE, ULIS, VNU regarding their performance in MTPE In particular, the study focused on translated texts of junior students in the Translation and Interpreting training program of Cohort 2018-2022 to detect their most frequent errors of post-editing GT output in the Advanced Translation final exam Due to time constraints and the convenience of error detection, the study was conducted on a group of students from translation-majored classes that had
4 the lowest scores in the Advanced Translation final exam, particularly 40 students’ translations Moreover, after analyzing students’ translations, a questionnaire was sent via email to 13 lecturers of the Translator and Interpreter Training Division, FELTE for their recommendations for students’ translation improvement.
Organization of the study
Chapter 2 summarizes prior studies related to the present research Then, Chapter 3 presents the detailed methodology Then, in Chapter 3, the detailed methodology is presented Following that, Chapter 4 analyzes and discusses the data collected Finally, Chapter 5 discusses the conclusion, which includes limitations of the current research and suggestions for future studies The questionnaire is included in the Appendix
LITERATURE REVIEW
Post-editing
Since the introduction of MT systems, the translation process has been much less tedious as MT reduces the translator’s workload The machine- translated texts can be used as drafts to produce the final translation The task of
“polishing up” the raw MT output, or post-editing (PE), was defined by Schọfer
(2003) as carefully comparing the source text and the raw output to recognize machine errors and proofreading for accuracy, coherence, and naturalness upon the completion of the target text Somers (2001a, p 138, as cited in De Almeida,
2013) described PE as “tidying up the raw output, correcting mistakes, revising entire, or, in the worst case, retranslating entire sections.” ISO 18587 (2017) also provided a similar definition of PE, stating that PE tasks aim at examining the
MT output’s accuracy and comprehensibility and producing an error-free text Even though PE definitions vary, they all share the utmost aim of PE, which is to correct the MT output for machine errors to ensure the target text’s accuracy and comprehensibility
Due to differences in levels of expected quality, there are different PE levels The classification varies considerably among translators and translation agencies (Vieira & Alonso, 2018) Moreover, each linguist has different names and descriptions concerning the levels of correction
One of the earliest classifications of PE levels was Loffler-Laurian’s (1984) fast and conventional PE The choice of PE levels lies in the purpose of the text,
6 whether the text is for gisting or for publication For gisting, fast PE is employed, which is faster and requires only necessary corrections that might affect target readers’ comprehension For publication, conventional PE is applied as it demands equal quality to human translation, even though it is more time- consuming Other classifications also rely on those two criteria: the time required for the task and the quality of the final product Wagner (1985) reported that the European Commission used rapid and full levels for their PE tasks Meanwhile, there was an extension in Allen’s (2003) PE classification: MT with no PE, rapid
PE, partial or minimal PE, and full PE
In the publication on PE guidelines of the Translation Automation User Society (TAUS) (2016), TAUS proposed two standards of the output’s expected quality, specifically “good enough quality” and “human translation quality.” ISO
18587 (2017) shared similar requirements regarding the PE classification, including light PE, equivalent to “good enough quality,” and full PE, another interpretation of “human translation quality.” The former is defined as “a process of post-editing to obtain a merely comprehensible text,” which aims at delivering the main idea of the text and solely corrects errors to ensure the comprehension and accuracy of the text The latter aims at producing “a product comparable to a product obtained by human translation,” or, more precisely, requires not only comprehension and accuracy but also stylistic adequacy (ISO 18587, 2017, p.10)
By now, light and full PE levels have been the most prevalent among PE classifications (Hu & Cadwell, 2016)
The choice of PE level depends on the expected quality of the translation; hence, full PE is preferred in this study as students are expected to deliver an accurate, comprehensible, and natural translation
The PE guidelines are for post-editors to minimize effort but still achieve high-quality translation Due to the final exam’s requirements of an accurate, comprehensible, and natural translation, only full PE guidelines are relevant to the study In general, full PE guidelines reflect that the intended purpose of the target text is to achieve the quality of human linguists’ translation However,
7 there are no standard PE guidelines available, so each translator or translation company designs their personal or internal PE guidelines that suit their purposes
As one of the most prestigious sources on MTPE, TAUS (2016) suggested
PE guidelines of “human translation quality” for the users of MTPE to ensure higher quality translation These guidelines were referred to as the basic guidelines for PE that should be tailored for specific purposes The “human translation quality” guidelines advised fully exploiting the raw MT output but ensuring the accuracy and comprehensibility of the target text; nevertheless, stylistic, syntactic, grammatical, or formatting issues should also be addressed Furthermore, cultural appropriateness was mentioned as one of the requirements, specifically modifying “any offensive, inappropriate, or culturally unacceptable content” (TAUS, 2016, p.18) The TAUS PE guidelines (2016) are not system- or language-specific; therefore, they can be used for MT texts in various MT systems and language pairs
O’Brien (2010) delivered a speech on PE introduction at the AMTA conference Her full PE guidelines closely coincided with the TAUS 2016 guidelines, but she omitted the stylistic guideline, added tagging in the formatting guideline, and added two others, specifically high throughput expectations and medium quality expectations
Rico and Ariano (2014) adopted and tailored a set of language-independent guidelines from Torrejón and Rico’s (2002) The guidelines paralleled the TAUS
2016 guidelines; however, they were written in a clearer language with specific instructions provided Rico and Ariano concurred with O’Brien regarding the exclusion of stylistic issues, but they provided a further explanation that stylistic issues should be unmodified if they do not hinder the understanding
Flanagan and Christensen (2014) provided translation trainees with the TAUS PE guidelines (2010) and asked for their interpretation of the guidelines for publishable quality Based on the trainees’ understanding, they tailored the TAUS guidelines and designed their own set of PE guidelines to use in their PE training course The TAUS guideline 5 (use as much of the raw MT output as possible) was purposely moved to the top, and the word “but” was added to
8 emphasize the importance of following this guideline while complying with the other guidelines Other guidelines were modified to avoid overlapping information, rewritten, or slightly edited for better clarity as translation trainees tend to strictly adhere to the guidelines
Google Translate
Google Translate, introduced by Google in April 2006, is a service that provides a quick, automated translation of written texts from one language to another Since its release, GT is expected to help users overcome translation challenges and break the existing language barriers between people of two different nationalities GT users can translate not only single words, phrases, and long written texts, but also web pages This MT system was first a statistical machine translation model, a more effective system than other previous rule- based systems At that time, Google Translate could only provide automated translations via a pivot language, English The program collected linguistic data from United Nations and European Parliament documents for its text corpora Accordingly, it could search for the most appropriate translation option among previously translated texts by human linguists
In September 2008, Google Translate added Vietnamese to its supported languages, extending the list to 35 languages In November 2016, Google
Translate switched to a neural machine translation model, which offered a more accurate translation in a shorter time Thanks to developments in machine learning, there have been numerous improvements in the quality of Google Translate output (Caswell & Liang, 2020) By then, it had supported 103 languages, attracted over 500 million users, and translated more than 100 billion words daily (Turovsky, 2016) Working relentlessly in the next four years, the
GT team successfully added five more languages, expanding the list of Google Translate’s supported languages to 108 languages (Caswell, 2020)
Google Translate also launched its mobile application for Android in January 2010 and iOS in February 2011, which turned mobile phones into portable translation devices for personal use (Sommerlad, 2021) Throughout years of improvement, the mobile application is now capable of translating not only written texts but also images, handwriting, or speech In March 2021, the application hit one billion installs on Google Play Store, making GT the most popular MT service in the world (Pitman, 2021)
Following the team motto, “Enable everyone, everywhere to understand the world and express themselves across languages,” GT output has been improved through advancements in machine learning technology and active contribution from the Google Translate Community, as well as users worldwide (Caswell,
2020) However, this MT system still falls prey to typical MT errors, which will be thoroughly discussed in 2.3.2 MT errors.
Translation errors
The classification of translation errors has long been a research field that engaged various scholars and researchers to invest their time and effort However, it remains a challenge to construct a holistic taxonomy of all the translation errors The translation errors are discussed in three following aspects: translation errors committed by human linguists, by MT systems, and by machine-translated post- editors
Pym (1992) classified translation errors into two categories: binary and non-binary errors A binary error means an error of incorrect translation On the contrary, a non-binary error refers to a possible incorrect translation, or more precisely, out of the two possible correct options, the less appropriate one is chosen According to Pym (1992), binarism concerns solely correct or incorrect translation, while non-binarism concerns a number of choices, involving at least two correct and the other wrong ones However, Pym only focused on lexis and semantics while completely excluding other related fields such as grammar, syntax, and punctuation
American Translators Association (ATA) is a professional association for translation and interpreting, which has fostered the professional development of individual translators and interpreters in over 100 countries ATA Framework for Standardized Error Marking, therefore, is an authoritative framework to assess translation errors Its 2021 version includes three main error categories: target language mechanics errors, meaning transfer errors, and writing ability errors, which could be further divided into 24 translation errors The list of ATA errors includes: 1) Grammar; 2) Syntax; 3) Word form/Part of speech; 4) Spelling/Character for non-alphabetic languages; 5) Capitalization; 6) Diacritical marks/Accents; 7) Punctuation; 8) Addition; 9) Omission; 10) Terminology; 11) False friend; 12) Verb form; 13) Ambiguity; 14) Cohesion; 15) Faithfulness; 16) Literalness; 17) Misunderstanding; 18) Indecision; 19) Unfinished; 20) Usage;
21) Text type; 22) Register; 23) Style; 24) Illegibility Despite ATA’s reputation and popularity, the framework is complicated and time-consuming in detecting translation errors of undergraduate translators
Another noteworthy translation error framework is Error Typology Best Practice Guidelines by TAUS (2017), which covers four main error categories: Language, Terminology, Accuracy, and Style, and applies four severity levels:
Minor, Major, Critical, and Neutral Yet, what these two error frameworks of ATA and TAUS share in common is the excellence acknowledgment, which encourages translators to receive feedback for their excellent translations, thereby
“ensuring continued high levels of quality” (TAUS, 2017, p.6) Nevertheless, this typology lacks descriptions of each error category and severity level, which might lead to unnecessary ambiguity during the process of error detection
2.3.1.4 Kafipour and Jahanshahi’s error typology
Kafipour and Jahanshahi (2015) analyzed the translation error types and frequency in Islamic-to-English religious texts based on Liao’s (2010) error typology, which includes three genres of errors, namely rendition errors, language errors, and miscellaneous errors Rendition errors refer to any errors of the inaccurate meaning from the source text, while language errors involve any errors of inappropriate expression in the target text Miscellaneous errors are committed when the translator omits any information from the source text According to Kafipour and Jahanshahi (2015), translation errors occur when the translator fails to adhere to one or more than one of the three following categories: culture, syntax, and semantics Study findings reveal that the most frequently committed errors are language errors, in which the inappropriate register obtains the highest rank among other sub-categories
Pham (2017) adopted Popescu’s (2012) error taxonomy to detect errors committed by Vietnamese translation students in a Vietnamese-English translation test Translation errors account for the most common errors detected in students’ tests Linguistic errors and comprehension errors follow Within the error sub-categories, inaccurate renditions of lexical items, syntactic and collocational errors are the three most frequent errors It was indicative that students failed to effectively perform translation tasks relating to lexical choice, syntactic structures, and English collocations
2.3.1.6 Nguyen and Trieu’s HT errors
Nguyen and Trieu (2015) attempted to record Vietnamese-English translation errors committed by second-year translation-major students using
Wang Baorong’s error typology Translation errors are divided into two main categories, including linguistic errors and translational errors Linguistic errors cover lexical choice, punctuation, the use of articles, prepositions, and singular and plural forms, whereas translational errors involve lengthy and awkward expressions, terminology, and inconsistency The collected data showed that lexical choice and lengthy, awkward expressions were the two most common errors Therefore, enhancement in grammar, vocabulary, and background knowledge is recommended for second-year students to improve their translation quality
Wongranu (2017) adopted Pojprasat’s translation error model, including three main categories: semantic, syntactic and miscellaneous errors The first category deals with wrong word choice at the word or phrase levels The second category covers 20 sub-categories, in which three most frequently committed sub-errors were countability, determiners, and tense The last category comprises misspelling, under-translation, and unnatural translation, which are errors that could not be classified into neither syntactic nor semantic errors Wongranu suggested four potential reasons of errors, namely translation procedures, carelessness, low self-esteem, and anxiety
Machine translation (referred to as Google Translate in this research) aims to assist humans in communication across different languages without much time and effort spent on translation tasks MT systems are believed to provide the same level of quality as human trans, yet a myriad of studies have debunked those expectations Despite substantial investments in MT research and high expectations of the fully-automated translation, the output is of low quality, full of errors and imprecisions, thereby requiring a post-editor (De Almeida, 2013)
Besides HT errors, researchers are now interested in the identification and classification of MT errors ISO 18587 listed stylistic errors, literal renderings,
13 grammar mistakes, and translation of names that should not be translated in the common MT error list (2017) However, the list was not exhaustive, thereby not suitable for the error analysis
Zaretskaya et al (2016) used the adapted Multidimensional Quality Metric (MQM) Taxonomy to annotate MT errors There are two main error categories: accuracy and fluency The former includes mistranslation (terminology, unit conversion, overly literal, number, false friend, entity, and should not have been translated), locale convention, inconsistency, omission, addition, and untranslated The latter comprises style/register, unidiomatic, spelling (capitalization), typology (punctuation), grammar (word form (part of speech and agreement), word order, and function words (missing function word and erroneous function word), and unintelligible Out of 23 error types found in the
MT output, mistranslation, word form, and function words were the three most significant errors Nevertheless, the adapted MQM Taxonomy is too intricate to be applied in the error analysis of students’ translations
Vilar et al.’s error typology
Vilar et al (2006) proposed a classification of MT errors, which included
“missing words,” “word order,” “incorrect words,” “unknown words,” and
“punctuation” errors Missing word errors include content words and filler words, while incorrect word errors cover sense, precisely wrong lexical choice and incorrect disambiguation, incorrect form, extra words, style, and idioms Word order errors comprise errors on the word level and phrase level Unknown word errors are errors of unknown stem or unseen forms The last class, punctuation errors, is caused due to the lack of fixed punctuation rules in the MT system
Schọfer (2003) investigates translation errors in four different MT systems to establish a systematic error typology The framework is claimed to be independent in terms of language pairs, MT systems, and text types Four main classes are as follows: lexical errors, syntactic errors, grammatical mistakes, and errors due to defective input text Lexical errors are errors relating to general
Summary
This chapter reviewed the previous literature that is relevant to post-editing, Google Translate, and translation errors The chapter covered PE definitions, classifications, and guidelines, along with a brief history of GT, and different categorizations of translation errors, including HT errors, MT errors, and MTPE errors
In the next chapter, the researcher describes the research methodology, including the research design, subjects of the study, sampling, data collection, and data analysis
METHODOLOGY
Research design
This study employed a qualitative approach to study translated texts of a group of translation-majored students who yielded the lowest results in the Advanced Translation final exam Its main purpose was to point out common errors when students post edited the Google Translate output According to Nisbet & Watt, a case study is “a specific instance that is frequently designed to illustrate a more general principle” (1984, p.72) This research design, therefore, was adopted not to represent the whole population (translated texts of junior translation-majored students), but to represent the case (translated texts of a group of junior translation-majored students with the lowest scores) The study was intended to portray, analyze and interpret the most frequent MTPE errors junior translation-majored students committed in the final exam of the Advanced Translation course, thereby providing suggestions for improvement.
Subjects of the study
The subjects of this study are the junior students’ written translations in the final exam of Advanced Translation, one of the compulsory courses for the Translation and Interpreting major at FELTE, ULIS, VNU The population size is 133 translations, equivalent to the total number of students taking the final exam of this course
The fourth COVID-19 outbreak in May 2021 forced Vietnam education to transit from physical to virtual classrooms At that time, ULIS had to switch to online final examination; therefore, students sat the exams from home instead of gathering at school Advanced Translation was one of the 6th-semester courses to adopt online assessment An alternative assignment for the final exam was sent to all the students via email The students were required to submit their translations of four excerpts of news covering four topics: tourism, environment, health, and
18 education within 24 hours The assignment allowed reference materials, such as dictionaries, news reports, or MT
A quick survey with class 18E20 revealed that 90.9% of the students used
GT output as a draft translation, and all of them post-edited the translation to correct translation errors Out of 20 respondents who claimed to use GT, 13 stated that GT reduced half of their workload while 6 needed to retranslate almost entire texts by themselves Only 1 of them claimed to make slight adjustments to the GT output The exam results, however, were not desirable Only 35 out of 133 students (26.3%) scored above 8 Scores in the range 7.6-8.0 and 6.5-7.5 accounted for 34.6% and 33.1%, respectively Many errors were detected in students’ translations despite their attempts to post-edit the GT output Hence, the written translations in the Advanced Translation final exam would be of great use to analyze common MTPE errors committed by students
In addition, it was vital to carry out a questionnaire for lecturers of the Translator and Interpreter Training Division, FELTE To offer possible solutions to those common errors of students’ translation requires the expertise and experience of the lecturers, who have been professionals in translator training for years.
Sampling
Purposive sampling was deployed in this research paper Due to time constraints, the researcher only investigated a group of students with the lowest scores in the Advanced Translation final exam in the 6th semester It was expected that a larger number of errors would be detected in lower-score translations than in higher-score ones On this account, 30% of the total translations, or 40 students with the lowest scores, were selected to uncover the most frequent errors in their MTPE translations
Regarding the selection of lecturers in the Translator and Interpreter Training Division, FELTE, convenience sampling was adopted Lecturers were chosen for the questionnaire based on their availability and willingness to participate For a total of 13 lecturers in the division, it was desirable to receive at least 4 responses (30% of the population)
Data collection
The data collection instruments were selected on the grounds of two criteria: effectiveness and cost-efficiency Therefore, two types of data collection instruments were employed, namely document observation and questionnaire
To answer the first research question: “What are the common errors in students’ post-editing translation using Google Translate?”, document observation was adopted to carefully record the most prominent errors in students’ translations of the final assignment Unlike questionnaires and interviews, observation does not rely on individual self-reports, which might lower the validity of the information and reliability of the study It involves observing the reality based on a systematic review of related research Therefore, written documents, namely the Advanced Translation final exam, students’ translations, and lecturer’s checks, were closely investigated, and the researcher could gain a deeper understanding of the target participants
The researcher adopted Schọfer’s (2003) adapted error typology, including four main error categories: lexical errors, syntactic errors, grammatical errors, and format errors To minimize potential confusion and mistakes in error detection and categorization, the researcher determined that syntactic errors cover errors at the sentence level, whereas grammatical errors cover errors at the word or phrase level The fourth category in the original framework, errors due to defective input text, was disregarded as four source texts were carefully selected to be included in the final exam Meanwhile, format errors, a new error category, were added due to a number of format errors identified in students’ translations
For the second question: “What are the recommendations for post-editing Google Translate output, as perceived by translation trainers?”, the researcher conducted a questionnaire for 13 lecturers in the Translator and Interpreter Training Division, FELTE Regarding the severity of the COVID-19 pandemic, a web-based questionnaire, particularly Google Forms, was conducted to ensure
20 full compliance with the pandemic precautionary measures Apart from its high accessibility and cost-efficiency, Google Forms facilitates distribution on various platforms via just a mouse click It also allows customization for a more participant-friendly questionnaire
The questionnaire items were divided into three main sections: PE guidelines, students’ translation errors in the Advanced Translation final exam, and the introduction of MTPE into the current translator training academic program
In the first section, there were 4-point Likert scale items regarding the adapted Flanagan & Christensen’s (2014) full PE guidelines, which were originally based on the TAUS (2010) guidelines for publishable quality According to three translation quality assessment criteria of the Advanced Translation final exam, namely accuracy, comprehensibility, and naturalness, it was required that students’ translation reach the TAUS (2016) “human translation quality” Additionally, it was advisable to avoid word-for-word translation and edit the source text or modify the sentence structures for a natural translation of the target text Based on those requirements, Guideline 1, “Use as much of the raw MT output as possible, but [ensure the following guidelines],” was excluded to avoid possible confusion The researcher omitted the second clause of Guideline 4, “but do not restructure sentences solely to improve the flow of the target text,” and Guideline 7, “Ensure any untranslated terms belong to the client’s list of ‘Do Not Translate’ terms, if available,” because they were inapplicable in the final exam Since the students did not use any computer- assisted tool in the translation process, Guideline 10, “Ensure the same ST tags are present and in the correct positions in the target text,” was also disregarded
Table 1 Flanagan & Christensen’s (2014) adapted guidelines for full PE
Adapted Guidelines for Full PE
G1 Ensure the target reader perfectly understands the content of the target text
G2 Ensure the target text communicates the same meaning and message as the source text
G3 Ensure target-text language is appropriate
G4 Aim for a grammatically, syntactically and semantically correct target text
G5 Ensure key terminology is correctly translated
G6 Edit any offensive, inappropriate or culturally unacceptable content for the target reader
G7 Apply basic rules regarding spelling and punctuation
In the second section, the lecturers were asked to give reasons why students committed lexical, syntactic, and format errors After revision, the researcher changed the original open-ended questions into closed-ended questions with four options and an “Other” option The first four options were poor language competence (Nguyen & Trieu, 2015), poor search skills (Nguyen & Trieu, 2015), inappropriate translation procedure (Wongranu, 2017), and inability to control test anxiety (Wongranu, 2017) The “Other” option was for lecturers to add if they thought there were more reasons
In the last section, the lecturers were asked if students should study MTPE in the training program and why (not) If the lecturers agreed that MTPE should be taught, they would be asked three following optional questions about the time of MTPE introduction and reasons, as well as other suggestions for improvement in students’ translation quality using GT with post-editing
After piloting and revising the questionnaire twice with the supervisor, the researcher finalized the questionnaire, including:
• Title: Khảo sát về cách cải thiện chất lượng bản dịch của sinh viên khi hiệu đính bản dịch của Google Translate (English title: A survey on methods to improve students’ translation quality when post- editing Google Translate’s output)
• 19 questionnaire items: 7 Likert scale, 6 closed-ended and 6 open- ended questions
The questionnaire items were split into three main sections for better readability The expected time to fill in the form ranged from 5 to 7 minutes The questionnaire was written in Vietnamese, the first language of participants, in order to encourage engagement in answering open-ended items It was more likely that the participants submitted more detailed open-ended answers when they used their mother tongue The questionnaire clearly stated the title and summarized the rationale and the significance of the study in the instructions to provide the participants with basic background knowledge of the research topic The questionnaire was anonymous to strictly maintain respondents’ confidentiality, thereby encouraging honest responses The questionnaires acted as interviews via email because, in the current situation, the researcher was unable to conduct face-to-face interviews with the lecturers
Step 1: Constructing the error typology
The researcher selected ten students’ translations for error analysis After collecting the results, the researcher chose an error typology that covered the main error categories discovered in the translations The error typology was then adapted in accordance with the characteristics of the errors found
Step 2: Conducting the error typology
In this stage, the lecturer applied the adapted error typology to detect translation errors in the chosen translations and categorize them
First, the researcher constructed a set of questionnaire items based on the conceptual framework After that, a pilot study was carried out to address any possible issues related to phrasing and the relevance of question items The researcher revised and finalized the questionnaire accordingly to avoid possible misunderstandings or irrelevant items, thereby producing the final version
In this stage, the researcher generated the questionnaire on Google Forms for easier distribution Then, the researcher asked the supervisor for the email addresses of lecturers in the Translator and Interpreter Training Division, FELTE
In the emails, the researcher clearly stated the title, the rationale, and the significance of the study, and then asked for lecturers’ availability and willingness to participate One week was allotted for completing the form
A friendly reminder to submit responses was sent to all the lecturers one day before the submission deadline The lecturer finished gathering responses on the following day.
Data analysis
After data collection, the researcher sorted the errors detected in the chosen translations by category and direction Then the researcher calculated the sum
24 and the percentage of each error category in Excel spreadsheets Prominent examples were picked out and organized in a table to represent the most common sub-errors of each main error category
After gathering the responses, the researcher exported data from Google Forms into Google Sheets for further analysis Because all the responses were written in Vietnamese, the researcher went through an additional step— translating the data into English The responses were then analyzed using content analysis to answer three questions, as perceived by translation trainers:
● What are the full PE guidelines for translation students?
● What are the reasons for students’ translation errors in the Advanced Translation final exam?
● Should MTPE be introduced into the current translator’s training program?
Summary
This chapter started by describing the research design in detail, including the approach, the subjects of the study, and the sampling size and methods adopted This was followed by a section on data collection, in which the instruments employed to collect and categorize the data were described, as well as the procedures for the entire data collection process In the final section, the procedures for data analysis were discussed step-by-step
The next chapter analyzes the results of the data collected and discusses two aspects in detail: the common errors in students’ post-editing translation using
GT and recommendations for post-editing GT output, as perceived by translation trainers
FINDINGS AND DISCUSSIONS
The common errors in students’ post-editing translation using Google
Translation errors detected in the study are sorted into four error categories—lexical errors, syntactic errors, grammatical errors, and format errors—and prominent sub-errors in Table 2 In Table 3, the frequencies and percentages of each error category were calculated by direction
Table 2 Error categories and prominent sub-errors
No Error category Prominent sub-errors
Table 3 Frequencies and percentages of each error category
Lexical errors accounted for the highest frequency (57.4%) in the total English-to-Vietnamese errors and the second-highest (20.6%) in the total Vietnamese-to-English errors (Table 3) Despite the use of reference materials, such as dictionaries, news reports, or MT, a number of students selected the incorrect words and phrases, which could be inferred that they did not fully understand or completely misunderstood the source text Lexical errors could be divided into three prominent sub-errors, wrong meaning, word choice, and idioms/collocations, as described in Table 2 The corresponding examples of each lexical error sub-category were provided in Table 4
Wrong meaning errors occur when students have little or no comprehension of the words/phrases in context in the source language, thereby providing a merely literal translation In the first example (Table 4.a), the student directly translated “perfect anti-yuppie, anti-elitist brew” (một thức uống bình dân hoàn hảo) as “thức uống “chống chỉ định” cho chủ nghĩa Yuppie và tầng lớp tinh anh,” which was a literal translation as it made little sense in Vietnamese The
“anti-yuppie, anti-elitist brew,” when placed in this context, means that you do not need to be a yuppie or an elitist to enjoy this brew, because it is for everyone
Vietnamese-to- English errors Total
Lexical errors 401 (57.4%) 122 (20.6%) 523 (40.6%) Syntactic errors 212 (30.4%) 329 (55.7%) 541 (42.0%) Grammatical errors 0 (0.0%) 95 (16.1%) 95 (7.4%) Format errors 85 (12.2%) 45 (7.6%) 130 (10.1%)
Students’ literal translation might confuse readers of the target language because they might not understand why yuppies and elitists are not permitted to drink bia hoi, as the phrase “chống chỉ định” implies In Example 1 (Table 4.b), students misinterpreted the phrase “chưa biết bao giờ chấm dứt”, thereby providing an inaccurate translation, “will never end.”
Word choice refers to errors involving selecting the wrong meaning of polysemic words The second example (Table 4.a) exemplifies word choice errors The verb “launder” has several meanings; therefore, it is highly likely that students’ word choice is incorrect if they do not fully comprehend the context Most students translated the word “launder” in “launder wild-caught stock” as in the phrase “money laundering” (rửa tiền), thereby producing a translation of inappropriate register, “tẩy trắng/rửa trôi số lượng động vật hoang dã bị săn bắt.” The word “launder” in this context refers to the act of transferring wild stock that has been caught illegally into wildlife farms, which would be better translated as “hợp pháp hóa” (legalize) In Example 2 (Table 4.b), the student mistook “có ý thức” (self-compassion) for “conscious” (nhận thức)
Idioms/Collocations mostly contain errors in translating word-for-word the source language’s idioms/collocations In the third example (Table 4.a), the idiom “a breath of fresh air” was converted as “hơi thở của không khí trong lành,” which is a lengthy and awkward expression in Vietnamese A better approach is to use a parallel idiom in Vietnamese, “một làn gió mới”, for a more natural translation In Example 3 (Table 4.b), the Vietnamese idiom “bụng đói thì đầu gối phải bò” is challenging for translation trainees Some struggled to find an equivalent idiom in English This student failed to do so and literally translated the idiom instead Not only was “hungry belly, the knee must be a cow” an incomprehensible translation but it also significantly distorted the meaning of the original idiom
Table 4 Examples of lexical errors a English-to-Vietnamese lexical errors
Sub-error Original text Student’s translation
The ridiculously cheap price and the fact that it is served out of plastic cups makes this the perfect anti-yuppie, anti- elitist brew , suited to the ideals of a socialist country
Chính giá tiền rẻ một cách bất ngờ và hình thức phục vụ bằng cốc nhựa làm cho bia hơi trở thành thức uống
“chống chỉ định” cho chủ nghĩa Yuppie và tầng lớp tinh anh , phù hợp với lý tưởng của một quốc gia xã hội chủ nghĩa
Bia hơi được phục vụ bằng cốc nhựa, bán với giá rẻ bất ngờ Vì vậy bia hơi trở thành một thức uống bình dân hoàn hảo , phù hợp với lý tưởng của một đất nước xã hội chủ nghĩa
Word choice But wildlife farms may increase trade, and launder wild-caught stock
Song, các trang trại động vật hoang dã có thể làm gia tăng hoạt động buôn bán, và tẩy trắng số lượng động vật hoang dã bị săn bắt
Tuy nhiên, các trang trại động vật hoang dã có thể khiến hoạt động buôn bán động vật hoang dã gia tăng và giúp hợp pháp hóa số lượng động vật bị săn bắt trong tự nhiên
Idioms/ The Vietnamese Thủ đô Việt Nam Hà Nội - thủ đô
29 collocations capital is like a breath of fresh air được ví như hơi thở của không khí trong lành của Việt Nam – giống như một làn gió mới b Vietnamese-to-English lexical errors
Sub-error Original text Student’s translation Suggested translation
Cuộc chiến còn đang tiếp diễn và chưa biết bao giờ chấm dứt
The fight is ongoing and will never end
It is unclear when the war against Covid-19 will end
Word choice Covid-19 buộc con người sống chậm hơn và có ý thức hơn với cả chính bản thân
Covid-19 forces people to live more slowly and be more conscious of themselves
Covid-19 has imposed a slow living and self- compassion practice for many people
“ bụng đói thì đầu gối phải bò ”
“hungry belly, the knee must be a cow” with an empty stomach, you’ll work your fingers to the bone
In contrast to lexical errors, in terms of frequency, syntactic errors ranked first (55.7%) in the total Vietnamese-to-English errors while came second (30.4%) in the total English-to-Vietnamese errors (Table 3) The results might imply that a number of students failed to apply syntagmatic structures or adhere
30 to the correct punctuation rules of the target language, as most syntactic errors were the use of lengthy or awkward expressions Two main subcategories of syntactic errors were listed in Table 2, syntagmatic structures and punctuation Examples of each syntactic error sub-category could be found in Table 5
Syntagmatic structures mainly contain errors of active/passive voice and unparalleled structures Most English-to-Vietnamese syntagmatic structure errors involve the misuse of active/passive voice (Examples 1, 2, and 3, Table 5.a) The significant distinction between English and Vietnamese is the use of active or passive voice The English language prefers passive sentences, which is completely the opposite of Vietnamese, a language of active voice The students who failed to realize this key difference might commit this error over and over again in their English-to-Vietnamese translations Meanwhile, in Vietnamese-to- English translations, the majority of syntagmatic structure errors are unparalleled structures (Example 1, Table 5.b) It is also another major difference between the two languages In English, failing to adhere to parallelism is considered an error The students might not be aware of this rule, thereby making parallelism errors Another common error of this sub-category in this direction is run-on sentences (Example 2, Table 5.b) Students sometimes translated long sentences word-for- word but did not pay attention to conjunctions, which resulted in run-on sentence errors
Punctuation errors are detected when students misused punctuation marks in the target language In Example 4 (Table 5.a), such a long sentence without punctuation is likely to confuse Vietnamese readers Similarly, in Example 3 (Table 5.b), the student used a question word order in a statement, which violates punctuation rules
Table 5 Examples of syntactic errors a English-to-Vietnamese syntactic errors
Sub-error Original text Student’s translation Suggested translation
Last-ditch efforts to save the Vietnamese subspecies of Javan
Rhinoceros found just one individual remained
Những nỗ lực cuối cùng để cứu loài tê giác Javan Việt Nam thì chỉ tìm thấy còn lại duy nhất một cá thể
Trong những nỗ lực cuối cùng nhằm cứu các phân loài Tê giác Java Việt Nam, người ta chỉ tìm thấy duy nhất một cá thể còn sót lại
Real emotion pours out of the thousands who come to view his body each day […]
Niềm xúc động thực sự trào dâng trong hàng ngàn người đến ngắm nhìn cơ thể Bác mỗi ngày […]
Mỗi ngày có hàng ngàn người xúc động nghẹn ngào đến viếng thăm thi hài của Bác […]
[ ] vendors who can frequently be spotted underneath conical hats , triggering the photographic instinct in tourists
[ ] những người bán hàng rong, những người mà thường có thể được tìm thấy bên dưới những chiếc nón lá , kích thích bản năng chụp ảnh của
Recommendations for post-editing Google Translate output, as perceived
In the survey, the lecturers expressed their opinion on three issues: full PE guidelines for translation students, reasons for students’ translation errors in the Advanced Translation final exam, and the introduction of MTPE into the current translator training academic program
4.2.1 Full PE guidelines for translation students
The lecturers’ responses for the first issue indicated that they all agreed or totally agreed with all of the adapted guidelines for full PE, as illustrated in Table
8 It could be inferred that Comprehensibility, Clarity, Register, Grammar, Syntax, Terminology, Cultural Appropriateness, and Formatting criteria could be composed as a set of criteria to evaluate MTPE tasks When using GT with post- editing, students should also be aware of those criteria to avoid corresponding errors
Table 8 Lecturers’ perceptions of Flanagan & Christensen’s (2014) adapted guidelines for full PE
Adapted Guidelines for Full PE Agree Totally agree
G1 Ensure the target reader perfectly understands the content of the target text
G2 Ensure the target text communicates the same meaning and message as the source text
G3 Ensure target-text language is appropriate 1 7
G4 Aim for a grammatically, syntactically and semantically correct target text
G5 Ensure key terminology is correctly translated
G6 Edit any offensive, inappropriate or culturally unacceptable content for the target reader
G7 Apply basic rules regarding spelling and punctuation
4.2.2 Reasons for students’ translation errors in the Advanced Translation final exam
The lecturers’ opinions varied when it came to reasons for students’ translation errors in the Advanced Translation final exam Besides four suggested reasons, including poor language competence, poor search skills, inappropriate translation strategy, and inability to control test anxiety, they added several other reasons The perceived reasons were sorted into three main factors: (1) linguistic factors, (2) translation skills factors, and (3) personal factors, as illustrated in Table 9
Regarding lexical errors, results indicated that lack of linguistic competence and translation skills were highly perceived as two major causes All lecturers agreed that poor language competence (1.1) was the main culprit for this error category, followed by poor search skills (2.1) (5 responses) and inappropriate translation procedure (2.3) (4 responses) One lecturer added that mainly due to their poor background knowledge of the topics in the translation exam, students might “not have fully understood the source text’s content” (1.3), thereby committing lexical errors
Meanwhile, all three factors were mentioned when the lecturers explained the cause of making syntactic errors The lecturers reached a consensus on the most prominent cause of committing syntactic errors, which is poor language competence (1.1) Another highly agreed-upon reason for syntactic errors is students’ inappropriate translation procedure (2.3) (4 responses) One lecturer voiced his/her opinion that students might have been “too dependent on MT output and let it dominate their role” (1.4) Two others also included “limited
40 understanding of linguistics and linguistic conventions” (1.5) and “not carefully checking the final translation product” (3.2) as reasons for committing this error category
Similarly, the lecturers brought up all three factors as reasons for committing format errors The two most common reasons were poor language competence (1.1) (4 responses) and lack of technology skills (2.2) (4 responses) Some lecturers further elaborated on students’ lack of technology skills as “not having reviewed the final product carefully before submission” and “not having enabled the format checker tools on their computers.” Two others added carelessness (3.2) as one of the reasons while another said that “because students failed to manage time effectively, they lack time to review and correct those [format] errors” (3.3)
Table 9 Lecturers’ perceptions of reasons for students’ translation errors in the Advanced Translation final exam
1.3 Misunderstanding of the source text
3.1 Inability to control test anxiety
4.2.3 Introduction of MTPE into the current translator training program
As MTPE is currently an emerging trend in the translation market, a question arises, “Should students be taught MTPE in the current training program in universities?” Asked this question, 100% of the lecturers responded “Yes.” The reasons varied Half of the lecturers claimed that MTPE should be taught at school so that students could “meet the demands of the translation market after graduation” as it would be “a competitive advantage in the job market.” Others added that students should learn about “technology’s application in translation,” as well as “MTPE guidelines to increase translation productivity.” Moreover, MTPE was perceived as a useful tool to “handle long documents in a short time, especially in individual translation projects.” Students should be “fully aware of the translation quality, thereby understanding the importance of quality assurance for not only themselves but also other translators.” Yet, one notable point was that “there was no change in the syllabus in recent years”, implying the current syllabus has not been updated to the latest translation trends
With regard to which stage to introduce MTPE in the translation training program, 6 out of 8 lecturers chose “the Advanced Translation or Translation for Specific Purposes course (after passing the fundamental Translation course) of Bachelor’s degree.” The responses were diverse, but most of them agreed that students needed to learn about “the core knowledge of translation theory, e.g procedures, factors that affect the translation quality,” and “fundamental translation skills, e.g source text analysis, search skills, quality assessment based on given criteria.” Therefore, students would be “able to apply appropriate techniques to post-edit the MT output in accordance with the quality requirements.” Without the translation knowledge, skills, and experience acquired in the previous Translation course(s), students might be “bewildered when dealing with MT output,” thereby being dependent on MT, or even worse, compromising with it The lecturers claimed that MTPE was one of the
42 fundamental translation skills, which would assist students in meeting the market demands after their graduation Only one lecturer selected “the fundamental Translation course of Bachelor’s degree” with the belief that “students already used MT before the lecturer introduced it,” so MTPE could be introduced in the second half of the course Also, only one lecturer believed that MTPE should be introduced in “the translator training courses of Master’s degree,” when students already had an excellent grasp of translation theory as well as translation skills
As regards some recommendations for students to improve MTPE skills, reading various materials, especially MT post-edited documents, was the highest- voted solution, with 6 responses Using CAT tools followed with 5 responses Others suggested that students constantly practice to enhance their language and translation competence It was recommended that students actively seek job or internship opportunities at translation agencies, or translate documents on their own using Google Translate to accumulate linguistic and technical knowledge, thereby gradually improving the translation quality.
Summary
This chapter provided a discussion of the data collected in document observation and lecturers’ questionnaires Four most common errors in students’ post-editing translation using Google Translate were analyzed and discussed in detail, followed by some recommendations of lecturers for post-editing Google Translate’s output, particularly on full PE guidelines for translation students, reasons for students’ translation errors in the Advanced Translation final exam, and the introduction of MTPE into the current translator’s training program Two research questions were referenced during the discussion of these aspects
In the next chapter, Conclusion, the researcher discusses the findings in relation to the objectives of the current research, presents the limitations, and proposes suggestions for future studies