Using notepad and voice recorder technology (audionote) in teachers’ feedback to improve the classroom interpreting performance of university students majoring in translation and interpreting at a university in hanoi

The research results showed the students‘ remarkably positive responses to and high evaluation of Audionote in improving the teacher‘ s feedback quality thanks to a variety of advantages

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES

FACULTY OF POST-GRADUATE STUDIES

NGUYỄN THỊ THUỲ TRANG

USING NOTEPAD AND VOICE RECORDER TECHNOLOGY (AUDIONOTE) IN TEACHERS’ FEEDBACK TO IMPROVE THE CLASSROOM INTERPRETING PERFORMANCE OF UNIVERSITY STUDENTS MAJORING IN TRANSLATION AND INTERPRETING AT A UNIVERSITY IN HANOI

(Sử dụng công nghệ ghi âm đồng thời với ghi chú (phần mềm Audionote) trong việc đưa nhận xét của giảng viên để cải thiện kĩ năng dịch nói của sinh viên chuyên ngành Biên phiên dịch tại một trường Đại học ở Hà Nội)

M.A MAJOR PROGRAMME THESIS

Field: English Teaching Methodology Code: 8140231.01

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES

FACULTY OF POST-GRADUATE STUDIES

NGUYỄN THỊ THUỲ TRANG

USING NOTEPAD AND VOICE RECORDER TECHNOLOGY (AUDIONOTE) IN TEACHERS’ FEEDBACK TO IMPROVE THE CLASSROOM INTERPRETING PERFORMANCE OF UNIVERSITY STUDENTS MAJORING IN TRANSLATION AND INTERPRETING AT A UNIVERSITY IN HANOI

(Sử dụng công nghệ ghi âm đồng thời với ghi chú (phần mềm Audionote) trong việc đưa nhận xét của giảng viên để cải thiện kĩ năng dịch nói của sinh viên chuyên ngành Biên phiên dịch tại một trường Đại học ở Hà Nội)

M.A MAJOR PROGRAMME THESIS

Field: English Teaching Methodology Code: 8140231.01

Supervisor: Dr Huỳnh Anh Tuấn

HÀ NỘI - 2020

DECLARATION

I hereby certify that the thesis entitled “Using Notepad and Voice Recorder technology (Audionote) in teachers’ feedback to improve the classroom interpreting performances of university students majoring in translation and interpreting at a university in Hanoi" was carried out and submitted in partial

fulfilment of the Degree of Master of Arts at the Faculty of Post-Graduate Studies, University of Languages and International Studies, Vietnam National University, Hanoi I also declare that this work is original and all the sources used in the paper were comprehensively documented in the Reference list

ACKNOWLEDGEMENTS

For the accomplishment of this research study, I owe my profound indebtedness

to a lot of people who have given me tremendous support and valuable advice throughout the process of conducting my thesis

First of all, I would like to express my sincere gratitude to my supervisor, Dr Huỳnh Anh Tuấn for his enlightening guidance and constant encouragement which were the decisive factors for the fulfilment of this paper

Secondly, my sincere thanks go to Ms Trần Thị Lan Anh, a senior lecturer in the Faculty of English Language Teacher Education, ULIS - VNU Not only did she help

me come up with the initial research ideas and plan, but her enthusiastic and intellectual guidance and consultancy also supported me a lot in the accomplishment

of the whole study

Thirdly, I am deeply grateful to my two colleagues in my department, Ms Vương Thanh Nhàn and Mr Lê Hải Phong who kindly accepted to collaborate with

me in the research procedure Ms Nhàn was the senior interpreter trainer who observed my class and took part in the interview for comments on the use of Audionote feedback Mr Phong collaborated with me in the rating process for the pre-intervention and post- intervention tests which were of pivotal importance to find out the answers to the research questions

Fourthly, I would like to thank all the lecturers at the Faculty of Post-Graduate Studies, ULIS – VNU for their valuable lectures which provide a solid base for my research

I also sincerely thank all the students in my Advanced Interpreting class who have actively taken part in the action research

Last but not least, my heartfelt thanks go to my beloved family members for their tremendous love and support during the implementation of my thesis

1/ the students’ responses towards the use of Notepad and Voice Recorder technology

(Audionote) in teachers’ feedback for students’ simultaneous interpreting performance, 2/ the students’ actual improvement level in their interpreting skills with the aid of Audionote feedback and 3/ possible hardships for the teacher in the process

of using the app In this study, four data collection instruments, namely questionnaire,

interview, pre-intervention and post- intervention tests and teaching journal were used The research results showed the students‘ remarkably positive responses to and high evaluation of Audionote in improving the teacher‘ s feedback quality thanks to a variety of advantages including real-time comments which provided linguistic contexts for the teacher‘s feedback especially when the students encountered enormous cognitive load in simultaneous interpreting practices On the other hand, the students‘ actual interpreting skill enhancement from the pre-intervention and post-intervention tests was not as highly positive as the students‘ responses regarding their interpreting skill improvement in the survey and interview More notably, the teacher also encountered various challenges hampering the procedure of applying the new technology in class

TABLE OF CONTENTS

DECLARATION……….… …i

ACKNOWLEDGEMENTS……….ii

ABSTRACT ……… iii

TABLE OF CONTENTS.……… iv

LIST OF ABBREVIATIONS ……… …vii

LIST OF TABLES, CHARTS AND FIGURES….………viii

CHAPTER 1: INTRODUCTION 1

1.1 Statement of research problems and rationale for the study 1

1.2 Educational context and participants 3

1.3 Aims and objectives of the study 4

1.4 Scope of the study 5

1.5 Significance of the study 6

1.6 Structure of the thesis 7

CHAPTER 2: LITERATURE REVIEW 8

2.1 Feedback 8

2.1.1 Definition of feedback 8

2.1.2 Types of feedback 9

2.1.3 Importance of feedback 10

2.1.4 Methods of giving feedback 10

2.1.5 Characteristics of effective feedback 11

2.2 Interpreting 12

2.2.1 Definition of interpreting 12

2.2.2 Rubrics for assessing interpreting performances 13

2.2.2.1 Assessment criteria for interpreting performances 13

2.2.2.2 Marking rubrics for interpreting performances 16

2.2.2.3 Feedback rubrics for interpreting performances 22

2.3 Audionote and its application in giving feedback for speaking classes 23

2.4 Review of related studies 24

CHAPTER 3: RESEARCH METHODOLOGY 26

3.1 Research questions 26

3.2 Research approach 26

3.3 Data collection instruments 29

3.3.1 Pre-intervention tests and post-intervention tests 29

3.3.2 Pre-intervention survey and post-intervention survey 30

3.3.3 Pre-intervention interviews and post-intervention interviews 30

3.3.4 While-intervention classroom observer interview 31

3.3.5 While-intervention teaching journals 31

3.4 Data collection procedure 34

3.5 Data analysis procedure 41

3.5.1 Pre-intervention tests and post-intervention tests 41

3.5.2 Pre-intervention survey and post-intervention survey 42

3.5.3 Interviews (Pre-intervention and post-intervention interviews with the students and while-intervention interview with the classroom observer) 43

3.5.4 While-intervention teaching journals 48

CHAPTER 4: DATA ANALYSIS, FINDINGS AND DISCUSSION 49

4.1 Answer to research question 1: What were the opinions of the students about the use of Notepad and Voice Recorder technology (Audionote) in the teacher‘s feedback for the students‘ classroom interpreting performances? 49

4.1.1 The students‘ opinions on advantages of Audionote feedback as opposed to disadvantages of traditional feedback 49

4.1.1.1 Disadvantages of traditional feedback and the learners‘ expectations before the intervention 50

4.1.1.2 Advantages of Audionote in the teacher‘s feedback 54

4.1.1.2.1 Improving the teacher‘s feedback quality 54

4.1.1.2.2 Helping the students‘ reflection after their practice 60

4.1.1.2.3 Reducing the burden of cognitive load 61

4.1.1.2.4 Saving time 62

4.1.2 Disadvantages of Audionote 63

4.1.3 The students‘ suggestions and expectations after the intervention 64

students‘ interpreting performances? 68

4.2.1 Results of pre-intervention and post-intervention tests 68

4.2.2 The students‘ responses from the post-intervention survey and interview 74

4.2.2.1 The students‘ explanations for their high evaluation of using Audionote 75

4.2.2.2 Remaining problems of using Audionote 77

4.3 Answer to research question 3: What were the difficulties of the teacher when applying Notepad and Voice Recorder technology (Audionote) in giving feedback for the students‘ classroom interpreting performances (if any)? 79

4.3.1 Lack of a standardized feedback rubric 79

4.3.2 Lack of in-class time for the students‘ feedback 81

4.3.3 Lack of a standardized interpreting test for the pre-intervention and post-intervention test 81

4.3.4 The student‘s reluctance to review Audionote files at home for self-reflection 82

4.4 Summary of the chapter 82

CHAPTER 5: CONCLUSION 83

5.1 Recapitulation 83

5.2 Pedagogical implications 85

5.2.1 For teachers 85

5.2.1 For learners 87

5.3 Limitations of the study 87

5.4 Further research 88

REFERENCES 90 APPENDICES I

LIST OF TABLES, CHARTS AND FIGURES

Table 6 The students‘ evaluation of traditional feedback in the

pre-intervention surveys and interviews vs their evaluation of

Audionote feedback from the post-intervention surveys and

interviews

51

Table 7 Paired Samples Statistics (Improvement level) 70 Table 8 Paired Samples Correlations (Improvement level) 70

Table 10 Paired Samples Statistics (Version 2 and Version 2‘) 72 Table 11 Paired Samples Correlations (Version 2 and Version 2‘) 72 Table 12 Paired Samples Test (Version 2 and Version 2‘) 72

CHARTS Page Chart 1 The students‘ rating of their previous teachers‘ feedback for

improving their interpretation

52

Chart 2 The students‘ evaluation of Audionote in improving the

quality of their teacher‘ feedback for their interpreting

performances

56

Chart 3 The students‘ evaluation of the teacher‘s advice quality in

improving their interpreting skills with the aid of Audionote

Chart 6 Comparison of the students' marks for version 2' in the

post-intervention test with that for version 2 in the

pre-intervention test

71

Chart 7 The students‘ evaluation of the efficiency of using Audionote

feedback in improving their interpreting performances

74

Chart 8 The students' evaluation of the importance of having a

feedback rubric (specific criteria) when teachers give

feedback using Audionote

80

FIGURES Page Figure 1 Cyclical action research model (Kemmis and mc Taggart,

1988)

36

Figure 2 Coding examples for open-ended questions in the surveys 43

Figure 3 Process of analyzing data from the interviews 45

Figure 5 Set of codes and color-coding system for the interviews 48

CHAPTER 1: INTRODUCTION

This chapter generally introduces the whole research, leading the readers from a general to a specific field of research The section establishes the context and significance of the study by summarizing current background knowledge of the topic and stating the aims of the work The research problem, rationale, objectives, and importance as well as the organization of the study are presented one by one

1.1 Statement of research problems and rationale for the study

Giving feedback for students is of great significance in formative assessment Informative and constructive feedback will contribute a lot to students‘ skill enhancement As suggested in a book entitled "Giving feedback

on speaking" by Kerr (2017), evidence tends to prove that feedback can improve language learning indeed

However, giving comments on students' real-time interpreting is stressful

As an interpreter trainer, I am not satisfied with the rushed, hand-written feedback notes I want to find a tool to help giving feedback in interpreting classes more feed-forward, thus improving the students‘ interpreting performance more efficiently

To date, there is a consensus among social scientists that computer-assisted feedback is likely to bring about a variety of advantages Smith and Harris (2014) pointed out the role of advanced technology in empowering interpreter trainers by ―providing a superior experience for tech-savvy students over recordings on physical media" (p.120) Smith and Harris also gave an insight into particular improvements regarding "time-coded feedback and commentary" (p.120) between students and interpreter trainers who used new technology (GoREACT, in this case) in giving comments for students

Dodd and Kusaka (2014) conducted another action research study about

activities in their own institution Various merits of the app have been revealed

as it can record and synchronize spoken and written ‗texts‘ into a digital file which can be used on computers and smart devices including computers, phones and tablets According to Dodd and Kusaka (2014, p.44), with the aid of this technology, teachers are able to capture students‘ speaking activities along with any feedback notes and send those files to them after the lessons It is easy for the students to replay the digital files and see their teacher‘s feedback comments written on their device at the corresponding moments in the speaking activity This will definitely help them to locate the context in which the feedback applies ―This visual and auditory inter-textuality is likely to appeal to students

of varying learning styles, and to teachers who want their feedback to be retained

in its original context‖ (Dodd & Kusaka (2014, p.44)

More importantly, as the nature of simultaneous interpreting task makes the amount of information being processed become overwhelming especially for new interpreters and students, it is a real challenge for them (Seeber, 2011) Under great time pressure and cognitive stress of advanced interpreting courses which involve both consecutive and simultaneous modes of interpreting, the students have a tendency to forget a lot of things in their performance right after they finish interpreting Therefore, it is quite hard for them to match what the teacher comments with what they have said It is even more challenging for them

to remember details such as their mistakes in pronunciation or grammar, and word choice To put it differently, decontextualized feedback has become a headachy issue in my interpreting course With the help of Audionote, I can easily replay the recordings and let my students see my feedback typed on Audionote files at corresponding moments in the students' speaking activity Audionote is likely to help me provide more contextualized feedback for my students, thus supporting them more effectively in their learning process

These findings were really appealing to me as an interpreter trainer who has encountered several difficulties in giving effective feedback for my students in interpreting lessons Nevertheless, there has limited research body on the use of Audionote in giving feedback for interpreter trainees

Those findings and arguments are the main reasons triggering my research study

1.2 Educational context and participants

1.2.1 Setting of the chosen class

The research was undertaken at the Interpreter and Translator Training Division of a foreign language university in Vietnam A class of twenty participants was chosen All the participants are senior university students majoring in English translation and interpreting They have learnt at least one interpreting practice course before so that they have got some experience about how traditional interpreter trainers give feedback in interpreting lessons This semester, the participants took their Advanced Interpreting course – the students‘ second interpreting course in the curricula The intervention phase took place during this course

In terms of setting and available teaching facilities, the classroom is equipped with a projector, a speaker, two interpreting booths, one interpreting tool kit, and internet connection Teachers and students had smartphones and/ or laptops that could record audio files and read basic digital files

1.2.2 Identification of the problems in the class

It is apparent that with a large class size of 20 students, it is hard for the teacher to give all students chances to practice in-class with traditional individual activities Usually, during a three-period session, only about 10 students can be given chances to practice and receive detailed comments Indeed, large class size

is a burden for the teacher as it reduces the opportunities for students to speak up

in class

More seriously, giving comments on students' real-time interpreting is stressful As an interpreter trainer, I think that the rushed, hand-written feedback notes are not satisfactory enough Given the importance of formative feedback

Most importantly, due to the great deal of stress that student interpreters suffer from during their advanced interpreting courses, it is hard for them to recall what they have interpreted and do reflection when listening to teachers‘ feedback after their performances This even worsens the fact that teachers‘ traditional feedback often lacks necessary contexts for each point of their comments

In short, large class size which hinders students from practising in-class, and no chance to give feed-forward contextualized feedback for students' interpreting performance are the two main problems in my simultaneous interpreting class

1.3 Aims and objectives of the study

Not only did the thesis aim at shedding light on the influence of Audionote

on students‘ interpreting performance improvement, and their reaction to the

teacher‘s use of Notepad and Voice Recorder technology (Audionote) in giving

feedback for students’ classroom interpreting performance but it also sought for

any possible problems and difficulties the teacher might encounter when applying this technology in the class and possible solutions (if any) More specifically, my research study addressed the following three questions:

Research question 1: What were the opinions of the students about the use

of Audionote in the teacher’s feedback for the students’ classroom interpreting performances?

Research question 2: To what extent did the application of Audionote in

the teacher’s feedback improve the students’ interpreting performances?

Research question 3: What were the difficulties of the teacher when

applying Audionote in giving feedback for the students’ classroom interpreting performances (if any)?

It should be noted that all the research questions only aimed at finding answers to the specific problems in my own interpreting class with very particular contexts No generalization was intended to be made for other classes with different contexts and different students‘ features The results of the study

are used to offer solutions to the current problems in the chosen class, thus building the basis and input for future research in similar issues

1.4 Scope of the study

This action research study only focuses on investigating the use of Audionote (Notepad and Voice Recorder) in giving feedback for the students‘ interpreting performance

Firstly, in terms of the intervention tool, I only used Audionote – an application that provides notepad and voice recorder technology in the intervention stage Other applications that also provide notepad and voice recorder technology but may have different features will not be discussed in this paper

Secondly, the type of feedback mentioned and examined in this research is the teacher‘s formative feedback in the classroom setting Hence, other types of feedback in other settings are not expected to be discussed here

Thirdly, in terms of interpreting types, this action research study set out to investigate the students‘ Vietnamese – English simultaneous interpreting at an introductory level As a result, this study did not discuss the use of Audionote in providing feedback for other types of student interpreters in other types of interpreting with dissimilar educational contexts

Finally, in this study, the term ―performance‖ was used instead of

―competence‖ as more emphasis was put on the students‘ practice and their ability to perform interpreting skills rather than their underlying knowledge of interpreting Chomsky (1969) first drew the distinction between ―performance‖ and ―competence‖ by pointing out that ―competence is the knowledge of the language‖ while performance is the ―actual use of the language in concrete situations‖ Likewise, the idea was echoed by Bachman (as cited in Brown, Malmkjær & Williams, 2004) In other words, competence refers to the

―unobserved, underlying knowledge‖ of the language, meanwhile ―performance

the term ―performance‖ consistently throughout the paper instead of

―competence‖

1.5 Significance of the study

On completion, the study brought about certain benefits to interpreting students, interpreter trainers, researchers and the entire society in general who

share the same interest in this topic

In the first place, to the interpreting students‘ side, the findings of the research helped them to raise their voice in evaluating their own interpreter trainers‘ feedback which directly influences their learning process So far, little research has been carried out to delve into this aspect while it is obviously critical to listen to such reflection from students – users of the feedback This paper sought to explore student interpreters' opinions about trainers' feedback and its perceived effects on their learning The research findings are expected to provide insights into interpreter trainers' everyday teaching and learning and guide the teachers in how they can further assist the students in a more efficient way that is constructive to their learning and skill development

In the second place, to interpreter trainers' side, due to the limited number

of studies about assessing and improving quality of their in-class feedback for students, hardly can interpreter trainers find ways to enhance the quality of their comments on students‘ performance The idea from this action research may help interpreter trainers to think of applying a new promising method in their own classes Hopefully, this will contribute to the enhancement of interpreting training in general

In the third place, as for researchers who take interest in the same topic, they can refer to this paper as a source of updated and reliable information Last but not least, as this is an action research paper, the researcher does not aim at generalizing the effectiveness of using Audionote in every single interpreting class Rather, the study focuses on solving the current problems in the chosen class, thus building the basis and input for further large-scale research

1.6 Structure of the thesis

The study has five major chapters In Chapter 1 – Introduction, a big picture of the research problem, rationale, aims and scope of the study is depicted Chapter 2 is Literature Review which provides a theoretical framework, encompassing all major concepts of the research Chapter 3 – Methodology describes the research design, answering how the data was collected and analyzed Chapter 4 – Data Analysis, Findings and Discussion - presents the main findings of the research accompanied by the researcher‘s discussion The last chapter is Conclusion, grasping all the main points of the research, including its findings, limitations and suggestions for further studies

CHAPTER 2: LITERATURE REVIEW

As the theoretical framework for the whole research, this chapter mainly introduces and discusses different concepts related to the study such as feedback, interpreter training, criteria for assessing interpreting, and Audionote

2.1 Feedback

2.1.1 Definition of feedback

It is essential to start with the concept of feedback in this study According

to Kerr (2017, p 2), feedback is defined as any kind of information that students receive about their performance This can be in the forms of either corrective feedback which focuses a learner‘s attention on errors, or non-corrective feedback, such as praise or encouragement Besides, the feedback can also be about the performance of peers Information provided via feedback may include

―not only answer correctness, but other information such as precision, timeliness, learning guidance, motivational messages, lesson sequence advisement, critical comparisons, and learning focus‖ (Hoska, 1993)

Different scholars have different viewpoints about the aims of formative feedback According to Shute (2008, p.156), formative feedback is supposed to foster students' knowledge, skills, and comprehension in particular content areas

or general skills such as problem-solving The objective of giving feedback is to bridging the gaps between the current students' performance and a particular goal

or standard (Hattie & Timperley; 2007) Two major purposes of feedback, as

claimed by Shute (2008, p.157), are directive (telling learners what needs to be corrected and revised), and facilitative (giving comments and suggestions to

orient students in their own revisions and conceptualization) Each opinion touches upon particular aspects of formative feedback However, most of the above-mentioned scholars agreed that formative feedback aims at improving learners' knowledge or skills in accordance with specific learning objectives

2.1.2 Types of feedback

A search of the literature reveals several ways of categorizing feedback: formal and informal feedback, qualitative and quantitative feedback, teacher feedback and peer feedback, etc However, considering the relevance to this study, the researcher will just focus on two main ways of categorization

Firstly, feedback can be generally divided into qualitative and quantitative one Qualitative feedback includes (textual) comments, meanwhile, quantitative feedback is given in forms of "numerical marks" (Hamer et al., 2015, p 152) Formative feedback touches upon the "accuracy or adequacy of a student' performance or response" and may deal with specific errors and misconceptions (Shute, 2008, p 154)

Secondly, based on its source, feedback is categorized into peer feedback, teacher feedback and self-feedback External feedback from peers and teachers plays an important role in augmenting students' competence since it often offers

an objective viewpoint and constructive advice for the learners Besides, internal feedback or self-feedback generated by the students themselves help them to reflect on their own performance and figure out their strengths and weaknesses (Lee, 2018, p 154) Therefore, learners are more self-oriented to work on those particular points and build up their skills

In this research study, the researcher refers to feedback as any kind of comments that the interpreter trainer gives his or her students in interpreting lessons to help the students build up their interpreting competence In other words, the feedback that is of interest in this study concerns more about qualitative and formative feedback rather than quantitative one However, during the interpreter training course that the research study took place, the interpreter sometimes used both numerical marks and textual information in her comments Additionally, within the scope of this study, more focus was drawn to teachers‘

on learning and achievement‖ (Hattie & Timperley; 2007) Moreover, Lee (2018) also emphasized the importance of giving feedback for any types of learning, and ―interpreter training is no exception‖ In real English teaching practice, teachers‘ feedback can be considered one of the most efficient ways of helping students to correct their mistakes and building up language skills Though feedback is such an integral aspect of interpreter training, there has been limited research body on feedback in interpreting courses, let alone ways of enhancing feedback quality for trainers in the field Additionally, given the remarkable technology advancement that is taking place day by day, it is of great necessity to explore how advanced technology can support interpreter trainers in improving the quality of their feedback for student interpreters

2.1.4 Methods of giving feedback

In terms of mode, feedback can be provided either face-to-face or mediated

by technology Teachers can facilitate the students‘ performance and take notes then deliver the feedback afterwards or give students written or oral feedback after listening to recordings of student performance (Lee, 2018)

Research studies have pointed out several advantages of assisted feedback It is claimed that "technology-enhanced feedback, which enables flexible feedback provision, does seem to hold promise for the future" (Yang & Carless, 2013, p.292) More importantly, research has also reflected students' positive reception and reaction to technology-mediated feedback (Dodd and Kusaka, 2014) Finally, applying ―web-based learning management systems

technology-in technology-interpreter tratechnology-intechnology-ing helps facilitate feedback and technology-interactions between teacher and learners and thus enhances the benefits of collaborative learning‖ (Lee et al

2015, p.137)

A newly emerged mode of giving feedback for speaking classes is using audio and note-taking technology Recent research has proved several benefits that this kind of synchronizing technology can bring about in the process of giving in-class feedback Some applications which have been used are GoREACT (Smith & Harris, 2014) and Audionote This enables teachers to record and take notes or comments at the same time, thus providing time-coded and feed-forward feedback for students while not disturbing their fluency practice (Dodd and Kusaka, 2014) This new trend has drawn my considerable attention since the aid of this technology can be the answer to remaining problems with decontextualized feedback in most interpreting classes

2.1.5 Characteristics of effective feedback

To produce effective feedback, it is advised that teachers pay attention to its ―function‖, ―content‖, and modes of ―presentation‖ together with students‘ characteristics and instructional factors or context such as learning goals and learning tasks (Shute, 2008, p.173) Although timely and regular feedback is considered beneficial, the content of feedback seems to play a more important role in the effectiveness of giving feedback (Bangert-Drowns et al 1991; Scott

2014, p.50) More notably, research has shown that effective feedback should be specific and goal-related (Voerman et al 2014) Feedback with detailed suggestions to improve students‘ performance will be more useful than the one which merely points out accuracy and adequacy (Bangert-Drowns et al 1991; Scott 2014) Shute (2008) agreed with this viewpoint, stating that vague feedback without specificity may result in little positive effects on learners In short, a thorough search of the literature has revealed several important qualities

of effective feedback: specific, goal-related, timely, and regular

A great deal of previous research into feedback has focused on characteristics of formative feedback for English language teaching in general, yet there is a relatively small body of literature that is concerned with feedback

teachers‘ attention and support in terms of both academic and emotional needs in their provision of feedback given the excessive stress and constraint students have to suffer during their interpretation training This piece of information was

a valuable input to the researcher in planning for this particular research study about enhancing feedback quality in her own interpreting course

In the interpreting process, "communication is immediate, involving interaction between speakers, listeners, and interpreters‖ (AIIC, 2012) As a result, in interpreting practice, specific linguistic resources are made use of: "the original speaker's ideas are transmitted as spoken words, with a particular rhythm and intonation, making use of rhetorical devices and gestures" (AIIC, 2012)

Two major modes of interpreting are simultaneous interpreting in which

interpretation is carried out in real-time when the speaker is speaking, and

consecutive interpreting in which interpreters listen to speakers and then renders

the message when the speakers pause (Gillies, 2014)

Simultaneous interpreting can be defined as the procedure of linguistic transfer of meaning in real-time (Seeber, 2011) The term

"cross-―simultaneous‖ does not refer to ―the simultaneity of the comprehension and production of one and the same sentence constituent, but the general temporal overlap of language comprehension and language production‖ (Seeber, 2011)

As simultaneous interpreting involves the combination of language comprehension task and language production task, so it becomes a very stressful work for interpreters According to Seeber (2011), ―in cognitive processing terms, the real-time combination of the two means that they compete for available resources‖

2.2.2 Rubrics for assessing interpreting performances

Given the context of this paper about quality of feedback for interpreter training, it is essential to investigate different criteria for interpreting quality assessment as it may help to build up a framework for the researcher in giving feedback for the students Also, a detailed marking rubric for simultaneous interpreting was also needed in the process of marking the pre-test and post-test

to answer question 1 of the research To put it differently, this part of the literature review will analyze dissimilar theories to come up with both a detailed marking rubric and a simplified feedback rubric for Vietnamese – English simultaneous interpreting

2.2.2.1 Assessment criteria for interpreting performances

In the growing body of research on interpreting quality, very few papers have been written about criteria for interpreting quality assessment To date, some papers have discussed quality assessment criteria for interpreting performance, yet hardly has there been consensus on what quality is

Firstly, in his paper on the International Association of Conference

message‖, ―Logical cohesion of utterance‖, ―Correct grammatical usage‖,

―Completeness of interpretation‖, ―Fluency of delivery‖, ―Native accent‖, and

―Pleasant voice‖ Kahane (2000) presented two discrete groups of interpreters and users‘ evaluation of the importance of the above-mentioned criteria for interpreting quality assessment Though the percentage varies, interpreter and user groups both ranked the criteria alike in terms of importance The most important criteria are sense consistency with the original message and logical cohesion, while native accent and pleasant voice projection were the least important

More recently, in the article entitled ―Interpreters‘ Perception of Linguistic and Non-Linguistic Factors Affecting Quality: A Survey through the World Wide Web‖, Chiaro and Nocella (2004) presented the results of their quantitative research study regarding professional interpreters‘ perception of the quality of interpreting practice Based an original framework of assessment criteria for interpreting performance used by Buler (1986), Chiaro and Nocella (2004) also divided the set of assessment criteria into two major categories, namely linguistic criteria (consistency with original, completeness of information, fluency of delivery, etc.) and extra-linguistic criteria (concentration, ability to work in teams, physical well-being, etc.) An online survey was administered via the internet to investigate the evaluation of the targeted participants across five different continents about the importance of different linguistic and non-linguistic aspects It turned out that, among linguistic criteria, consistency with the original speech, completeness of information and logical cohesion were reported to be the most important, while concentration ranked the first among non-linguistic aspects

Zwischenberger (n.d.) conducted another study named ―Quality criteria in simultaneous interpreting: an international vs a national view‖ The research study illustrates conference interpreters‘ rating of the importance of different assessment criteria for simultaneous interpreting The study also sought to reveal the interpreters‘ perceptions of the correlation between the significance of different assessment criteria and changes in specific assignment types Finally, there was a comparison of the ideas from 2 different groups of participants,

namely members of the International Association of Conference Interpreters (AIIC) and the German Association of Conference Interpreters (VKD) Two content-related criteria, namely sense consistency with the original and logical cohesion received the highest overall ratings from both groups of participants The two above-mentioned studies by Chiaro and Nocella (2004) and Zwischenberger (n.d.) have several similarities as well as differences On the one hand, both studies aim to unveil fully-fledged interpreters' perspectives on the importance of different assessment criteria for interpreting performance In terms

of results, there was a consensus between the two papers that sense consistency

with the original was the most important factor to be considered when it comes to

interpretation The findings of Chiaro and Nocella (2004) and Zwischenberger (n.d.) were also similar to that proposed by Bühler (1986)

On the other hand, the most prominent difference between the two studies

by Chiaro and Nocella (2004) and Zwischenberger (n.d.) lies in the framework used to categorize criteria for assessing interpretation quality Chiaro and Nocella (2004) adapted an original framework of assessment criteria for interpreting performance used by Buler (1986) and divided the set of assessment criteria into two major categories, namely linguistic criteria (consistency with original, completeness of information, fluency of delivery, etc.) and extra-linguistic criteria (concentration, ability to work in teams, physical well-being, etc.) Their framework is relatively clear and easy to follow By the contrary, in Zwischenberger‘s study (n.d.), the source of her framework was nowhere to be found throughout the whole paper All criteria mentioned in Zwischenberger‘s study (n.d.) were linguistic rather than extra-linguistic ones Zwischenberger (n.d.) also categorized those items into three main groups, namely content-related criteria, form-related criteria, and delivery-related criteria Most of the specific criteria in each of those group are also mentioned in Chiaro and Nocella‘ s

framework (2004), except for synchronicity, which is quite an ambiguous

2.2.2.2 Marking rubrics for interpreting performances

Having a marking rubric plays a significant part in ensuring the consistency

of interpreting quality assessment process In the framework of this study, a marking rubric is required so that the students‘ performances in pre-tests and post-tests can be graded in a more reliable and consistent way

As can be seen in the previous part, a thorough search of the literature has unveiled some studies about different criteria for interpreting quality assessment and their order of importance Nevertheless, very little has been done about complete marking rubrics for interpreting practices Below are some notable studies about this issue

First, a Proposed Criteria of Peer-Feedback for Conference Interpreter Trainees was introduced on Online Resources for Conference Interpreter Training – ORCIT Website ("Proposed Criteria", n.d.) The rubric contains a quite comprehensive set of criteria for assessing interpreting performance, including four major macro-criteria expressed in the form of four questions Each macro-criterium is specified by 6 to 13 smaller micro-criteria For instance, question two about target language quality consists of 13 sub-criteria, namely

"appropriate use of synonyms'', "appropriate use of pronouns", "appropriate use

of linking words", "making sense", "natural expressions", "grammatical correctness", "concision", "vocabulary", "complete sentences", "frequency of self-correction", "clarity of self-correction", "interference", "level of formality" Some of the main criteria are the same as mentioned in Bühler (1986)‘s study Some other criteria have been added, including delivery of the Interpretation (confidence, clarity of articulation, liveliness of intonation, etc.) The full version

can be found in Appendix 1 One advantage of the list is that various criteria

have been grouped into four bigger macro-criteria for the sake of the evaluator Despite the comprehensiveness of this list, it may become impractical for teachers to use in interpreting course to make feedback for students since it will take a lot of time to go through such a long series of sub-criteria This list is too long with so many sub-criteria to be assessed in each macro-criterion suggested

in the four guided questions, which makes it unsuitable to become an in-class feedback rubric

The second research paper named ―The design and application of rubrics to assess signed language interpreting performance‖ was conducted by Wang et al (2015) In this study, the researchers designed a rubric to assess Auslan/ English simultaneous interpreting performances in both directions (from English to Auslan and vice versa) Four major criteria were mentioned in the rubric:

―accuracy, target language quality, delivery features, and processing skills‖ (Wang et al., 2015) The rubric had some particular strengths First, Wang et al (2015) divided it into four "macro-level assessment criteria" which were quite manageable for raters, compared to a long list with a dozen of micro-level criteria Second, there were specific percentages of achievement assigned to specific bands in the rubric, making the criteria more measurable for raters However, the rubric had some major weaknesses First of all, the band descriptors were brief and relatively general, causing a certain amount of ambiguity for raters Secondly, the names of sub-criteria in each main category were not clear and logical enough For instance, criterium 1 about ―accuracy‖ included ―Minimal unjustifiable additions, Minimal unjustifiable omissions, Minimal unjustifiable substitutions, Minimal unjustifiable intrusions, Cohesion and coherence‖ which mostly focused on students‘ mistakes, not what they achieve in their interpreting performance The sub-criteria in ―accuracy‖ criterium also overlapped with some sub-criteria in ―processing skills‖ criterium which included ―strategic additions‖ and ―strategic omission‖ Also, the sub-criterium of ―successful anticipations‖ in criterium 4 was quite hard to be evaluated given the context that raters can only listen to the examinee‘s target speech Finally, the second criterium about target language quality accounted for only 20% of the total score meanwhile this criterium including ―correct terminology‖ and ―correct grammar‖ ranked second among all the linguistic criteria in terms of importance according to studies by Chiaro and Nocella (2004) and Zwischenberger (n.d.) This macro-criterium should have received a

devised a marking rubric to assess interpreting performance and tested the reliability of the band descriptors as well as raters‘ feedback for the rubric itself There were three major criteria in the rubric: ―accuracy‖, ―target language quality‖, and ―delivery‖ which accounted for 40%, 40% and 20% of the total score respectively The rubric was quite rater-friendly thanks to several strengths First, the number of macro-criteria was three which is quite manageable for raters as it is easier and more practical to mark 3 criteria compared to giving grades for a long list of sub-criteria The names of the macro-criteria were also similar to those of the most important criteria for evaluating interpreting performances mentioned in previous studies in the field More importantly, the researcher provided a relatively detailed description of criteria and quite specific band descriptors compared to other available sources which were mostly short and difficult to measure Nonetheless, there remain some drawbacks of the rubric including lack of sub-criteria names and lack of percentages of achievement for the individual band

Within the framework of this thesis, in order to answer question 1 about the students‘ improvement in interpreting skills, a teacher-friendly marking rubric for simultaneous Vietnamese – English interpretation had to be devised based on available information in the literature Given the context of simultaneous interpretation, the emphasis was put on linguistic factors rather than extra-linguistic ones Therefore, extra-linguistic criteria such as eye contact, appropriate posture, concentration, preparation of documents, endurance, mnemonic skills, and encyclopaedic knowledge are excluded from the list for this marking rubric

All things considered, I decided to adapt the rubric by Lee (2008) as it seemed to be the most practical one available However, several changes were made to Lee's rubrics based on my knowledge through literature review and my own teaching and rating experience First, in terms of descriptions for the main criteria, specific names were added for sub-criteria within the three main macro-level criteria to make the categories become more detailed and specific Also, since the original rubric by Lee (2008) did not mention some important aspects such as "completeness of interpretation", so some significant sub-criteria were

selected from previous studies' results and added to the list to ensure its comprehensiveness Second, the names of the criteria, sub-criteria and keywords

in the band-descriptors were bolded and coloured systematically so that the raters would recognize and remember them more easily Third, to enhance the conciseness of the rubric by Lee (2008), the fourth column in that original rubric was deleted since it was just a summary of the third column Fourth, percentages

of achievement were assigned to each and every band so that the criteria became more measurable The idea arose from the proposed rubric by Wang, Napier, Goswell and Carmichael (2015) A full version of the rubric can be found in Table 1 below

Table 1: Marking criteria for Vietnamese – English simultaneous

- The interpreter voiced

all of the information

accurately with intended effects Accuracy: - the quality of

faithfully conveying the message of the speech with semantic and pragmatic equivalence i.e reproducing the same meaning and intended effect

- Deviations from the ST should be considered in terms of the effect on the

coherence/logic and faithful

rendering of the message

- Examples of deviations:

omissions, additions, and unjustifiable changes in the meaning

delivered with intended effects but a few minor deviations from

the source text were found, which

did not significantly affect the overall meaning or coherence

delivered but some deviations from the source text with an impact on the meaning and effect but coherence was maintained

delivered inaccurately with

many deviations from the source

compromised

inaccurately with serious deviations from the source text and incoherence

was incoherent and completely inconsistent with the source text

- The interpreter used

appropriate word choice and

terminology

grammatical usage

- The interpreter used

appropriate and correct

sentence structures

2.3 Pronunciation &

Intonation

- The interpreter had

clear articulation of sounds

and native accent

2.4 Naturalness

- The interpreting

performance sounds natural in

TT

- The interpreter used

production with few linguistic errors and appropriate target language expressions

TL quality:

- the quality of rendering in TL needs to be linguistically correct and appropriate in the context

- Examples of deviations from language norms: incorrect pronunciation, accent, and stress; incorrect grammar; unidiomatic language; interference from the source language;

inappropriate

production with a few minor linguistic errors that do not hinder immediate target language comprehension and

generally appropriate target language expressions

production with a few linguistic

immediate comprehension, but quite understandable A few

minor inappropriate target language expressions were found

production with some linguistic

natural expressions and

collocations or idiomatic

expressions in the TL

comprehension and some inappropriate target language expressions

language in the target culture and for the target audience (register misuse)

language production with many linguistic errors and inappropriate target language expressions were consistently found

production with inappropriate

target language expressions

- successful communication Examples of deviations: inarticulate speech, pauses, hesitation, false starts, fillers, irritating noise,

repetition, excessive repairs or self-correction,

unconvincing voice quality and monotonous

intonation,&

irritatingly slow speech rate

3.1 Fluency

- The interpreter spoke

fluently Natural pauses are

accepted

3.2 Voice projection

- The interpreter‘s

voice was at the right level,

not too soft or too loud

(Adapted from ―Rating Scales for Interpreting Performance Assessment‖ by Lee,

2008, The Interpreter and Translator Trainer, 2, p.172-173

Copyright 2008 by Taylor and Francis Group)

2.2.2.3 Feedback rubrics for interpreting performances

During the first cycle of this action research, I realized the need for a

feedback rubric that could be used in my own classroom together with

Audionote Therefore, I intended to base on and adapt those above-mentioned

criteria in previous studies about interpretation assessment to meet my classroom

context

The feedback rubric was a simplified version of the marking rubric

designed in the previous section in this chapter However, another section of

overall comments was added to allow overall feedback for the students‘

performance Besides, all the band descriptors were replaced by a ―Teachers‘

notes‖ column in which the teacher could note down any details or comments for

the students‘ performances while they were interpreting The reason was that

during the students' in-class learning process, the teacher's detailed comments in

the feedback seemed to be more meaningful and helpful compared to specific

marks without any comments and suggestions The full version of the rubric is

1 Accurate rendition of message

(40%)

Accurate rendition of message

(40%)

1.1 Sense consistency with the original message:

- The interpreter voiced all of the information

- The interpreter seemed to be faithful to ST

1.2 Completeness of interpretation

- The interpreter‘s sentences were complete

1.3 Logical cohesion

- Utterance had logical cohesion

2 Target language quality

(40%)

Target language quality

(40%) 2.1 Vocabulary and terminology

- The interpreter used appropriate word choice and

terminology

2.2 Correct grammatical usage

- The interpreter used appropriate and correct

sentence structures

2.3 Pronunciation & Intonation

- The interpreter had clear articulation of sounds

and native accent

2.4 Naturalness

- The interpreting performance sounds natural in TT

- The interpreter used natural expressions and

collocations or idiomatic expressions in the TL

3 Delivery-related criteria (20%) Delivery-related criteria

(20%) 3.1 Fluency

- The interpreter spoke fluently Natural pauses are

accepted

3.2 Voice projection

- The interpreter‘s voice was at the right level, not

too soft or too loud

- The interpreter‘s voice was clear and easy to

understand

Total mark

Overall comments:

_ _ (Adapted from ―Rating Scales for Interpreting Performance Assessment‖ by Lee,

2008, The Interpreter and Translator Trainer, 2, p.172-173 Copyright 2008 by

Taylor and Francis Group)

2.3 Audionote and its application in giving feedback for speaking classes

Audionote application (app) is a recent advance in mobile learning It combines the functionality of a notepad and voice recorder to create a powerful tool which can save time while improving the quality of the notes Dodd and Kusaka (2014) asserted that this application can empower teachers to record learners speaking performance along with any feedback notes, and later send

speaking activities As a result, the app helps to contextualize teachers' feedback for students' speaking performance

Appealed by the functions of this application, the researcher thought that this technology could also be used in giving feedback for interpreting performance, a special type of speaking activities where students need to orally convey messages from the source language to the target language Audionote, therefore, has been selected as the main intervention tool to address specific problems of giving feedback in the interpretation course

2.4 Review of related studies

There have been a number of studies on the use of computer-assisted feedback for students‘ spoken performance, yet hardly can such studies be found

in the context of Vietnam, especially for interpreting courses

Previous research has established that computer-assisted feedback can

bring about a vast array of advantages In a research article named ―Effective

interpreter feedback and instruction using GoREACT”, Smith and Harris (2014)

pointed out how advanced technologies can empower interpreter trainers by

―providing a superior experience for tech-savvy students over recordings on physical media" (p.120) The researchers also gave an insight into particular enhancements in terms of "time-coded feedback and commentary" (Smith & Harris, 2014, p.120) between students and interpreter trainers who used new technology (GoREACT, in this case) in giving comments for students‘ improvement

Another action research study about ―Using Web 2.0: Synchronizing technology to improve feedback on spoken academic texts‖ by Dodd and Kusaka (2014) thoroughly described the process of applying Notepad and Voice Recorder technology (Audionote) in teacher‘s feedback for students‘ speaking activities in La Trobe Melbourne (LTM) The researchers suggested several benefits of the app as it is able to ―capture and synchronize spoken and written

‗texts‘ into a digital file, deployable on computers and smart devices such as phones and tablets‖ According to Dodd and Kusaka (2014):

Through this technology, teachers can record students‘ speaking activities along with any feedback notes, and later send these files onto them The students can replay them and see their teacher‘s feedback comments highlighted on their device at the corresponding moments in the speaking activity, helping them to locate the context in which the feedback applies This visual and auditory inter-textuality is likely to appeal to students of varying learning styles, and to teachers who want their feedback to be retained in its original context (p 44)

The advantages of Notepad and Voice recorder technology (Audionote) in teacher‘s feedback has been proved in speaking classes However, there has very limited research body on the use of Audionote in delivering feedback for interpreter trainees There remains a research gap regarding the effectiveness of using Audionote and student interpreters‘ responses to the teacher‘s feedback with the aid of this app

CHAPTER 3: RESEARCH METHODOLOGY

This section describes the rationale for the application of particular procedures which were used to identify, collect and analyze the information applied to solve the research problem The whole chapter enables readers to critically evaluate the research’s overall validity and reliability

3.1 Research questions

This study sought to obtain data which helped to answer the following research questions:

Research question 1: What were the opinions of the students about the use

of Notepad and Voice Recorder technology (Audionote) in the teacher’s feedback for the students’ classroom interpreting performances?

Research question 2: To what extent did the application of Notepad and

Voice Recorder technology (Audionote) in the teacher’s feedback improve the students’ interpreting performances?

Research question 3: What were the difficulties of the teacher when

applying Notepad and Voice Recorder technology (Audionote) in giving feedback for the students’ classroom interpreting performances (if any)?

3.2 Research approach

This thesis follows action research approach, with an in-depth analysis of the use of Audionote in the teacher's feedback for the students' interpreting practices in the classroom setting

 Reasons for choosing action research approach

Ebbutt (1985) defines action research as a systematic study involving both action and reflection with a view to improving practice Cohen, Manion & Morrison (2007) mentions action research as "a small-scale intervention in the functioning of the real world and a close examination of the effects of such an intervention" Besides, in the process of doing action research, the researcher will "plan, act, observe and reflect more carefully, more systematically, and

more rigorously than one usually does in everyday life" (Kemmis and McTaggart, 1992, p.10)

Additionally, Zuber (1996, p.83) emphasizes that "the aims of any action research project or programme are to bring about practical improvement, innovation, change or development of social practice, and the practitioners' better understanding of their practices" Similarly, Kemmis and McTaggart (1992) claim that action research is an approach to improve education by

"changing" it and learn from the results of changes

The nature and aims of action research fit well with the aims and objectives

of this study, creating a rationale for me to choose action research as the research approach To be more specific, in this research project, I intended to use a new intervention (Notepad and Voice Recorder technology - Audionote) to create changes and improve an existing problem in the classroom – decontextualized and ineffective feedback from the teacher The systematic reflection during this research process is expected to help me gather data about the students' perceptions of the use of Audionote and hence solve my own problem regarding giving feedback in my interpreting courses In short, action research is the most suitable research approach for my study

 Potential pitfalls of action research

Despite various merits of action research, including bridging the gap between theory and practice and making practical improvement and positive changes in teaching and learning, some scholars have also pointed out some potential pitfalls of the approach

Firstly, it is subjectivity in the process of conducting action research that

affects its reliability (Cohen, Manion & Morrison, 2007) Since the researcher is also the teacher, sometimes it is hard to avoid personal bias or subjectivity in the

Secondly, lack of a theoretical framework is another noteworthy issue for

action research As this approach is based mostly on practice, action researchers are often at the risk of lacking a firm theoretical foundation or framework in their study

Finally, lack of generalizability is also worth considering Action research

involves a very specific educational context, and a limited number of participants (usually in a particular class), hence few possibilities to generalize for other settings and participants

 Measures taken to overcome those potential pitfalls of action research

To handle the above-mentioned possible demerits of action research, I have implemented several measures

First and foremost, to mitigate the inevitable subjectivity and increase the

reliability of the study, both researcher triangulation and method triangulation were used As for researcher triangulation, two independent raters including one

colleague in my faculty and I were involved in marking the pre-intervention and post-intervention tests on the students' simultaneous interpreting skills More notably, another senior lecturer was invited to observe my class during the intervention phase, so that I could interview her for objective comments on my

intervention Regarding method triangulation, a combination of more than one

data collection instrument was applied, so that the data collected from different angles could become more comprehensive For instance, I used student surveys and interviews to answer RQ1 Pre-intervention and post-intervention tests, survey and interviews with the learners were used to answer RQ2 Additionally, RQ3 was answered based on data from teaching journals and a classroom observer interview

In the second place, to tackle the shortage of a theoretical framework, I have carried out a thorough and comprehensive review of the literature related to feedback (definition, categorization, characteristics of effective feedback, etc.)