Survey research into an assessment of students’ speaking skill by teachers at equest academy

The results have revealed that classroom speaking assessment currently conducted in EQuest Academy has broadly employed performance-based tasks and that somewhat informative feedback has

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES

FACULTY OF POST-GRADUATE STUDIES

*********************

Nghiên cứu về việc đánh giá kĩ năng nói Tiếng Anh của học viên bởi

giáo viên tại học viện anh ngữ Equest

M.A MINOR PROGRAMME THESIS

Field : English Teaching Methodology Code : 60140111

HANOI – 2017

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES

FACULTY OF POST-GRADUATE STUDIES

*********************

Nghiên cứu về việc đánh giá kĩ năng nói Tiếng Anh của học viên bởi

giáo viên tại học viện anh ngữ Equest

M.A MINOR PROGRAMME THESIS

Field : English Teaching Methodology Code : 60140111

Supervisor : Assoc Dr Ngô Hữu Hoàng

HANOI - 2017

DECLARATION

I declare that the materials in this thesis have not been submitted previously or concurrently in the whole or as part of requirements for a degree of ULIS relating to the retention and use of M.A Thesis deposited in the library

I also certify that the work presented in the thesis is to the best of my knowledge and belief, original except as acknowledged in the text Any help that I have received during the preparation and completion of this thesis has been acknowledged In addition, I certify all information sources and literature used are indicated in the thesis

Date: 2017 8 15

Signature:

ACKNOWLEDGEMENTS

This research was accomplished with the help of many people First, I would like to express my heartfelt gratitude to my supervisor, Assoc., Dr Ngô Hữu Hoàng for his guidance, invaluable advice, and constant encouragement He has been my mentor academically and personally His constant encouragement has helped me complete this thesis

My further gratitude goes to the English department chairs at ULIS Especially, Assoc Prof Dr Lê Hùng Tiến, Assoc Prof Dr.Lê Văn Canh, and Dr Huỳnh Anh Tuấn , who have shared their knowledge with me during my study I am grateful to all the research participants, including my teacher colleagues who helped arrange the field work for me Many thanks go to all my friends, Hiền, Ngân and Trang who gave invaluable advice and a lot of encouragement to me

I would also like to express my appreciation and heartfelt gratitude to my parents who have always believed in me and supported my dreams I also thank my sisters: Thuỳ Linh and Quỳnh Mai for their cooperation all through Most of all, I would like to thank my husband, Quang Vũ, for his love, care, emotional and intellectual support, for his continuous encouragement, and bearing with me during all the tough times The completion of my master program would be impossible without him

ABSTRACT

The present study investigates the teachers‟ classroom speaking assessment in EQuest Academy, a famous English center in Hanoi The data is collected from questionnaires and interviews where teachers were the only targeted respondents 10 teachers recently working in EQuest participated in the questionnaire and are interviewed The results have revealed that classroom speaking assessment currently conducted in EQuest Academy has broadly employed performance-based tasks and that somewhat informative feedback has been offered to students in the form of criterion descriptions plus marking scores, one which resulted in teachers having an overall pessimistic attitude towards the positive effects of such testing on teaching and learning It is evident from this study that there is need for improvements in order to facilitate better learning outcomes in the classroom Since EQuest teachers are not trained adequately the theory speaking assessment, there is less confidence in assessing students‟ speaking skill

The study also finds that the speaking assessment tasks used by EQuest English teachers are those which gave the students less psychological burden; are time-saving and designed for the convenience of construction and administration and do not demand the teacher to take the role of an interviewer

TABLE OF CONTENTS DECLARATION I ACKNOWLEDGEMENTS II ABSTRACT III TABLE OF CONTENTS IV

PART A: INTRODUCTION 1

I Rationale 1

II Background to the research 2

II.1 English education in EQuest Academy 2

II.2 Characteristics of English assessment in EQuest Academy 2

III Research aims 3

IV Research questions 4

PART B: DEVELOPMENT 5

Chapter One: Literature Review 5

1.1.Key term definitions 5

1.1.1 Test vs Assessment 5

1.1.2 Classroom speaking assessment 5

1.2 The advantages of classroom assessment 6

1.2.1 Academic advantages 6

1.2.2 Advantages for speaking assessment 7

1.3 Issues related to reliability in classroom assessment 9

1.4 Issues related to validity in classroom assessment 9

1.5 Teachers' perceptions 10

Chapter Two: Methodology 11

2.1 A qualitative approach 11

2.2 Steps in the selection of participants 12

2.2.1 Questionnaire participants 12

2.2.2 Interview informants 13

2.3 Data collection procedures 14

2.4.Data analysis 15

Chapter Three 18

Findings and Discussions 18

3.1 Findings 18

3.1.1 Teachers' perceptions of speaking assessment in EQuest 18

3.1.2 Positive effects of classroom speaking assessment on teaching and learning 19 3.1.3 Types of speaking assessment tasks used by EQuest teachers 20

3.1.4 Teachers' perceptions of the practical constraints in conducting communicative speaking assessment 24

3.1.4.1 Difficulties caused by the class 24

3.1.4.2 Difficulties caused by the students 25

3.1.4.3 Difficulties caused by the teachers 26

3.2 Discussion 27

PART C: Conclusion 30

I Recapitulation 30

II Implications 31

III Limitations of the study 32

IV Suggestions for further study 33

References 34 APPENDIX A: QUESTIONNAIRE I APPENDIX B: INTERVIEW QUESTIONS III APPENDIX C: PARTICIPANT CONSENT FORM AND PARTICIPANT INFORMATION SHEET V APPENDIX D: SUMMARY OF INTERVIEWEES’BACKGROUND INFORMATION VI

PART A: INTRODUCTION

I Rationale

In Vietnam there has long been discussion about how Vietnamese people speak English, and this discussion usually arouses strong feelings in people Like many other Asian countries, Vietnamese people begin their English education very early

at school, yet they find themselves helpless when it comes to speaking English Words do not come out Many Vietnamese students do have the words, they either heard them or read them but they can't say them The reason might be that oral skills have not usually been emphasized in Vietnamese schools and oral skills are not tested in the matriculation exam However, speaking skills are increasingly important in work life, and many Vietnamese students have expressed the need for learning better oral skills and thought speaking to be the most important area of learning in English As a result, many Vietnamese take part in extra English classes

in English centers like EQuest, RES, ILA and so on Obviously, there is a clear need for better oral skills, but the means how to achieve them are still developing

My study focuses on the assessment of oral skills, and I wanted to emphasize English classrooms in EQuest Academy, not in any schools because of the following reasons Firstly, recently there are so many students learning English outside schools and most of the feedback and evaluation they receive come from their teacher during those lessons in English centers without being affected by national education system I wanted to investigate how teachers assess students‟ speaking skill in everyday classroom situations where time and equipment are limited Personally, I also find this topic interesting as I am also a teacher of English

in EQuest Academy, a famous English educating center Secondly, in Vietnamese schools, there are few opportunities to evaluate students' speaking skills because speaking assessment is not administered in any formal exams including the high school entrance exam and English education puts a greater emphasis on passing the entrance exam By contrast, in EQuest Acadery, students‟ speaking ability is tested regularly Thirdly, assessing speaking in a normal classroom situation has some obvious problems such as limited time and the large class of around 20 people if the situation is not recorded? Creating equal test or evaluation situations during lessons

is also challenging

II Background to the research

II.1 English education in EQuest Academy

In EQuest, an English language program is officially offered for all students at differrent levels with a variety of English courses (pre-intermediate, intermediate, upper intermediate and advanced) The curriculum has put an increasing emphasis

on enhancing students‟ oral communication skills However, there are few opportunities to evaluate students' speaking skills because speaking assessment is not administered in any formal exams including the high school entrance exam and final course examination Unlike with the curriculum focus, in practice, English education in almost school in Vietnam which puts a greater emphasis on passing the course exam EQuest Academy emphasis on enhancing oral communication skills, classroom assessment is widely practiced in EQuest classroom For many students

in EQuest, classroom assessment is the only opportunity to have their speaking skills assessed when they take tests such as the Test of Spoken English (TSE), the Test of English as a Foreign Language (TOEFL), and the International English Language Testing System (IELTS) Despite the strong recommendation of assessment by classroom teachers, there is still a challenge which stems from the emphasis for a measurement driven orientation toward assessment In other words, even if the rationale for classroom assessment is to provide both students and teachers with information about their current level of speaking skills and how to improve their speaking skills on the basis of their real language performance, teachers are nevertheless under pressure from such a policy to reflect the results of classroom assessment as a certain portion or percentage of the students‟ final courses

II.2 Characteristics of English assessment in EQuest Academy

High stakes test situation

In EQuest, most of students learn English for IELTS or TOEFL certificate to study oversea, so English assessment plays an important role in determining whether students move to the higher level This accounts for a high stakes test situation in class High stakes testing refers to tests upon which test-takers‟ futures or study plans In other words, test outcomes are likely to affect the students‟ future careers

or life directions Because of the substantial impact of such assessment on students‟ futures and careers, students and parents are greatly concerned with their grades of English in EQuest and the accurate judgment of this assessment work by their teachers Therefore, under the high pressure from students and parents, EQuest English teachers tend to develop less subjective testing in order to improve reliability and objectivity

Assessing four discrete skills

The high stakes test situation influences the traditional exam-dominated culture in Vietnamese schools For instance, in terms of the assessment format, pencil-and paper tests prevail and the style is based on multiple choice questions where the correct response is sought to each This test format may be seen as improving assessment reliability and objectivity because as Hughes (2003) points out, no judgment is required on the part of the scorer so that scoring may be objective At the same time, teachers are supposed to assess all four language skills in a balanced manner so as to satisfy the assessment guidelines as noted in the previous section Consequently, teachers tend to assess the four skills separately More specifically, conventional pencil and paper tests assessing reading comprehension including grammatical knowledge are the primary type of assessment while speaking and writing assessments which do not have any authorized assessment tools, may be conducted as part of classroom assessment

III Research aims

Despite the increasing attention paid to assessment of foreign language learners in recent years among linguistic scholars and practitioners, few studies have been found concerning teachers‟ perceptions on assessment of students‟ speaking skill, especially ways in which different teachers‟ perceptions affect how students speaking skill would be assessed The present study is to investigate what features classroom speaking assessment present in the situation where it is expected to achieve both pedagogic usefulness and meet the desired measurement purpose Also, this study explores whether the current practice of classroom assessment in EQuest Academy can be effective in terms of pedagogical benefits and speaking assessments There is still a strong tendency here towards traditional formal testing

to measure and report learning outcomes Moreover, there has been a lack of theorization of classroom assessment in the English language teaching field, with researchers pointing to much variability, a lack of systematic principles and procedures, and a reliance on traditional psychometric testing Moreover, there are few studies carried out in outside school context Thus, this study focuses on EQuest English teachers' perceptions of speaking assessment

IV Research questions

There seems no prominent research on speaking assessment from the point of view

of the teacher in Vietnam In particular, the perceptions of English-center teachers

in conducting speaking assessment remain unexplored To fulfill this concern two questions are involved

1 What are EQuest teachers‟ perceptions of the effectiveness of classroom speaking assessment?

2 What are their perceptions of the practical difficulties of assessing speaking?

3 In what ways do EQuest English teachers conduct assessment of students' speaking?

PART B: DEVELOPMENT Chapter One: Literature Review

This chapter presents the most relevant literature relating to the value of classroom assessment and draws on research concerning alternative assessment, classroom assessment, and communicative language testing The first section highlights research on the advantages of classroom speaking assessment Lastly, based on research findings, a discussion of issues relating to the reliability of classroom assessment is presented and conclusions drawn

1.1 Key term definitions

1.1.1 Test vs Assessment

In this study, the term assessment is distinguished from the term test According to Brown (2004), tests refers to prepared administrative procedures that occurs at particular times in a curriculum in which learners manifest their competence for the best achievement, knowing that their responses are being measured and evaluated Whereas used in a broader sense, assessment is an ongoing process of judgment, encompassing a teacher‟s comment and written phrase responding to students‟ performance as well as a form of reporting measurement (Brown, 2004) That is, unlike tests, the results of assessment do not need to be reported all the time, and furthermore, they are not necessarily reflected in the final grades Thus, the term assessment is used to incorporate a wide range of methods for evaluating student performance and attainment, including formal testing Specific terms (i.e., standardized tests, formal tests, or school administered tests) are used in cases where the discussion focuses on specific forms of assessment

1.1.2 Classroom speaking assessment

Since there is little research specifically into classroom speaking assessment, this study makes reference to definitions of a few terms used interchangeably: classroom based assessment, teacher assessment, and alternative assessment According to Brown (2001), classroom-based assessment encompasses both teacher-designed formal and informal assessment in the context of day-by-day interaction with students, contrary to large-scale testing initiated by the school It can be equivalent

to teacher assessment (Underhill, 1987) which refers to tests designed by individual

teachers (or a group of teachers) and administered in the process of classroom learning According to HuertaMarcías‟ (1995) claim, alternative assessment refers

to alternatives to traditional testing Thus, the characteristics of alternative assessment are summarized: (1) integrated in the curriculum; (2) provides information on the strength and weaknesses of each individual student; (3) provides multiple indices that can be used to gauge student progress; and (4) is more multiculturally sensitive and free of norms Although such terms highlight 10 different aspects of the assessment process, all tend to share common concerns: a more teacher-mediated, context-based, classroom-embedded assessment practice, explicitly or implicitly defined in opposition to traditional externally set and assessed large scale formal examinations used primarily for selection and/or accountability purposes

1.2 The advantages of classroom assessment

This section presents research findings on the advantages of classroom speaking assessment over traditional formal testing or external standardized tests in two aspects: pedagogical advantages and advantages for speaking assessment

1.2.1 Academic advantages

One major advantage of classroom assessment is that it can exploit its inherent feature, formative evaluation (Leung, 2005; SBA Consultancy Team, 2005; Davison & Leung, 2009) Bachman and Palmer (1996) suggest that the high feasibility of formative evaluation is largely due to the relatively fewer resources required for administering classroom assessment: for instance, a short vocabulary quiz and a sole teacher playing a role of an assessor According to Brown (2004), the purpose of formative assessment is to evaluate students in the process of forming their competences and skills with the goa 16 of helping them to continue that growth process Also, Davison and Leung (2009) claim that there are two key functions of formative evaluation−forming and informing−which bring benefits to the teachers as well as the learners

For the teachers, formative evaluation allows them to form more reliable judgments

in an ongoing process rather than making a quick decision based on a snapshot evaluation because here they are provided with sufficient opportunities to gather a

great deal of information about knowledge, abilities and skills of the learners during classes (Underhill, 1987; Genesee & Upshur, 1996; Brown, 2001; Harris, 2007; Fulcher & Davidson, 2007) In addition, the information from formative evaluation

is useful for teachers in planning their subsequent lessons in the teaching process, examining the effectiveness of their pedagogical objectives, and diagnosing student‟s strengths and weaknesses (Black & Wiliam, 2004)

Concerning the advantages to learners, feedback to such learners provided by formative assessment allows them to understand what they have learned and what they need to learn more (Black & Wiliam, 1998; Black, 2001; Black, Harrison, Lee, Marshall, & Wiliam, 2003; Brown 2004) In this regard, research to date has provided evidence of the importance of feedback formats Stobart (2006) explains the vice of „marks and grades only‟ feedback by citing Thorndike (1913) and other studies (Kohn, 1993; Reay and Wiliam, 1999; ARG, 2002) That is, grades can impede learning because they do not offer specific information but simply a comparison to others In line with this, Butler (1988) investigates the effect of three different forms of feedback−„comments only‟, „grades only‟, and „a combination of grades and comments‟−between two different ability groups The study identified that lower achieving students expressed most interest towards the type of

„comments only‟ feedback Byon (2005) and Gunn (1995) advance the argument of the effectiveness of criterion-referenced assessment in which the learner‟s performance is interpreted by reference to predetermined criteria According to them, unlike the summative tests primarily aimed at grading, marking criteria are informative and constructive regarding the extent to which learners can use language to achieve their communicative goals

1.2.2 Advantages for speaking assessment

In fact, little research on classroom assessment which targets oral skills has been conducted Most studies examine classroom assessment in general; for example, Dochy & McDowell (1997); Adamson & Davison (2003); Cumming & Maxwell (2004); and SBA consultant team, (2005) Thus, this section examines the advantages of classroom assessment for assessing oral skills by reference to research concerning performance assessment and communicative language tests as well as classroom assessment

First, classroom assessment is more likely to offer opportunities for teachers to directly observe linguistic performance through classroom observations or direct testing (Brown, 2001) According to the Universal Grammar approach, linguistic competence referring to the underlying linguistic abilities or knowledge of language cannot be directly observed, whereas linguistic performance―the domain of language use―can be directly observed (Mitchell & Myles, 2004) McNamara (1996) points out that the assessment of linguistic performance allows us to make direct inference of an individual‟s ability to use language appropriately or correctly

in a variety of situations Thus, a reliable and valid assessment method needs to evaluate a test-taker‟s performance rather than her/his linguistic knowledge In particular, if the target language domain of assessment is speaking, then the importance of direct testing using performance tasks is critically increased This is

in line with Brown‟s (2001) argument that performance assessment contributes to improving content validity since students actually perform the target language use Also, Moon and Callahan (2001) insist that performance assessment can promote test authenticity because it presents learners with situations or asks them to perform academic exercises that simulate real-life experiences or problems Test authenticity

in relation to performance assessment can be considered in the communicative language teaching approach In terms of Canale and Swain‟s (1980) communicative competence model, speaking by its nature needs to be judged on the basis of social context, interaction, communication, and integrated skills, all of which can be provided in the classroom context involving peers and a teacher

Third, classroom assessment can serve as a powerful motivation factor Crooks (1988) argues that tests in general have positive effects on enhancing learning motivation According to Savignon (1997), especially in relation to communicative assessment, discrepancy between the test taker‟s grammatical competence and communicative competence is manifested Lastly, classroom assessment may be beneficial for reducing any student anxiety associated with test taking Concerning the relationship between test anxiety and test results, both Phillips (1992) and Crooks (1988) provide evidence of a negative correlation between such anxiety and test achievements in the overall assessment situation Crooks (1998) concludes that

the negative influence of a learner‟s higher anxiety on achievement tends to be greater on standardized tests than in classroom assessment Underhill (1987) explains the positive function of classroom assessment in reducing test anxiety in relation to the test-taker‟s familiarity with the test environment and the assessor Namely, when assessed by the students‟ own teacher in the familiar classroom, students are more relaxed and confident and hence usually able to demonstrate better performance and proficiency than they can demonstrate in standardized tests

1.3 Issues related to reliability in classroom assessment

The concept of reliability is particularly important when considering communicative language testing (Porter, 1983) Reliability is concerned with the extent to which we can depend on the test results (Weir, 1990) Rater reliability is important to overall test reliability What raters need to do for this purpose is to achieve high inter-rater reliability for these assessments? The degree of inter-rater reliability is established

by correlating the scores obtained by candidates from rater A with those from rater

B The concern of the rater is how to enhance the agreement between raters by establishing explicit guidelines and maintaining adherence to them for the conduct

of this rating (Bachman, 1990)

Although reliability is something raters need to try to achieve in the tests, it may not

be the prime consideration all the time (Bachman, 1990) It is said that there is a reliability-validity tension Reliability offers a possible compromise It is occasionally essential to sacrifice a degree of reliability to enhance validity (Davies, 1990) For example, in certain circumstances, reliability and validity are mutually exclusive However, if a choice has to be made, validity is more important for speaking assessment (Bachman, 1990)

1.4 Issues related to validity in classroom assessment

Spolsky (1975) stated that validity is the central problem in foreign language testing Validity is concerned with whether a test measures what it is intended to measure (Weir, 1990) A test of speaking ability in a classroom setting is usually an achievement test An achievement test should have content and face validities (Davies, 1983) Since content validity asks if the test content matches the content of the course of study (Bachman, 1990), what teachers can do is to match the course

objectives and syllabus design with the test items This attitude by teachers is crucial in a classroom test because teachers may tend to use test tasks different from the course objectives especially when oral aspects are involved (Nakamura,1993) Face validity pertains to whether the test 'looks valid' to the examinees, the administrative personnel and other technically untrained observers (Bachman, 1990) Face validity is a must in a classroom speaking test, because the students' motivation is promoted for speaking if a test has good face validity (Hughes, 1989) Language testing can be put on a scientific footing through construct validity (Hughes, 1989) Bachman (1990) also highlighted that construct validity is the most fundamental validity for a speaking test Construct validity examines if the test matches a theoretical construct (Bachman, 1990) This cannot easily be handled by classroom teachers because of the abstract nature of language abilities (Nakamura, 1993)

1.5 Teachers' perceptions

Pham Lan Anh (2008) found that the teachers‟ classroom assessment practices were influenced by personal and contextual factors such as their beliefs of how children learn, constraints built into the curriculum, and institutional assessment requirements There was a complex and non-linear relationship between teaching, learning and assessment practices related to classroom assessment due

to the teachers‟ internalized conceptions of CA and contextual constraints including the educational policy Main sources of teachers‟ beliefs were their core beliefs of teaching and working with people, their in-service training and exchanged experiences from colleague The purpose, approach, procedure and focus of assessment depended on three types of assessment conducted in the three schools as specified in the official assessment document issued by on-going, periodic and final Teachers „classroom assessment practices were also shaped by the school culture and by the directives of the specialized groups Teachers‟ assessment process followed the planning, framing, implementing and using assessment results

Chapter Two: Methodology

This current study involving teachers‟ responses to the current practice of classroom speaking assessment in EQuest Academy, so I have decided to do a survey research

A survey is a study which focuses on a group‟s attitudes, opinions, and/or characteristics (Brown, 2005) To investigate the research problem, this study

addressed two research questions: (1) what are teachers’ perceptions of speaking assessment’s positive effects? (2) In what ways do EQuest English teachers conduct assessment of students' speaking? These two investigations involved a questionnaire

and an interview respectively and the sample of my survey is quite small (ten Equest teachers) and therefore qualitative method is used First, a qualitative dataset was collected from the questionnaire portraying the overall picture of classroom assessment of speaking in EQuest Second, the data from teacher interviews was collated to provide in-depth information about individual teacher‟s perception of its benefits

3.1 A qualitative approach

This study used a qualitative approach According to Wiersma (1995), qualitative research investigates the complex phenomena experienced by the participants by examining people's words and actions in descriptive ways Qualitative research uses the researcher as the data collection instrument and employs inductive analysis (Maykut and Morehouse, 1994) The researcher operates in a natural setting (Wiersma, 1995) Also, McDonough and McDonough (1997: 53) say, "qualitative research usually gathers observations, interviews, field data records, questionnaires, transcripts, and so on"

In this study, two qualitative data collection instruments were used: 'a questionnaire' and 'semi-structured interviews' with teachers

According to Maykut and Morehouse (1994), questionnaire research is popular among educational researchers in general and ELT research in particular McDonough and McDonough (1997: 171-172) state the advantages of questionnaires as follows:

The knowledge needed is controlled by the questions, therefore it affords a good deal of precision and clarity

* Questionnaire can be used on a small scale, in-house and on a large scale, requiring little more extra effort than photocopying and postage

* Data can be gathered in several different time slots: all at once in a class, in the respondents' own time as long as it is easy to return, at convenience when a suitable

respondent happens to come along, and in different locations at different times; but in all

of these the data is comparable, the questions are the same and the format is identical

* Self-completion questionnaires allow access to outside contexts so information can be gathered from colleagues in other schools and even other countries

Semi-structured interviews were conducted with some of the participants because

"the interview is a very good way of accessing peoples' perceptions" (Punch, 1998: 174) As well, the interview was considered a method of triangulation, a "checking out the consistency" (Patton, 1990: 464) of the data obtained from the questionnaire responses Further, it was believed that such triangulation of data may yield factors not mentioned by the participants in the questionnaire (Punch, 1998) McDonough and McDonough (1997: 184) remark 'a semi-structured interview' is regarded as

"being closer to the qualitative paradigm because it allows for richer interaction and more personalized responses"

3.2 Steps in the selection of participants

3.2.1 Questionnaire participants

A questionnaire (Appendix A) was administered to ten English teachers who were working at EQuest Academy Two males and eight females responded to the questionnaire Table 1 shows that the participants ranged in age from 24 to 31 years, with the majority in their 20s The participants' experience in teaching English ranged from one to 6 years All participants had experience in conducting speaking assessment in EQuest

Table 1: Background of Questionnaire Participants

Teachers Age Gender

Years of teaching experience

Level taught

Highest degree completed Teacher 1 24 F 1 Intermediate BA Teacher 2 24 M 1 Pre Intermediate BA

Questionnaire

The questionnaire consisted of two sections The first section had questions about teacher profiles including gender, age, and length of teaching experience In addition, the size of the class they taught was included

The second section contained the two subsidiary questions and built on the information gathered in the first section above: what the primary purposes of classroom speaking assessments are and what the assessment practices in terms of assessment tasks, frequency, and teacher feedback are There were three types of questions used here: open-ended question, Likert scale and questions with fixed alternatives Two questions about current assessment purposes and the importance

of assessment purposes as perceived by the teachers were asked: (Q3) indicate in effect for what purpose you employ classroom speaking assessment and (Q4) indicate the importance of the purpose of classroom speaking assessment Teachers were encouraged to indicate one or more among the seven options in response to Q3 and to evaluate the importance of the assessment purposes as they perceived on a scale 1 to 5 in response to Q4

In research question about assessment practices, the three survey questions included assessment methods, specific speaking task formats, frequency, and format of teacher feedback as follows:

(Q5) Which are the primary methods of classroom speaking assessment? (Q6) Please indicate the kinds of tasks/activities that you use and (Q7) Please choose one primary feedback that you would use below

Because these were closed questions, teachers were asked to choose options

The questionnaire took approximately fifteen minutes to complete Teachers were asked to complete the survey with their own nicknames to identify themselves in case of withdrawal The questionnaire was piloted by two qualified English teachers prior to the main study The full questionnaire is provided in Appendix A

3.2.2 Interview informants

Among those who indicated their willingness to participate in the interviews, teachers actually participated in these follow-up interviews, and of these, all ten teachers were ultimately selected for interview by following "Patton's maximum

variation sampling" (cited in Lincoln & Guba, 1985: 200) The researcher considered maximum variation in participants' age, gender, teaching experience, teaching setting, and taught It was decided that teachers must be represented in the group of interview informants and as well Two other parameters, informants' age and years of teaching, were also included to ensure as much variety as possible In this way, ten original participants were selected for interview (See Table 2)

Table 2: Background of Interview Informants

Teachers Age Gender

Years of teaching experience

Level taught

Highest degree completed Teacher 1 24 F 1 Intermediate BA Teacher 2 24 M 1 Pre Intermediate BA

3.3 Data collection procedures

The research was carried out using a questionnaire and interview method Firstly, in an attempt to develop an appropriate survey instrument for this study, a pilot questionnaire was administered to ten English teachers who were attending a workshop for English teachers in EQuest The pilot questionnaire served to identify those items which were unclear, repetitive, and unnecessary The final version of the questionnaire (Appendix A) included open-ended question, Likert scale and questions with fixed alternatives generated from the data collected in the pilot survey It asked for opinions on speaking assessment as well as the teachers' professional, educational, and personal background The questionnaire was written

in English

The responses to that questionnaire were gathered from ten EQuest English teachers After they had received an explanatory statement and signed consent forms, it was explained that their participation in the project was entirely voluntary After analysis of the questionnaire responses, these participants were chosen for interviews on the basis of maximum variation in age, gender, teaching experience, teaching setting, and grades taught These ten were invited to be interviewed so that their perceptions of speaking assessment could be further explored All interviews were conducted at the quiet restaurant and each interview lasted approximately 10 minutes

While all ten interviews basically followed the same format and questions prepared prior to the interview, the researcher attempted to maintain the flow of natural conversation if the interviewee did not request any additional explanation on a question All the interviews were conducted in English Each interview lasted about ten minutes and was audio-taped

3.4 Data analysis

Data analysis is not a simple description of the data collected but a process by which the researcher can bring interpretation to the data (Powney & Watts, 1987) Wiersma (1995: 216) suggests that Qualitative data analysis requires organization

of information and data reduction Thus, the researcher is required to reorganize and select related information from disordered, unorganized and discursive data

After all, analysis in qualitative research is a process of successive approximation toward an accurate description and interpretation of the phenomena (Wiersma, 1995: 216) The themes and coding categories in this study emerged from a thorough examination of the data They were not determined beforehand and not imposed on the data (Bogdan and Biklen, 1992) The researcher repeatedly read through the completed questionnaires and the interview transcripts As a consequence, recurrent themes and salient comments were identified based on the ideas provided by the participants and interview informers In this process content analysis was performed by first listing the range of responses by the participants, and then grouping common features and recurrent themes

In terms of questionnaire, the responses to Q3 and Q4 were analyzed: the frequencies of responses to Q3 were calculated, and in Q4 teachers were encouraged to indicate the importance of their perceived purposes of assessment by choosing one rating from 1 to 5 These two questions answer for “what is the primary purpose of classroom speaking assessment?”

The response from the Q5, Q6, and Q7 were analyzed in terms of assessment methods, frequency, and teacher feedback To explore the nature (and quality) of assessment methods, three aspects were considered: (1) the methods of classroom speaking assessment, (2) specific speaking tasks, and (3) the format of grouping students Responses to Q5 were classified into seven categories to identify the methods of classroom speaking assessment Three categories indicated choosing a sole method such as marking specific speaking tasks, observation, and peer assessment The other four categories were a combination of two or more methods: for instance, marking specific speaking tasks and observation; marking specific speaking tasks, observation, and peer assessment; marking specific speaking tasks, observation and self-assessment; and all four methods

The interview‟s purpose was to provide an answer to research question The interview data was transcribed and analyzed according to the interview questions under the three themes: (1) background information on the length of teaching experience, teaching approach, and assessment practices, (2) teacher‟s perception of its effectiveness, and (3) difficulties in assessment administration First the

background information on the ten teachers‟ teaching experiences, teaching approaches, assessment methods, and a focused domain of assessment were summarized in a table Second, teachers‟ perceptions were categorized according to their attitudes: positive, negative, and neutral For example, teachers who have positive attitudes express the desire to keep conducting classroom speaking assessment in spite of some challenges, whereas those with negative attitudes did not want to implement it Teachers who were not fully convinced of its useful role

or positive effects even if they acknowledged its effectiveness to a degree were classified in the neutral group Lastly, teachers‟ comments on the difficulties in assessment administration were analyzed along with the teachers‟ responses to the closed question (Q2) about the reasons that they do not conduct classroom assessment

Chapter Three Findings and Discussions

This chapter presents the findings from the questionnaire and semi-structured interviews, which were conducted as described in the previous chapter and discussions

3.1 Findings

3.1.1 Teachers' perceptions of speaking assessment in EQuest

3.1.1.1 Teachers' perceptions of the effectiveness of speaking assessment

The results of the analyses of the interview data are reported in this section The two questions asked relating to teachers‟ perceptions of effectiveness of classroom speaking assessment were: what they thought the role of classroom speaking assessment was, and what positive effects of classroom speaking assessment they perceived on teaching and learning were 2 teachers had approximately average 4 year teaching experience in EQuest, and the main teaching approach was a grammar-translation method They used the responsive, the extensive (monologue), and the interactive type of speaking tasks as similarly as the frequently used task formats revealed from the questionnaire survey Further details are provided in Appendix

3.1.1.2 Role of classroom speaking assessment

Eight of ten teachers showed positive attitudes towards classroom speaking assessment as indicated by the comments involving benefits for the students‟ confidence, learning motivation, class participation and so on:

Assessment is an effective tool to encourage students to learn especially in a short term Without assessment, it would be difficult to get students eager to involve speaking activities (Teacher 2)

Teacher 2‟s belief that assessment can encourage students‟ participation during the regular class time was in accordance with her assessment practices Namely, she commented that her only assessment criterion was a student‟s attempt to use English Thus, she gave marks if a student participated in class activities Teacher 3 said that classroom assessment may contribute to reducing students‟ inhibition caused by their low proficiency She maintained that students can have more

opportunities not only to practice oral skills while preparing the assessment but also

to speak out in front of many people She expected that eventually, her students could somewhat build up their confidence

However, the other two teachers had a negative outlook regarding the role of classroom speaking assessment, in particular, that the current speaking assessment methods hardly allow teachers to assess students‟ genuine speaking competence Teacher 10 made the strongest claim of all as shown below:

I think that the current classroom speaking assessment does not play any important role to facilitate speaking or to motivate learning Currently, it just serves to mark students’ instant memorizing abilities and to report the outcomes Moreover, it creates a great deal of assessment stress to students.(Teacher 10)

3.1.2 Positive effects of classroom speaking assessment on teaching and learning

Only two teachers mentioned any positive effects of classroom speaking assessment on their teaching, the other four teachers not perceiving any evident effects The first two teachers commented that classroom speaking assessment was beneficial for speaking instructions and organizing the next teaching plan That is, Teacher 2 said that she employed a greater variety of communicative activities apart from the structured curriculum based on the textbook, in order to conduct speaking assessment

The reason given was that assessment should be in line with what students have already learned during classes Also, Teacher 5 stated that she used general feedback about students‟ weaknesses provided by the classroom speaking assessment to set up her next teaching plan In terms of learning, teachers‟ attitudes can be divided into three categories: positive, neutral, and negative The majority of interviewees, four teachers, had a somewhat neutral position, and the rest of two teachers revealed respectively positive and negative opinions about its effectiveness for learning Only Teacher 3 perceived positive effects on students‟ motivation towards learning, stating “classroom speaking assessment may increase students‟ recognition of how important speaking is in learning English Although