A survey on the effectiveness of teaching english for non majors at an giang university m a thesis

ABSTRACT On consideration of the importance of evaluation to improvements in teaching, this study was an attempt to make an evaluation of the effectiveness of the English program for non

VIETNAM NATIONAL UNIVERSITY – HO CHI MINH CITY

UNIVERSITY OF SOCIAL SCIENCES AND HUMANITIES

LAM THI THUY TIEN

A SURVEY ON THE EFFECTIVENESS OF TEACHING ENGLISH FOR NON-MAJORS AT

DECLARATION

I declare that this thesis entitled “A survey on the effectiveness of teaching English for non-majors at An Giang University” is the result of my own research except as cited in the reference The thesis has not been accepted for any degree and is not concurrently submitted for any other degree

Signature:

Name: Lam Thi Thuy Tien Date: March, 2009

ACKNOWLEDGEMENT

This M.A thesis has been completed thanks to the help of many people from whom I have received support I would like to express

my deep attitude to all of them

First of all, I would like to express my deepest gratitude to Dr Vu Thi Phuong Anh, my supervisor, for her support and guidance throughout the study

My special thanks also go to my colleagues at An Giang University from whom I have received a lot of advice and encouragement The thesis could not have been possible without the participation of the students at An Giang University I am thankful to their participation in the study

Finally, my special thanks are given to my family, especially my husband, who have shared my difficulties with love, care and tolerance

ABSTRACT

On consideration of the importance of evaluation to improvements in teaching, this study was an attempt to make an evaluation of the effectiveness of the English program for non-majors at An Giang University (AGU) It was also aimed at finding what affects the effectiveness of the program, both positively and negatively From the findings, recommendations will be made to enhance the positive factors, and to exclude the negative ones

The study was a descriptive research The sample was 116 students and 7 teachers at AGU The instruments for data collection included a test, structured interviews and questionnaires

After the data analysis process, it can be concluded that the English program at AGU is not as effective as expected The factors that have good effects on the program are students’ self study, teaching and testing The factors that have opposite influences

on the program are infrastructure, administration and lack of language environment To get higher quality in teaching, great efforts have to be made to change the current situation at AGU by strengthening the positive factors and reducing the negative ones Suggestions for improvement were consequently put forward in the study

The usefulness of language learning does not only exist in the field of economy With the existence of the Internet, learning foreign languages is considered an important tool to access the huge source of information there Mastering foreign languages, therefore, is also necessary for anyone who always seeks for new and up-to-date knowledge The role of language learning is really remarkable, that creates the popularity of foreign language learning in Vietnam

Amongst the foreign languages to be learned in Vietnam, English is most chosen because of its widely international uses English has soon been one of the required language courses in the national syllabus for all levels of education Besides learning English as a required subject in school, people rush to learn English in an increasing number of language schools or centers throughout the country There have not been any official statistics on the number of people learning English in Vietnam, but it is believed that the number might be in great quantity

While the quantity of language schools increases rapidly, the quality of teaching

or the teaching effectiveness in those schools is still not under control This situation occurs in both state and private schools In October, 2006, Tuoi Tre newspaper raised a series of articles that attracted a lot of readers: “Vietnamese students cannot

communicate in English after years studying the language.” This revealed two urgent problems that have existed for a long time: the English teaching is not as effective as expected and the effectiveness or the outcome of the teaching does not receive enough care as it should do

Evaluation of teaching effectiveness is carried out as a required activity in schools in the world It is required because it is an extremely essential step to ensure their quality of education People make evaluation in order to figure out positive and negative factors affecting their teaching Then, they develop conscious policies of their teaching improvement by strengthening the positive factors and improving the negative ones Therefore, evaluation is necessary for schools on the way to get perfect teaching quality

While no one can deny the importance of evaluation in education, few schools

in Vietnam care of carrying out evaluation in the process of their teaching This is an unacceptable contrary It is the time for this contrary to be changed Every school should have its teaching evaluated in order to get better teaching effectiveness Once the evaluation is carried out regularly, once the strength and the weakness in teaching are figured out, and considerable improvement is built, the education in Vietnam will

be able to keep pace with that of the other countries in the world

An Giang University (AGU) has been established in 2001 Through its eight years of existence, it has been training a number of students with different disciplines Nearly all the students take the program of English for non-majors and the program of ESP With a big number of the students to be trained, there have not been any evaluations of the outcome of its training, which shows the effectiveness of the English teaching in the school As mentioned above, it is the time for AGU to have a throughout look on the quality of its training in general and its English teaching in particular There should be regular evaluations on the English teaching In this study, it

is not the researcher’s ambition to cover the evaluation of both of the two programs of English, because the evaluation of the ESP program needs further study before being carried out With a limited scope of a master thesis, this study only deals with the effectiveness of teaching English for non-majors (the English teaching, in short) at AGU

To make that evaluation does not simply mean to judge how good the English teaching for non-majors at AGU is Evaluations always go with improvements Hence, there are a lot of things to be done in the process of the evaluation The first thing to be considered is to determine how effective the English teaching is in comparison to what

is expected to get In addition, there must be an analysis of factors affecting the teaching Finally, there will be some policies for improvements The three ideas just mentioned serve as the basis for giving the research questions for the present study

2 Research questions

The present study is aimed at evaluating the English teaching for non-majors at AGU Therefore, it is included some issues of an evaluation process as discussed in the previous part The first issue is to access the quality of the English teaching in comparison to the objectives of effectiveness announced For the next issue, there is an analysis of positive as well as negative factors affecting the English teaching With the results from the analysis, some recommendations for improvement will be given Totally, the study has three specific objectives: effectiveness assessment, analysis of factors affecting the teaching and recommendations for effectiveness improvement

To achieve these objectives, the study is intended to seek answers to the following questions:

Research question 1: How far has the program of English for non-majors met the

stated objectives?

Research question 2: What factors affect the effectiveness of the program, both

positively and negatively?

Research question 3: How can the effectiveness of the English program at AGU be

improved?

3 Organization of the study

Apart from the Acknowledgement, Abstract, and Reference, the study is composed of five chapters: Introduction, Literature Review, Methodology, Results and Discussion

The first chapter, namely Introduction, sets the scene for the evaluation in the

study This chapter states the reasons for choosing the topic and carrying out the thesis

It also defines the objectives as well as the scope and the organization of the thesis This chapter provides an overview of the problem studied in the thesis

The second chapter is the Literature Review This chapter includes two main

sections The first section presents some terms and concepts related to the study The following section gives a short description of prior studies on the topic This chapter provides the theoretical background for this study

Chapter three, entitled Methodology, presents how the research is carried out

The chapter begins with information of the research design Then it goes with the most important issue of this study, the procedure for carrying out the study, including the following sections: Description of Population and Sampling, Instruments for Data Collection, and Data Collection and Analysis

Chapter four, Results, reports all the results obtained from the analysis of the

data collected Then, interpretation of the results is given, which serves as the basis for the conclusion in the last chapter

The last chapter, Discussion, is constituted by three sections The opening

section is for the most important issue: the discussion of the research questions The next section presents some limitations of the study The last section provides some suggestions for problems that require further research beyond the limit of the study

Chapter 2

LITERATURE REVIEW

Chapter two is devoted to the background theory concerning the evaluation of teaching effectiveness This chapter is constituted by two sections It begins with the section presenting some terms and concepts related to the study The next section is a short review of some prior studies on the topic chosen to be studied in this thesis

1 Terms and concepts related to the study

The thesis deals with the evaluation of teaching effectiveness Therefore, there are two basic concepts to be clarified before going on: evaluation and effective teaching or effective schooling The following is the definitions of the two concepts with other related issues

1.1 Evaluation

1.1.1 Evaluation in social life

According to the Oxford Advanced Learner’s Dictionary, the word “evaluation”

itself means to access or form an idea of the amount, quality or value of something With that definition, people are more or less engaged in the evaluation of many aspects

of their daily life such as weather, food, price or behavior of someone…Evaluation then as Rea-Dickins and Germaine stated “is a feature of social life.” However, evaluation in such a context is just a natural activity which may be undertaken spontaneously and unconsciously without any purposes That kind of evaluation has no criteria or the criteria “may be vague or ill-defined” (Rea-Dickins and Germaine, 1992)

1.1.2 Evaluation in education

As mentioned above, evaluative judgments in our daily life are often made unconsciously and maybe no decisions are made after that evaluation However, evaluation in education is a systematic and scientific process with well-defined criteria that can help people have some decision-making In fact, in education evaluation, an evaluation of a teacher performance may affect his/her promotion; a text book evaluation may help people decide if it can continue to be used An evaluation in education may have different effects on the issues being accessed, and those effects almost always lead to some improvement for education

1.1.2.1 Evaluation and decision making

There are a lot of definitions of evaluation from different authors Almost all of

them share the idea of the relation between evaluation and decision-making The Longman Dictionary of Language Teaching and Applied Linguistics defines evaluation

in general as “the systematic gathering of information for purposes of making.” Tankard (1974) observed that “in recent years, there has been a developing definition of evaluation which views evaluation information as vital to the decision-making.” More specifically, Gay and Airasian (2000) declared that evaluation is concerned with making decisions about the quality, effectiveness, or value of educational programs It can be said that in modern education, evaluation is considered extremely essential for improvement as discussed later in the next part

decision-1.1.2.2 Evaluation and improvement

Evaluation is often equated with the consideration of what is good and what is bad After that consideration, the good is appraised, the bad is excluded Hence, evaluation is often disliked and it becomes a “sensitive” activity to anyone who does not want to show their weakness This problem was noted by Rea-Dickins and

Germaine (1992) that “evaluation question preconceived assumptions, sometimes to confirm them but also to dismantle them This is one reason why evaluation is often disliked or seen as suspect, and why there are sometimes difficulties for the evaluators themselves Most societies and institutions demonstrate dynamic conservatism (Everard and Morris, 1990), they actively resist change.” Such a problem does exist in

a number of schools in Vietnam In those schools, to some teachers, making evaluation means revealing their weakness and the evaluation results may affect their position and their benefits For that reasons, they do not want evaluation Their teaching remains unchanged even if it is effective or not As a result, the quality of their teaching is still

it should result from an evaluation of some kind.” Rea-Dickins and Germaine also gave the reason for evaluation in the figure below

Yes Confirm

Evaluation Is practice OK?

No Innovate

Figure 1.Reasons for evaluation

From the figure, it is clear that improvement or innovation is the destination of

an evaluation Thus, the teachers should not consider evaluation as a serious problem

that can affect their promotion Instead, they should consider evaluation as feedback for better teaching

To sum up, different authors may have different definitions of evaluation Nevertheless, there is one common assumption amongst them That is the involvement

of evaluation with decision-making and improvement In teaching, evaluation is an analysis of information related to the teaching From that analysis, both factors that affect the teaching positively and those that affect the program negatively are revealed Then some decisions are made for better teaching or for teaching improvement Specifically, those decisions are made on the enhancement of the positive factors and the exclusion or changes of the negative ones

1.1.3 Types of evaluation

In the literature of evaluation research, evaluations have 2 main types: formative and summative Each type includes several subdivided types according to the evaluators’ purposes This part begins with the definitions of the two main types of evaluation, and then continues with more details of the subdivided types

1.1.3.1 Formative evaluation

The first main type of evaluation is defined by a number of different authors However, their definitions agree in two features First, formative evaluation is carried out during the development of the curriculum Second, the purpose of formative

evaluation is for the curriculum improvement As defined in the Longman Dictionary

of Language Teaching and Applied Linguistics, formative evaluation is considered as

the process of providing information to curriculum developers during the development

of a curriculum or program, in order to improve it Not only sharing the idea but also sharing how formative evaluation improves the curriculum, Rea-Dickins and Germaine stated that “Evaluation intended to improve the curriculum will gather information from different people over a period of time This is known as formative evaluation Such evaluations are ongoing and monitor developments by identifying the strength and weakness of all aspects of teaching and learning … formative evaluation is designed to provide information that may be used as the basis for future planning and action It is formative since it aims to strengthen and improve the curriculum.” Formative evaluation provides feedback for what is happening in the education so that

people can improve the reality for better education The role of providing feedback of formative evaluation was remarked by Anderson (1990) that “Sometimes the major purpose of evaluation is to improve a new program or activity This is generally referred to as formative evaluation as its results are intended to feed back and improve ongoing practice ” Formative evaluation is necessary for the development of teaching, since it provides useful information for the improvement of teaching

1.1.3.2 Summative evaluation

Unlike formative evaluations, summative evaluations pay much attention to the end products of the instructional process As a result, summative evaluation is carried out at the end of a teaching program and the results from the evaluation affect the existence of the program Anderson (1990) held the view that “summative evaluation

is directed at evaluating the consequence of a program and is often seen as the principal mechanism for making crucial resource allocation decisions about a program, including its continuance or termination.” Summative evaluation leads to the continuance or termination of a teaching program because it is “the process of providing information to decision makers, after the program is completed, about whether or not the program was effective and successful.” (Richards, Platt, J., and Platt, H., 1992) In short, summative evaluation is intended to measure the success of

an instructional program by examining its effectiveness, and from that examination, the program will be decided to continue or not

1.1.3.3 Subdivided types of evaluation

The two main types of evaluation are subdivided into many specific types of evaluation Different authors have different categorization of evaluation types Each type is directed to a specific purpose of evaluation Thus, according to the goals of evaluation, evaluators choose the most suitable type for their evaluation

Among the subdivision of evaluation types, the author of this study prefers the one provided by Tankard, Jr (1994) His subdivision is well-defined and it can cover nearly almost the common purposes of evaluation in education He uses the term

“evaluation strategies” instead of “evaluation types” He pointed out that “there are four education decisions that need to be considered in accessing an instructional program” and accompanied with the four decisions, there are also four kinds of

evaluation strategies The definitions of the decisions and the accompanied strategies are followed

a Four educational decisions

1 Planning decisions are those which focus on needed improvement by specifying the

area, major goals, and specific objectives to be served

2 Programming decisions are those which specify procedures, personnel, facilities,

budget, and time requirements for implementing planned activities

3 Implementing decisions are those in directing programmed activities

4 Program improvement (recycling) decisions are those including terminating,

continuing, evolving or modifying activities

b Four evaluation strategies

1 Need and feasibility evaluation consists of defining the environment where change

is to occur, the unmet needs, problems underlying those needs, and opportunities for change

2 Input evaluation consists of identifying and assessing relevant capabilities, strategies

which may be appropriate for meeting program goals, and designs which may be appropriate for achieving specific objectives

3 Process evaluation consists of detecting or predicting defects in the procedural

design or its implementation during the implementation stages

4 Product evaluation consists of determining the effectiveness of the project after it

has run full cycle by measuring and interpreting outcomes as they are related to need and feasibility, input, and process

According to the definitions above, this thesis belongs to the third kind of evaluation in the following reasons This evaluation research is aimed at evaluating the effectiveness of the program of teaching English for non-majors However, the underlined purpose is to discover the defects in the implementation of the program and then to improve it to reach better quality in teaching In other words, the research has the goal of making some implementing decisions that can direct the program to a more effective instruction As the result, the strategy needed to carry out the study should be the Process evaluation as suggested by Tankard

1.2 Effective teaching

The second interest for this study is that of effective teaching In the literature, the effective teaching is also mentioned as quality teaching, or school effectiveness For this term, there are two kinds of definitions to be considered, conceptual and operational definitions, in order to have full understanding of the term and to measure

it exactly

Before going on, it is necessary to understand what “operational definition” is

According to Six Sigma SPC's Quality Control Dictionary and Glossary, an

operational definition is “a very clear and very precise explanation of the items being measured or the terms that are used to ensure comprehensible knowledge of the terminology and the ability to operate a process, procedure, or service and/or collect data consistently and reliably.” From that view, it is clear that having the operational

definition of the item being measured is crucial to ensure the consistency and reliability of the data collection, and therefore to have an exact measurement of the item

Thus, this section is devoted to the two definitions of the term “school effectiveness”, the main item to be measured in this thesis

1.2.1 Conceptual definitions of effective teaching

The second interest for this study is that of effective teaching Amongst the definitions available in the literature, the definition given in the Longman Dictionary

of Language Teaching and Applied Linguistics is the most specific, that proposes rather enough characteristics of what to be considered as effective teaching

Effective teaching generally refers to teaching which has the following characteristics

1 Expectations for learning Students are expected to learn and class activities relate to learning

2 Student behavior Student behavior and conduct is monitored

3 Class routines and procedures Class administrative routines are handled efficiently

4 Standard Students, teachers and parents agree on standards

5 Grouping Grouping arrangements match instructional needs

6 Objectives Objectives are used to focus learning

7 Instruction and direction Teachers give clear directions and all students are involved in instruction

8 Learning time Class time is used for learning

9 Reteaching When student do not learn, they are retaught

10 Teacher student Interactions Teacher-Student interactions are positive

11 Student rewards and incentives Incentives and rewards are set for learning

From the definition, it can be said that there are three main factors involved in the effective teaching: learning, teaching and administering They mutually affect to get the most effectiveness in teaching Learners cannot learn well without good teaching from teachers and good conditions provided by the administrators Teachers cannot teach well without students’ regular self-study as well as good administration Moreover, only good conditions for teaching and learning provided by the administrators cannot make the teaching effective without the efforts of both teachers and students To sum up, the teaching cannot get the most effectiveness if there is an absence of any of the factors

1.2.2 Operational definitions of effective teaching

In the history of teaching effectiveness or school effectiveness research, there has been the appearance of several operational definitions for school effectiveness Teddlie and Reynolds (2000) studied and summarized 6 operational definitions of school effectiveness, and put them in the order of their appearance in time Amongst the six definitions, the author of this study prefers the fifth definition, and uses it as the tool to operate the evaluation of teaching effectiveness at AGU in the present study

The definition defines school effects as measuring the unique effect of each school (in a system) on their students’ outcomes

School effectiveness as defined in this definition is considered as “the difference between a school’s average level of performance and some standard, after adjusting for the intake characteristics of the school.” In the other words, the teaching effectiveness of a school can be measured by comparing the average level of the students in that school with the standard or the level of performance that the students are expected to reach in a program, in regard to the background of the students

Following the definition as a guidance to operate the evaluation of how effective the English teaching effectiveness at AGU is, the thesis tries to figure out the present English level of the students, and then consider if it matches the announced objective level of the program Finally, there is a conclusion on the effectiveness of the program of English for non-majors at AGU

1.3 Principles for effective teaching

Beside studying the effectiveness of the English teaching at AGU, the thesis also aims at looking for ways to improve the current situation of English teaching at AGU To have appropriate suggestions for improvements, it is necessary to understand what should be done to have effectiveness in teaching; or in other words, to understand principles or requirements for effective teaching

Surely, effective teaching is the demand of any educational institution To reach the effectiveness in teaching, there should be the so-called “standards” for them to refer to The standards provide requirements or principles for the teaching to follow to

be effective In the literature, a lot of requirements for effective teaching are proposed The review of literature for this issue begins with the widely utilized “Seven Principles for Good Practice in Undergraduate Education” argued by Chickering and Gamson in 1980s (cited by Achatemeier, Morris and Finnegan) With the replication research in the literature the understanding of the principles becomes deeper and broader The next part of this section discusses the 7 correlates or principles of effective school, which are considered deeper and broader than those in the early 1980s Day by day, when quality teaching gets better and better care, the principles are specified into detailed requirements that can be easily understood and carried out The last part of this section cites the top ten requirements for good teaching declared by Prof Leblanc

1.3.1 Seven principles for good practice in undergraduate education

These principles were compiled in a study supported by the American Association

of Higher Education (AAHE), the Education Commission of the States, and the Johnson Foundation In this 1987 AAHE classic, Chickering and Gamson argue from their research that good practice in teaching and learning must do the following

- Encourage student-faculty contact,

- Encourage cooperation among students,

- Encourage active learning,

- Give prompt feedback,

- Emphasize time on task,

- Communicate high expectations, and

- Respect diverse talents and ways of learning

From these principles, to get effectiveness in teaching, much emphasis is put on teachers and students However, nowadays, when the need for quality teaching gets higher and higher, teaching needs to be supported by many more factors to get most effectiveness such as administration, environment, clear mission, etc… These factors are mentioned in the correlates of effective schools provided by the Association for Effective School, Inc

1.3.2 Correlates of effective schools

The correlates of effective schools are defined as follows

- Clear school mission

- High expectation for success

- Instructional leadership

- Frequent monitoring of student progress

- Opportunity to learn and student on task

- Safe and orderly environment

- Home-school relations

Most of the requirements for quality teaching pay much attention to the teaching, the learning and the administration However, these seven correlates of effective schools reveal one important factor, the environment The role of environment is even more significant to language teaching and learning Today, with the philosophy of teaching languages for authentic communication, there should be an environment available for learners to use the language That environment can be English speaking clubs, camps or even some courses taught in English The students really need an environment because even if all the practices are good, the language cannot stay in the learners long without having an environment to use it authentically

1.3.3 Top ten requirements of good teaching

Not providing enough factors of effective schooling as the previous part, however, Prof Leblanc pointed out the top ten requirements of good teaching that are detailed and match the reality of teaching The requirement can be summarized as following

- Good teaching is as much about passion as it is about reason

- Good teaching is about substance and treating students as consumers of knowledge Good teaching is also about bridging the gap between theory and practice

- Good teaching is about listening, questioning, being responsive, and remembering that each student and class is different

- Good teaching is about not always having a fixed agenda and being rigid, but flexible, fluid, experimenting, and having the confidence to react and adjust to changing circumstances

- Good teaching is also about style Good teachers are the conductors and the class is the orchestra Good teaching is about humor Students learn in a more relaxing atmosphere

- Good teaching is about caring, nurturing, and developing minds and talents It’s about devoting time, often invisible, to every student It is also about thankless hours of grading, designing or redesigning courses, and preparing materials to still further enhance instruction

- Good teaching is supported by strong and visionary leadership, and very tangible institutional support – resources, personnel, and funds

- Good teaching is about mentoring between senior and junior faculty, teamwork, and being recognized and prompted by one’s peers

- Good teaching is about having fun, experiencing pleasure and intrinsic rewards Good teachers practice their craft not for the money or because they have to, but because they truly enjoy it and because they want to

With the ten requirements, Leblanc leaves a heavy duty on the teachers who always seek for effectiveness in their teaching All of the requirements are reasonable and should be followed in the reality of teaching However, the teaching in Vietnam nowadays does not meet some of those requirements Firstly, the teaching in most of the schools in Vietnam does not treat the students as consumers of knowledge If the

students are considered as customers of the knowledge, the teaching, the service provider has to serve what his/her customers need Nevertheless, there are seldom any student need surveys Thus, the students have to learn what they do not know if it is necessary for them Second, good teaching is also about bridging the gap between theory and practice The English teaching in Vietnam has not fulfilled this job Most of the teaching and learning English take place inside the classrooms Nearly almost schools fail to make the knowledge of English stay in the students long outside the classrooms They seldom organize activities in which students can use the knowledge they have mastered in class Day by day, unconsciously, that knowledge goes out of the students’ mind Third, good teaching is about offering efforts to design courses and prepare materials In contrast, not all schools have enough experts in course design, so they often use materials available in the market without any evaluation of their suitability As a result, generations of students are in the same course design, and use the same materials There is a lack of keen consideration on their effects Teaching in Vietnam in general and teaching English in particular need to devote much time and effort to meet all of the requirements

2 Prior studies related to English teaching effectiveness evaluation

In the literature, there are few studies related to evaluating a certain program available In 2006, Tuoi Tre Newspaper made a survey on the communicative competence of students in university This survey had a disappointing result when it revealed that not only non-English majored students could not speak English, but the majors also could not use their English knowledge to communicate appropriately Sooner than that survey, in 2004, an evaluation of the teaching of English as a foreign language in universities in Ho Chi Minh City was carried out at Vietnam National University (Vu Thi Phuong Anh, 2004) The findings of this research were also undesirable The effectiveness of teaching English in university was not as high as it was expected The study also showed reasons accounting for the poor quality of English teaching in universities in Ho Chi Minh City

The present study shares the same interest with the two studies mentioned about However, it is not the author’s ambition to cover a large field like the mentioned studies This study has different questions to be investigated with different

methodology, and its findings will be significant first of all for her teaching and the teaching in her institution

Chapter 3

METHODOLOGY

This chapter presents all aspects related to the methodology of carrying out the study It includes 2 main parts The first part introduces the description of the research design The second part depicts the 3 stages of the procedure to carry out the research namely: Description of population and sample, Instruments for data collection, and Data collection and analysis

1 Research design

1.1 Approaches

In the field of evaluation research, a number of evaluation methods and approaches are provided Evaluation researchers choose the one that serves best their purposes of evaluating Thus, to choose the most suitable approach to this study, it is important to have a back look at the goals of the study This study is aimed at investigating the general situation of the English program for non-majors, not any specific aspects of the program Besides, it investigates all possible factors that can affect the teaching Furthermore, from the results of the investigation, these will be suggestions for better teaching quality With such a philosophy, the researcher of this study prefers to use the Logical Approach in her evaluating process Before getting into the reasons of such a choice, there is a description of what Logical Approach is

Logical Approach

The Logical Approach is developed by the Office of the Controller General (OCG, 1981) and is used for mandatory evaluations of all Federal programs in Canada The OCG approach then was adapted by Anderson (1990) and his colleagues The adapted approach suggests four major issues to be addressed in program and project evaluation

a Program rationale: the consideration of whether or not a program makes sense

b Program effectiveness: the consideration of the extent to which the program has achieved its objectives

c Program efficiency: the consideration of whether or not the program is in the most efficient way even with the existence of objective achievement

d Program effects and impacts: the consideration of the lasting contribution of the program

Reason of the choice

The main purpose of this research focuses on considering whether or not the English teaching for non –majors at AGU is effective With this purpose, it seems that the study only focuses on the second issues, program effectiveness mentioned in the OCG approach Nevertheless, all of the four issues of the OCG approach are included

in the evaluating process for the study to be sufficient Actually, an evaluation of the effectiveness of a teaching program cannot only insist on the consideration of the objective achievement without examining the necessity and significance of the program Therefore, the program rationale should be addressed Moreover, even when the teaching program gets its stated objectives, there should be inspection of whether the program is carried in the best way Maybe, the program can also get the same effectiveness, however, in another way which is more efficient Thus, the program efficiency is another important issue to be considered In addition, the consideration of the lasting contribution of the program is also really essential At the end of the program, students get its objectives of learning However, outside the classrooms, language students cannot use the knowledge they have learned in class, or after a certain time, they forget the knowledge There is the need of having an environment for students to consolidate their language

In short, to have a scientific survey of the effectiveness of a teaching program in general and a language program in particular, it is essential for all of the four issues of the Logical Approach to be addressed in the survey research As the result, the Logical Approach is the best choice to bring out this study

1.2 Research design

As regards to the aims of the study, the study can be said to be a descriptive research Actually, the study is descriptive in the fact that it is aimed to have an overall

look on all of the possible factors that can affect the effectiveness of the English teaching at AGU After having that description of the situation of the English teaching

at AGU, there will be further studies of the factors revealed in the description

Besides, the present study is also a combination of quantitative and quantitative types of research in which data are collected by means of a test administered to students, questionnaires to be completed by both teachers and students, and interviews with teachers in terms of their grading of students’ speaking Consequently, first, the methodology involves locating the degree of the English teaching effectiveness through the results of the test Then, the researcher wishes to explain the possible reasons of the effectiveness achieved with the data that are synthesized from the questionnaires and the interviews In other words, with the data from the questionnaires and the interviews, the factors that affect the teaching program, both positively and negatively, are revealed Finally, when the factors are revealed, suggestions for the enhancement or exclusion of the factors are given

2 Procedure

The procedure for carrying out the present research includes 3 stages, namely description of population and sampling, instruments for data collection and data collection and analysis

2.1 Description of population and sampling

2.1.1 Population

Since the study aims to investigate the effectiveness of teaching English for non-majors at AGU, two groups of objects, the students and the teachers of English in the university, are involved in the research

As required in the curriculum framework, English is taught to all students (of course, except the English majors) as a non-majored subject Therefore, the first group

of population, the students, is very large because it is constituted by all of the students

at AGU The other group includes all of teachers to be in charge of teaching English for non-majors

2.1.2 Sampling

2.1.2.1 Students

Although all of the students at AGU are involved in learning English, it is not practicable and feasible to assemble data from the entire population Sampling, therefore, was carried out with 116 students from the Faculty of Education The sample students were taken randomly by drawing lots with the number of them on the student lists provided by the Training Department of AGU (At first, 140 students were invited to be the sampling, however, there were only 116 taking part in the test and the questionnaires) There are 5 faculties at AGU, however, the students of the Faculty of Education were chosen because the faculty trains different majors with students having

a range of income levels The 116 students came from 12 classes that were at the beginning of the 5th semester learning at the AGU Such kind of students was chosen because they had finished 3 semesters learning English Their English could reflect rather sufficiently the effectiveness of the English teaching of AGU Furthermore, the sampling was carried out with the students at the beginning of the 5th semester, not at the end of the 3rd semester There was a time gap to see the lasting contribution of the English teaching or the Effects and Impacts, one of the major issues to be addressed as mentioned in the Logical Approach

2.1.2.2 Teachers

The number of 19 teachers of English of the Department of Foreign Languages

at AGU is not a big one However, not all of the teachers were invited to be the sampling of the study, only the teachers who frequently teach the program of English for non-majors were invited to take part in the questionnaire and the interview The reason is that they need a certain time teaching that program so that they have a thorough understanding of the English teaching situation at AGU and then give precise feedback for the program Consequently, only 7 teachers were asked to be the questionnaire respondents and the interviewees of this survey study

2.2 Instruments for data collection

The decision was made on the types of instruments for collecting data as well as the types of data to be collected in accordance with the purposes of the research The instruments selected for the data collection include the tests administered to the students, the questionnaires for both the teachers and the students and the interview with the teachers about their grading of the students’ speaking skill

2.2.1 Test

The study attempted to make a survey on the effectiveness of the English teaching at AGU, and that teaching effectiveness can be shown from the level of English that the students get after finishing 3 semesters of learning English As a result, a test is necessary to determine the students’ level of English (See Appendix 1 for the full test.)

The test task-types and test content were taken into careful consideration to get the reliability and validity of the test The task-types were various with multiple choice tasks to ensure the objectivity of the test results, and with essay writing task to understand the ability of language production of the students The content of the test covered rather enough the knowledge that the students had to master at the end of the English teaching program The knowledge to be tested was chosen with a careful search on the course book, Lifelines The test should have accessed the four basic skills of English (Speaking, Reading, Writing, and Listening) However, only 3 skills were tested, the speaking was not included in the test Actually the number of sample students was rather big It would take a lot of time if the speaking test were administered Moreover, it was very difficult to gather such a big number of students

to take the speaking test because the students came from different classes and the classes had different study schedules Being unable to organize the speaking test for all students could not prevent the study from getting information about the students’ speaking competence as well as speaking examination organization This information was collected from the interview with the teachers and it will be discussed later

The level of the test was also the most important issue being considered when designing the test Because the level of the test could help to locate the real English level of the students, it should not be too difficult or too easy so that it could reflect precisely the students’ competence in English The students at AGU are required to get intermediate level in English when the program finishes, but there are no suggested criteria for determining if the students’ performance gets the required level Therefore

to identify the student’s level, the researcher of this thesis prefers to use the description

of the six levels of English of the Common European Framework (CEF) to understand what the students need to have to be leveled as Intermediate and then to design the test

appropriately In Vietnam so far, the term “Intermediate” has been used without consistent agreement on the criteria to identify it The so-called Intermediate level as addressed in Vietnam may be lower than the one defined in the CEF Hence the test of this study was designed at the CEF B1 level which is equivalent to Pre-intermediate level so that it would not go far beyond the students’ competence

To check the level of the test, 7 sample teachers were asked to evaluate its difficulty Five of them agreed that the test was suitable for their students’ ability and two of them believed it was rather easy To be more objective in determining the level

of the test, this study used the concept “readability” as suggested by Dr Vu Thi Phuong Anh in her ETGA report In her report, the indicator Flesch Reading Ease (FRE) (available in MS-Word software on Microsoft Window) was encouraged to use

to indentify the level of English tests The following is the table of the equivalence of the FRE and the level of the tests

Equivalent education degree of the native speakers

0-20 Mastery (C2), equivalent to IELTS

21-35

Effective Operational Proficiency(C1), equivalent to IELTS 7.0 and above

Graduate

36-50 Vantage (B2), equivalent to IELTS

51-65 Threshold (B1), equivalent to IELTS

66-80 Waystage (A2), equivalent to IELTS

81-100 Breakthrough (A1), equivalent to

The test has the FRE of 64 which is equivalent to level B1 (Threshold or intermediate) in CEF English levels With the FRE of 64, the test matches the intended level to be investigated in the survey

Pre-2.2.2 Questionnaire

The study also uses questionnaires as a method to collect both qualitative and quantitative data In attempt to get the required information, two sets of semi-structured questionnaires were developed and administered after the test was carried out One was for the teachers, and the other was completed by the students The expected responses to the items of the questionnaires, both structured and opened, were intended to focus on the objectives of the research and to facilitate the answer to the research questions The wording was paid careful attention as well to increase the reliability and validity of the data collected Besides, the questionnaires were written in Vietnamese to confirm the full understanding from the informants; thus reliable information could be gathered The students were also told that they did not have to include their names if preferable so that they did not feel reluctant when reporting anything bad or personal The two tables below provide the summaries of the student and teacher questionnaires (see Appendix 2 and 3 for the detailed questionnaires)

questions

3 Information of conditions supporting the

4 Evaluation of the English program for non-majors and suggestions for the program

improvement

9

Summary of questionnaires for students

questions

2 Information of conditions supporting the

3 Evaluation of the English program for non-majors and suggestions for the program

improvement

8

Summary of questionnaires for teachers

2.2.3 Interview

As mentioned above, due to the time limit, the speaking test couldn’t be administered to the sample students Consequently, interviews were employed as instruments for collecting data of the students’ speaking ability as well as the organization of the speaking examination administered every semester at AGU The interviews were structured with 4 questions being put forth to all teachers of English at AGU The 4 questions included the following items

- The teacher’s evaluation of the student’s speaking competence

- The existence of criteria during the grading process

- The agreement among the teachers on the criteria

- The procedure to carry out speaking examinations

2.3 Data collection and analysis

to

For the teachers, it was difficult to have the presence of all teachers at the same time Thus, the teachers were asked to respond to the questionnaire at home, where they had time to think and then gave careful consideration on responding to the questionnaires

After the administration of the test and the questionnaires, the interview was carried out with each teacher at the agreed time Every teacher was asked the same questions, however, according to their answers, there were some modifying questions

All of the data collected from the test, the questionnaires and the interviews were carefully stored to be analyzed in the next step, data analysis

2.3.2 Data analysis

The SPSS software was employed to process the data obtained from the collection phase Both quantitative and qualitative techniques were used for analyzing the data in the study The quantitative analysis was conducted with the use of descriptive statistics to analyze the test results The test results were categorized into the results of each skill tested Descriptive statistics was also used to analyze the informants’ responses to the questionnaires As for qualitative analysis, the information of the interview was analyzed to form the idea of how good were students

at speaking and how well the speaking examinations were organized Qualitative techniques were also used to classify the suggestions for improving teaching quality from the informants of the questionnaires The suggestions were classified into categories according to the factors that were suggested to be improved Nevertheless, the suggestions for improvement do not only come from such kind of data The suggestions for improvement were given by the present authors on the basis of the results of the analysis of all data obtained

In addition, the correlation was also used to seek for the mutual relations amongst the factors related to the teaching program

After the data had been analyzed, a report of the results from each of the instrument was made A discussion of these results followed to form the basis for conclusions later on

Chapter 4

RESULTS

After the data was collected, they were carefully analyzed The outcomes of the analyzing process are presented in this chapter In other words, the results from the data analysis needed for answering the research questions are written down in details

in this part, which is considered as the basis for the discussion and conclusion in the next chapter

This chapter is constituted by two sections The first section is for the summary

of the data collection and analysis The next section reports the results from the data analysis This second section is subdivided into three parts, one for the results from the test, another for the results of the interview and the other for the outcomes of the questionnaire analysis

1 Summary of the data collection and data analysis

1.1.2 The questionnaire

The questionnaires were made to get the teachers’ and the students’ evaluation

of all possible factors related to the program To the students, both of the test and the questionnaires were administered on the same day for each group of sample students Therefore, the number of students taking the test and the questionnaires was the same

140 students were expected to be the sampling; however, only 116 students took part

in the test and the questionnaires The absence of the other students was due to their personal reasons To the teachers, seven of them who had certain time teaching the program received the questionnaires

2.1 Results from the test

116 students took the test The test was intended to access the students’ Listening, Reading, Writing, Grammar and Vocabulary The test was carefully designed and graded for getting the most reliability Therefore, almost items in the test were in multiple choice type of test However, the Writing part was designed with the open-ended type of test, an essay, to see the student’s ability of language production The reliability of such a type of test may be affected by the subjectiveness of graders

To clarify this problem, the inter-rater reliability of the writing test was put into consideration This reliability is presented later in a separate part The results of the test now are shown with the total marks and the marks the students got in each accessed skill in order to see the students’ English performance in common and their performance in each language skill in particular Table 1, 2, 3, and 4 present the frequencies of the students’ performance in the test Table 5 shows the mean and standard deviation of the test and each skill accessed in the test

Frequency Percent Valid Percent

Cumulative Percent

Table 1 Results of Listening

Frequency Percent Valid Percent

Cumulative Percent 40 2 1.7 1.7 1.7 80 2 1.7 1.7 3.4 1.00 3 2.6 2.6 6.0 1.20 6 5.2 5.2 11.2 1.40 9 7.8 7.8 19.0 1.60 12 10.3 10.3 29.3 1.80 12 10.3 10.3 39.7 2.00 11 9.5 9.5 49.1 2.20 12 10.3 10.3 59.5 2.40 17 14.7 14.7 74.1 2.60 20 17.2 17.2 91.4 2.80 7 6.0 6.0 97.4 3.00 3 2.6 2.6 100.0 Valid

Total 116 100.0 100.0

Table 2 Results of Reading

Frequency Percent Valid Percent

Cumulative Percent 000 14 12.1 12.1 12.1 .100 7 6.0 6.0 18.1 .200 6 5.2 5.2 23.3 .300 7 6.0 6.0 29.3 Valid

.400 7 6.0 6.0 35.3

.500 6 5.2 5.2 40.5 .600 8 6.9 6.9 47.4 .700 5 4.3 4.3 51.7 .800 5 4.3 4.3 56.0 .900 8 6.9 6.9 62.9 1.000 11 9.5 9.5 72.4 1.050 2 1.7 1.7 74.1 1.100 4 3.4 3.4 77.6 1.125 1 .9 .9 78.4 1.200 4 3.4 3.4 81.9 1.225 1 .9 .9 82.8 1.250 1 .9 .9 83.6 1.300 1 .9 .9 84.5 1.400 4 3.4 3.4 87.9 1.500 6 5.2 5.2 93.1 1.600 3 2.6 2.6 95.7 1.700 2 1.7 1.7 97.4 1.800 3 2.6 2.6 100.0 Total 116 100.0 100.0

Table 3 Results of Writing

Frequency Percent Valid Percent

Cumulative Percent

Table 4 Results of Grammar and Vocabulary

Minimum Maximum Mean Std

Table 5 Descriptive Statistics of the Test

The consideration of the inter-rater reliability of the Writing

To check the inter-rater reliability of the writing, after the first grader, the researcher of the present study, had marked the writings, the second grader was asked

to mark at random 20 students’ writings without knowing the marks the first grader had given After the second grading, it could be seen that the two graders gave the same marks for 7 writings Then, a discussion was carried out between the two graders

to get the common ideas about the other 13 writings After that discussion, they got the common marks for another 8 writings The other 5 writings were still received different marks from the two graders However, the difference of the two graders was not significantly big The results of the two times grading and the average of them are given in Table 6 The average was used to add to the total marks of the test which are presented in the Table 5 above

Writing Independent Grading Discussed Grading

1 grader 2 grader 1 grader 2 grader Average

Table 6 Results of two times writing grading

2.2 Results from the interview

After the interview with 7 teachers about their speaking grading and the students’ speaking competence, some of remarkable results are summarized as below

2.2.1 Procedure of the speaking examination

The teachers did not have a common procedure to carry out the speaking examination Some teachers gave their students oral topics before the examination, the others did not They just told their students which speaking functions to be tested in the examination In the examination, the students drew lots with a topic, had some time to prepare and then presented it After the presentation, the teachers asked the students some questions before the marks were given The speaking test took 20% of the total marks of the end-of-term test

2.2.2 The teachers’ grading

There were no common criteria amongst the teachers for marking the students’ speaking They did not discuss beforehand about the criteria, each of them had their own ones for their grading

2.2.3 The teachers’ evaluation of the students’ speaking competence