Summary of doctoral thesis of science education: Authentic assessment of english majored undergraduates’ english proficiency

This study aims to provide a procedure and specific measures that can be used in the authentic assessment of English-majored undergraduates’ English proficiency in order to increase the validity, reliability and comprehensiveness of assessment activities.

MINISTRY OF EDUCATION AND TRAINING VIETNAM NATIONAL INSTITUTE OF EDUCATIONAL SCIENCES

SUMMARY

Hanoi, 2019

THIS THESIS IS COMPLETED AT VIETNAM NATIONAL INSTITUTE OF EDUCATIONAL SCIENCES

Supervisors:

1 Assoc Prof Dr Nguyễn Thị Lan Phương

………

2 Dr Lê Đông Phương ………

Reviewer 1:

Reviewer 2:

Reviewer 3:

The thesis will be defensed by the author before the Thesis Assessment Board of Vietnam National Institute of Educational Sciences, at 101 Tran Hung Dao Street, Hanoi, Vietnam At , date month year

Copies of this thesis are available at:

- National Library of Vietnam

- Library of Vietnam National Institute of Educational Sciences

INTRODUCTION

1 Rationale

Assessment plays a significant role in the instruction of languages in general and English in particular Assessment can be used to measure learners' achievements (assessment of learning) and to provide feedback on the teaching and learning so that teachers and students can make adjustments to improve their teaching and learning (assessment for learning) In other cases, no longer considered

a separate part of the instructional process, assessment is integrated into this process and becomes its integral part, and accordingly in this way assessment activities are actually learning activities (assessment as learning)

In spite of its importance, assessment activities are not always carried out effectively so that they can be informative enough for those concerned and supportive of learning With regard to educational assessments in Vietnam, research has found various problems Following are some of them:

- Assessment activities fall short of expectations of those concerned in terms

of validity, reliablity, comprehensiveness

- In many cases, assessment activities place more focus on the measurement

of students’ knowledge and lower-order thinking skills rather than their order thinking skills such as critical thinking and problem-solving skills, and thus the assessment activities are not able to create positive washback on the process of teaching and learning [10, 14, 17]

higher-It can be seen that assessment activities do not effectively support the teaching and learning process In other words, there remains a gap between instructional and assessment activities More specifically, communicative and effective learning activities do not directly increase students’ scores in tests/exams and high scores in tests/exams do not necessarily mean that the students have a high level of English proficiency Therefore, there is a need for a balanced assessment which emphasizes the direct relationship between instructional and assessment activities of the English language [71] In this regard, a good and effective language

learning activity (interactive and communicative) can at the same time be used as an assessment activity and vice versa [90]

Authentic assessment requires students to communicate using English in real-life communicative situations In addition, authentic assessment enables us to

"assess what we value so that we value what we assess" [90] In some cases, English learners are required to take tests that are designed to measure their knowledge of the English language These tests often focus on the measurement of students' knowledge and understanding of separate elements of English such as grammar, vocabulary or pronunciation by choosing or checking an option among several (often 3 or 4) alternatives They do not directly measure students' actual use

of the language or their communicative language ability In this regard, there remains a gap between students' communicative language competence (what we value) and their knowledge of separate elements of the language (what we assess) Authentic assessment is believed to help deal with this problem in that it can play a role in narrowing the gap between what we value and what we assess

Despite its strengths, in Vietnam little research has been done on authentic assessment in general and on its employment in the teaching, learning and assessment of English in particular All this makes room for the author of this study

to conduct an investigation into the employment of authentic assessment in measuring English-majored undergraduates' communicative English competence or their "English proficiency” in this thesis

2 Aim of the thesis

This study aims to provide a procedure and specific measures that can be used in the authentic assessment of English-majored undergraduates’ English proficiency in order to increase the validity, reliability and comprehensiveness of assessment activities

3 Research area and object of the study

- Research area: the assessment of English-majored undergraduates’ English proficiency

- Object of the study: authentic assessment activities in the process of teaching, learning and assessment of English-majored undergraduates’ English proficiency

4 Hypothesis

If the proposed precedure (including three stages: designing, implementing and evaluating) and measures (each includes sample tasks, authentic assessment criteria and rubrics) were used in the assessment of English-majored undergraduates’ English proficiency with focus placed on the four communicative skills of listening, speaking, reading and writing, then they would help increase the quality (validity, reliability and comprehensiveness) of the assessment activities of English-majored undergraduates’ English proficiency at universities in Vietnam

5 Objectives of the study

- to build up the body of literature on: (1) authentic assessment in the teaching, learning and assessment of EFL students; (2) communicative language competence; (3) authentic assessment of English-majored undergraduates’ English proficiency;

- to investigate the current practice of the assessment of English-majored undergraduates’ English proficiency at various universities in Vietnam;

- to propose a precedure and measures (including sample tasks, authentic assessment criteria and rubrics) that can be used to measure English-majored undergraduates’ English proficiency in the instructional process at universities in Vietnam;

- to experiment with the procedure and measures proposed to test their feasibility and effectiveness

6 Scope of the study

- The study was conducted with regard to the authentic assessment of English-majored undergraduates’ English proficiency within the framework of assessment activities conducted in the instruction of the language skills of listening,

reading, speaking and writing in the curriculum of English studies for majored undergraduate students at universities in Vietnam

- A survey was conducted at 6 different universities which provide English-major courses for full-time undergraduate students

English The experiment was carried out between 8/2017 and 10/2017 at the Department of English Studies – Hanoi University

7 Approaches and methods

Approaches: (1) systematic approach; (2) context-based approach; (3)

activity-based approach; (4) process approach

Methods

Secondary research methods: (1) reviewing the literature on the topic; (2)

comparing and contrasting; (3) analyzing, synthesizing and evaluating

Primary research methods: the tools used to collect primary information

include: (1) survey questionnaires; (2) interviews; (3) seminars; (4) observations; (5) experiment

Other methods: consulting; quantitative and qualitative methods

8 Points to defense

- Authentic assessment enables students to use components of communicative language competence to communicate and interact with others in reallife contexts, so it is necessary that authentic assessment be used in the process

of teaching, learning and assessment of English-majored undergraduates’ English proficiency at universities in Vietnam;

- The employment of authentic assessment is underresearched in Vietnam;

- Authentic assessment is not commonly used at universities in Vietnam;

- If the proposed procedure and measures are used in the assessment of English-majored undergraduates’ English proficiency with focus placed on the four communicative skills of listening, speaking, reading and writing, then they would

help increase the quality (validity, reliability and comprehensiveness) of the assessment activities of English-majored undergraduates’ English proficiency at universities in Vietnam

9 Contributions of the thesis

In theory, the thesis:

- helps build upon the body of literature on authentic assessment of communicative language competence of EFL learners

- helps clarify the theoretical basis and develop a theoretical framework for the integration of authentic assessment into the assessment of English-majored undergraduates’ English proficiency at universities in Vietnam

- clarifies purposes, definitions, roles and features of authentic assessment

- describes and analyzes the procedure, methods, criteria and rubrics used in the incorporation of authentic assessment in the process of teaching, learning and assessment of English-majored undergraduates’ English proficiency

In practice, the thesis:

- identifies and describes the common practices of the testing and assessment

of English-majored undergraduates’ English proficiency at universities in Vietnam, and provides an analysis and evaluation of the practices, with focus placed on the assessment activities conducted in the teaching and learning process

- proposes a procedure (including three stages: designing, implementing and evaluating) and 4 measures (used to assess the four language skills of listening, reading, speaking and writing) for teachers to employ authentic assessment in their teaching and assessment of English-majored undergraduates’ English proficiency

- helps confirm that the proposed procedure and measures worked effectively

by getting feedback from specialists, teachers and students in the areas of language teaching, learning and assessment and by experimental activities

10 Organization of the thesis

In addition to the parts of Introduction, Conclusions and Recommendations, References and Appendices, the present thesis consists of 4 chapters:

Chapter 1: Literature review

Chapter 2: Current practices of integrating authentic assessment into the assessment of English-majored undergraduates’ English proficiency

Chapter 3: Procedure and measures for integrating authentic assessment into the assessment of English-majored undergraduates’ English proficiency at universities in Vietnam

Chapter 4: Experiment

CHAPTER 1: LITERATURE REVIEW 1.1 A brief overview

1.1.1 Research on assessment in language instruction

A lot of research has been done on instructional assessment with focus placed on the discussion of definitions, goals and objectives, roles and functions, characteristics, forms and methods of assessment

Based on the findings of various studies [11, 14, 23, 43, 97], assessment can

be defined as a process of gathering information about learners, discuss, analyze and explain the information in a systematic way in order to understand what learners know and can do from the knowledge and experiences they gather, compare with the goals and objectives of the curriculum and provide stakeholders (learners, teachers, managers, employers, etc.) with the information so that they can make decisions (e.g for recruitment) or make adjustments or changes to increase the quality of teaching and learning

With reference to the findings of various studies [11, 43, 54, 60, 61, 92], assessment can be used to serve:

- administrative goals: for placement, selection, certification, making by stakeholders (e.g school managers, recruiters, employers);

decision instructional goals: to gain an understanding of the teaching and learning process, learners’ achievements, learners’ competence, etc so that teachers and learners can make adjustments or changes to improve the instructional process

Studies also discuss different forms of assessment categorized based on different criteria For example, based on the criterion of time of assessing, assessment can be diagnostic (before the learning process), formative (during the learning process) or summative (after the learning process) [11, 23]

Methods used in the assessment of learning vary, but to a certain extent they can be summarized as follows: assessment of knowledge (e.g.: objective testing), performance-based assessment (e.g.: presentation, project-based assessment, portfolio) and observation [11]

1.1.2 Research on communicative language competence and the assessment of communicative language competence

There has been an enormous amount of research done on communicative language competence with focus placed on the clarification of its definition and structure

Early pieces of research propose a narrow definition of language competence which only covers the knowledge of a language that a person has Together with that narrow definition of language competence, the term “language performance” is also introduced Language performance is used to refer to learners’ ability to produce the language with the application of their knowledge of the language [49]

More recently, the definition was expanded by several linguists and reseachers to a broader notion of “communicative language competence” which means the ability to communicate in various contexts using the language, not only using elements of the language such as grammatical knowledge but also gestures, intonation, strategies and other non-verbal means of communication” [45, 46, 50, 68]

Two major approaches to the assessment of learner communicative language competence are: (1) testing; and (2) performance-based assessment tasks [44, 82]

1.2 Research on authentic assessment in language instruction

1.2.1 Definition and characteristics of authentic assessment

Researchers have come up with various definitions of authentic assessment

Based on their findings, authentic assessment can be summarized and defined as an approach to assessment in which learners are asked to do authentic tasks that demonstrate meaningful application of essential knowledge and skills in real-world contexts [57, 58, 65, 70, 72, 79, 82]

From the findings of various studies [10, 11, 44, 61, 70, 71, 82, 86], main characteristics of authentic assessment can be summarized as follows:

- Resembling real-world tasks and activities;

- Based on problem-solving activities;

- Formative, supportive of learning;

- Emphasis placed on interaction and collaboration;

- Performance-based activities;

- Requirements and scoring criteria made clear to students before learning begins;

- Measuring both learning products and process;

- Can be used together with or to replace standardized tests

Authenticity

Authenticity is an important characteristic that distinguishes authentic assessment from other forms of assessment Based on the studies of Bachman and Palmer, Cumming and Maxwell, Ozkan and Palm [42, 51, 85, 86], an assessment is regarded as authentic if:

- students are engaged in cognitive processes that are important in successful behaviors in life beyond school;

- language input used in the assessment is authentic, i.e materials which are produced for real life, not changed or simplified for instructional purposes;

- the context in which the assessment takes place should be real-life situations or similar to real-life situations

1.2.2 Goals and objectives of authentic assessment

Authentic assessment is designed to serve three main objectives:

- to measure learners’ competence;

- to provide a statement of learners’ competence or language ability (e.g certification) so that stake-holders can make appropriate decisions;

- to promote learning (e.g provide feedback on the teaching and learning process/activities so that teachers and students can make adjustments suitable for their own teaching and learning practices) [71]

1.2.3 Principles in using authentic assessment

During the process of using authentic assessment, it is important for teachers

to take account of the following factors: (1) validity; (2) reliability; and (3) authenticity [82]

1.2.3.1 Validity

Validity is to do with whether teachers can assess what they want to assess More specifically, an assessment task is considered valid if it can measure exactly what it is designed to measure [67]

1.2.3.2 Reliability

Reliability refers to how consistently a method measures a learner’s language proficiency If the same result can be consistently achieved by using the same methods under the same circumstances, the result is considered reliable [44, 67]

1.2.3.3 Authenticity

Apart from validity and reliability, in order for an authentic assessment task/activity to be distinctive from other assessement activities, it is necessary for

the assessment to have authenticity More specifically, in carrying out the task, students need to be engaged in cognitive processes that are important in successful behaviors in life beyond school, the language input used in the assessment need to

be authentic (i.e materials which are produced for real life, not changed or simplified for instructional purposes), the context in which the assessment takes place should be real-life situations or similar to real-life situations

1.2.4 Some models of using authentic assessment in language teaching, learning and assessment

According to various studies [40, 71, 72], within the framework of this thesis, it is suggested that teachers embed authentic assessment in the learning process using the following pentagon model:

With focus placed on assessment, authentic assessment can be employed using the Quad model introduced by Anthony et al.[71]:

Lectures

Tests Authentic

assessment Assignments

- Provincial or state exams

1.2.5 Procedure, approaches and types of authentic assessment

With reference to the findings of various studies [44, 71, 82], it can be summarized that the procedure suggested for the employment of authentic assessment includes three main stages: (1) designing; (2) implementing; and (3) evaluating

It is also learned from various pieces of research that there are two major approaches to authentic assessment: (1) observation of assessment products (portfolios, reports, learning journals, research pieces, etc.); and (2) observation of performance (e.g.: interviews, conversations, roleplays, presentations, simulations, etc.) [71, 82]

1.3 Research on communicative language competence

1.3.1 Definitions of communicative language competence

Different researchers come up with different definitions of communicative language competence The author of this thesis uses the definition introduced by the Canadian Language Benchmarks: English as a Second Language for Adults, adapted from Bachman (1990), Bachman and Palmer (1996, 2010) and Celce-Murcia, Dornyei and Thurrel (1993, 1995): communicative language ability refers

to “the ability to communicate: to interact, to express, to interpret and to negiotiate meaning, and to create discourse in a variety of social contexts and situations”

[45]

1.3.2 Structure of communicative language competence

With regard to the structure of communicative language competence, in this thesis the author uses the model provided by Celce-Murcia, Dornyei and Thurrell