Using data driven learning to improve non english majors vocabulary learning in toeic reading an action research at nong lam university ho chi minh city m a

VIETNAM NATIONAL UNIVERSITY- HO CHI MINH CITY UNIVERSITY OF SOCIAL SCIENCES & HUMANITIES FACULTY OF ENGLISH LINGUISTICS & LITERATURE USING DATA-DRIVEN LEARNING TO IMPROVE NON-ENGLISH M

VIETNAM NATIONAL UNIVERSITY- HO CHI MINH CITY

UNIVERSITY OF SOCIAL SCIENCES & HUMANITIES

FACULTY OF ENGLISH LINGUISTICS & LITERATURE

USING DATA-DRIVEN LEARNING TO IMPROVE NON-ENGLISH MAJORS’ VOCABULARY LEARNING

IN TOEIC READING: AN ACTION RESEARCH AT NONG LAM UNIVERSITY – HO CHI MINH CITY

A minor thesis submitted to the Faculty of English Linguistics & Literature

By

PHAM QUYNH MAI

Supervised by

NGUYEN THI NHU NGOC, Ph.D

HO CHI MINH CITY, NOVEMBER 2021

Acknowledgments

This thesis would not have been completed without the support and encouragement of many people I could not thank them enough for helping me throughout this wonderful experience

First and foremost, I would like to express my deepest gratitude to my supervisor, Dr Nguyen Thi Nhu Ngoc, for her continuous support of my thesis, for her patience, motivation, enthusiasm, and immense knowledge Her guidance and feedback inspired me in all the time of research and writing of this thesis It was my pleasure to

be one of her students

My sincere thanks go to all the lecturers of the MA program for their valuable discussions and lectures during my learning journey at University of Social Sciences and Humanities, Vietnam National University – Ho Chi Minh City

I am extremely thankful to Mr Dao Duc Tuyen, the Director of the Center for Foreign Studies of Nong Lam University, for his approval of my conducting this research, to the Division Manager and the IT staff for providing me constant assistance during the pilot stage and the main study I also acknowledge my students for their enthusiasm and active participation in this study

Last but not least, I would like to thank all of my friends, my family members for encouraging and supporting me whenever I needed them

Statement of authorship

I certify that this thesis entitled “Using data-driven learning to improve non-English majors’ vocabulary learning in TOEIC Reading: An action research at Nong Lam University – Ho Chi Minh City” is my own work

This thesis has not been submitted for the award of any degree or diploma in any other institution

Ho Chi Minh City, November 2021

Pham Quynh Mai

Retention and use of the thesis

I hereby state that I, Pham Quynh Mai, being the candidate for the degree of Master of TESOL, accept the requirement of the University relating to the retention and use of Master's Theses deposited in the Library

In terms of these conditions, I agree that the originality of my thesis deposited in the Library should be accessible for the purposes of study and research, in accordance with the normal conditions established by the Library for the care, loan, or reproduction of theses

Ho Chi Minh City, November 2021

Pham Quynh Mai

Abstract

That language learners should have a number of encounters with words in multiple contexts in the classroom before they understand, remember, and apply them has been highlighted in previous studies in vocabulary teaching and learning One prominent approach promoting this is data-driven learning (DDL), in which learners interact with corpora’s concordances as a kind of language data In fact, DDL is gaining more popularity as it reflects language learning theories, such as noticing hypothesis, constructivism, and socio-cultural theory Although researchers have called for more replication of DDL studies to validate the effects of DDL, very few studies were found

in Vietnam’s context This study is an effort to fill this gap and, through the use of DDL,

to solve the problems of non-English majors’ vocabulary learning in Test of English for International Communication (TOEIC) Reading at The Center for Foreign Studies

of Nong Lam University (NLU-CFS) By adopting the action research design, the study aims to investigate the effects of DDL on 38 non-English majors’ vocabulary learning

in TOEIC Reading Data from the pretest and posttest shows that the students made significant improvements in vocabulary performance after the DDL intervention This success is consistent with the results from the post-questionnaire and the semi-structured interviews From these findings, the researcher’s self-reflections on the study and a plan for future actions were drawn out

Keywords: data-driven learning, vocabulary learning, TOEIC Reading,

non-English majors

Table of contents

Acknowledgments i

Statement of authorship ii

Retention and use of the thesis iii

Abstract iv

Table of contents v

List of tables ix

List of figures xi

List of abbreviations xii

CHAPTER 1 INTRODUCTION 1

1.1 Background to the study 1

1.2 Statement of the problem 2

1.3 Aim and research questions 4

1.4 Significance of the study 4

1.5 Scope of the study 5

1.6 Organization of thesis chapters 5

CHAPTER 2 LITERATURE REVIEW 6

2.1 Overview of the TOEIC test and vocabulary acquisition 6

2.1.1 Overview of the TOEIC test 6

2.1.2 Vocabulary acquisition 7

2.1.2.1 Receptive and productive vocabulary 8

2.1.2.2 Aspects of knowing a word 9

2.1.3 Vocabulary learning strategies 10

2.2 Data-driven learning (DDL) 12

2.2.1 What is DDL? 12

2.2.2 Theories underpinning DDL 16

2.2.3 Using DDL in EFL and vocabulary learning 17

2.2.3.2 DDL activities 20

2.3 Previous studies 21

2.4 Conceptual framework 25

CHAPTER 3 METHODOLOGY 27

3.1 Research design 27

3.2 Research site 27

3.3 Participants 29

3.4 Learning materials and procedure 30

3.4.1 Selection of textbook and target vocabulary 30

3.4.2 DDL materials and activities 35

3.5 Research tools 38

3.5.1 The TOEIC Reading tests 38

3.5.1.1 Description of the TOEIC Reading tests 38

3.5.1.2 Piloting 42

3.5.1.3 Test administration and scoring 43

3.5.2 Questionnaires 44

3.5.3 Semi-structured interviews 47

3.6 Data collection procedure 47

3.7 Data analysis procedure 48

CHAPTER 4 RESULTS AND DISCUSSION 50

4.1 Research question 1: What experience of vocabulary learning did non-English majors have before the use of DDL at NLU-CFS? 50

4.2 Research question 2: How much does DDL improve the students’ vocabulary learning in TOEIC Reading at NLU-CFS? 54

4.2.1 Students’ results on the pretest and posttest 54

4.2.1.1 Total TOEIC Reading 55

4.2.1.2 Total vocabulary 56

4.2.1.3 Form, Meaning, and Use 57

4.2.1.4 Target vocabulary 58

4.2.2 Students’ changes of experience in the posttest 59

4.3 Research question 3: What do students think about the effects of DDL in their vocabulary learning at NLU-CFS? 60

CHAPTER 5 CONCLUSION 79

5.1 Conclusions and contributions 79

5.2 Reflections on self-development and a plan for future actions 81

5.2.1 Reflections on self-development 81

5.2.2 A plan for future actions 82

5.2.2.1 Planning the use of DDL for foundation classes at NLU-CFS 82

5.2.2.2 Designing DDL activities and materials 82

5.2.2.3 Utilizing guided induction 83

5.2.2.4 Upgrading the computers and internet connection at NLU-CFS 83

5.3 Limitations of the study 83

5.4 Recommendations for further research 84

REFERENCES 86

APPENDIX 1 Approval of conducting the research at NLU-CFS 94

APPENDIX 2 Course syllabus – EE-Exam classes 95

APPENDIX 3A 114 selected vocabulary 96

APPENDIX 3B The final list of target vocabulary 100

APPENDIX 4 A sample of Powerpoint slides for a consolidation game 108

APPENDIX 5 A sample of vocabulary paper-based worksheet 112

APPENDIX 6 A sample of vocabulary practice worksheet 115

APPENDIX 7 A sample of vocabulary computer-based tasks 119

APPENDIX 8 Full set of samples of DDL activity procedure 124

APPENDIX 9A TOEIC Reading pretest 125

APPENDIX 9B TOEIC Reading posttest 150

APPENDIX 10 Specifications of question items in the pretest and posttest 173

APPENDIX 11A Pre-questionnaire (English version) 175

APPENDIX 11B Pre-questionnaire (Vietnamese version) 178

APPENDIX 11C Post-questionnaire (English version) 181

APPENDIX 11D Post-questionnaire (Vietnamese version) 184

APPENDIX 12 Semi-structured interview questions 187

APPENDIX 13 A sample of interview transcription 189

APPENDIX 14 SPSS output of normality test 197

APPENDIX 15 SPSS output of pretest and posttest results 199

List of tables

Table 2.1 Vocabulary aspects setting for the present study 10

Table 2.2 Summary of the definitions of DDL 14

Table 2.3 Summary of previous studies 23

Table 3.1 Classification of students into EE-Exam classes based on their latest TOEIC scores 29

Table 3.2 The breakdown of test sources 31

Table 3.3 Sample selected words from Barron’s essential words for the TOEIC 6th edition 32

Table 3.4 Sample supplemented words from Part 5, 6, and 7 of Practice test 1 33

Table 3.5 The use of guided induction in DDL activities 37

Table 3.6 Sample DDL activity procedures of sessions 10 and 11 38

Table 3.7 Examples of Form, Meaning, Use questions and Target vocabulary questions in the pretest and posttest 39

Table 3.8 Summary of item types in the pretest and posttest 42

Table 3.9 Pilot test results 42

Table 3.10 Summary of the questionnaire item 45

Table 3.11 Reliability of the items in each theme of the post-questionnaire 46

Table 3.12 Timeline of the research and teaching progress 48

Table 3.13 Analysis scheme for the research tools 49

Table 4.1 Students’ general thinking about vocabulary learning 50

Table 4.2 Students’ difficulties in vocabulary learning 51

Table 4.3 Students’ previously employed vocabulary learning strategies 52

Table 4.4 Shapiro-Wilk test of normality 54

Table 4.5 Comparison of students’ reading scores on the pretest and posttest 55

Table 4.6 Comparison of students’ scores on total vocabulary questions of the pretest and posttest 56

Table 4.7 Comparisons of students’ scores on Form, Meaning, and Use questions of

the pretest and posttest 57

Table 4.8 Comparison of students’ scores on target vocabulary questions of the pretest and posttest 58

Table 4.9 The usefulness of DDL activities 61

Table 4.10 The benefits of DDL to vocabulary learning 70

Table 4.11 The drawbacks of DDL 71

List of figures

Figure 2.1 An example of corpus query results for the word “request” in COCA 13Figure 2.2 The conceptual framework of the study 26Figure 4.1 The score type distribution of vocabulary questions of the pretest and posttest 56Figure 4.2 Students’ overall evaluation of the DDL implementation 74Figure 4.3 Students’ opinions on actions they would take regarding DDL 75Figure 4.4 Students’ opinions on whether DDL should be used in EE-Exam classes at NLU-CFS 76

List of abbreviations

CEFR Common European Framework of Reference for Languages COCA Corpus of Contemporary American English

DDL Data-driven learning

EFL English as a Foreign Language

NLU-HCM Nong Lam University – Ho Chi Minh City

NLU-CFS The Center for Foreign Studies of Nong Lam University TOEIC Test of English for International Communication

CHAPTER 1 INTRODUCTION

1.1 Background to the study

Over recent decades, the crucial importance of vocabulary has been highlighted

in English language learning Indeed, vocabulary is a necessary component for effective communication, as confirmed by Wilkins (1972), “… without grammar, very little can

be conveyed, without vocabulary, nothing can be conveyed” (pp 111–112) Additionally, Vermeer (1992) points out that learners’ vocabulary, not grammar knowledge, builds up their language proficiency (p 147) Also, one can definitely tell the difference between foreign learners and native speakers based on their vocabulary size (Laufer, 1998, p 255) Therefore, enhancing vocabulary learning is a strategic goal that language learners should strive for

The lack of vocabulary is an obstacle to learning language skills, especially reading comprehension Researchers have so far proved that vocabulary knowledge strongly correlates with reading comprehension (Anderson & Freebody, 1981, as cited

in Nagy, 1988; Nation, 2013; Schmitt, 2000) According to Anderson and Freebody (1981, as cited in Nagy 1988, p 1), a reader’s general vocabulary knowledge is the best predictor of how well that reader can understand the text Likewise, Stahl (2003) puts forward the role of vocabulary knowledge for “the correlations between measures of vocabulary and reading comprehension routinely are in the 0.90s almost regardless of the measures used or the populations tested” (p 241) This indicates that increasing vocabulary knowledge should be emphasized in teaching and learning second language reading comprehension

Nowadays, although the increasing importance of vocabulary is clearly seen in many widely-used textbooks and students seem to have more opportunities to be exposed to a variety of sources for learning vocabulary, many English as a Foreign Language (EFL) learners still struggle with vocabulary One major reason is that

vocabulary is still introduced in the format of a long list of isolated words without many specific language contexts This may limit learners’ vocabulary knowledge to form-meaning connections; in other words, they may have a relatively larger vocabulary size but would know little about how these words are used (Schmitt, 2014, p 915) Besides, limited time dedicated to vocabulary teaching and the lack of materials and resources with authentic contexts for vocabulary can prevent language teachers from creating favorable learning conditions for their students to enhance vocabulary learning

Thus, investigating vocabulary teaching and learning methods is still an ongoing journey in the world as well as in Vietnam

1.2 Statement of the problem

Under the policy of the 2008-2020 and 2017-2025 National Foreign Languages Projects, Vietnamese non-English majors are required to achieve level 3 of the Vietnam Foreign Language Framework, which is equivalent to level B1 in the Common European Framework of Reference for Languages (CEFR) to be awarded the bachelor's degree As a result, NLU-HCM requires non-English majors to either submit the TOEIC Listening and Reading certificate or pass an in-house TOEIC Listening and Reading test with at least 97 correct answers as the standard for graduation Since most non-English majors opt for the in-house TOEIC, NLU-CFS is currently holding TOEIC classes (with the name of English-Exit Exam 1 & 2) for these students to prepare for the test In these classes, the researcher notices that vocabulary learning has been a daunting task for most of her students It is shown from the pre-questionnaire of the current study that the students did not have good vocabulary knowledge and faced difficulties in almost all aspects of vocabulary Also, they usually translated everything into Vietnamese for understanding but at the same time admitted that this strategy could cause misunderstandings in some situations Such habits might be because there was a lack of exposure to language contexts in their vocabulary learning process; thus, they could not keep the acquired vocabulary and retrieve it later on However, the students

in the current study acknowledged the importance of vocabulary learning in TOEIC Reading skills and wanted to improve their vocabulary knowledge

Schmitt (2000) suggested that one kind of language data that provides authentic contexts for vocabulary is from corpora In fact, researchers into vocabulary teaching and learning have given much interest to an approach to language learning called DDL (Johns, 1986, 1991), which promotes students’ interaction with corpora What is distinctive about the DDL approach is inductive learning (or discovery learning), in which learners act as research workers to explore patterning in the target language and form generalizations of use patterns The teachers then take a facilitator role, guiding learners’ language discovery rather than a transmitter of knowledge The DDL implementation in the classroom is in line with language learning theories such as the noticing hypothesis, constructivism, and socio-cultural theory

Over the past two decades, DDL research has been integrated into vocabulary teaching and learning Learners’ significant achievement and favorable thoughts have been observed in many DDL studies in vocabulary, especially in terms of collocations (Chan & Liou, 2005; Geluso & Yamaguchi, 2014), vocabulary selected from the existed coursebooks (Binkai, 2012; Karras, 2016; Trang, 2019) or target vocabulary prepared for international certificates such as TOEIC and Cambridge examinations (Allan, 2010; Boulton, 2010) Although the DDL implementation can be subject to several criticisms due to its time-consuming and demanding feature for lower-level learners, DDL problems can be solved through prepared, printed materials and guided induction However, such options have not been highlighted in previous studies because most researchers have learners work autonomously with a concordancer in a total learner-led environment By using paper-based and computer-based DDL materials as well as guided induction, this study hopes to fill this gap

Besides, more replication of DDL studies from Asia and the Middle East with unique cultural backgrounds is still necessary Researchers should test how learners from different cultures, particularly non-western students from non-Indo-European language backgrounds, react to corpus examples (Frankenberg-Garcia, 2014, p 15)

Only two DDL studies by Karras (2016) and Trang (2019) focus on vocabulary learning

in Vietnam's context Besides, very few studies involve TOEIC vocabulary as the language input, except Boulton’s (2010) These are the gaps that this study hopes to fill

1.3 Aim and research questions

This study aims to investigate the effects of DDL on non-English majors’ vocabulary learning in TOEIC Reading at NLU-CFS and are guided by the following questions:

1 What experience of vocabulary learning did non-English majors have before the

1.4 Significance of the study

Even though several studies have been conducted on the integration of DDL and vocabulary learning in the world, few have been reported in Vietnam’s context Thus, theoretically, this study contributes to existing knowledge of the successful the DDL implementation in language education in general and vocabulary learning in particular

Practically, it is hoped that the findings of this study would offer empirical evidence for the effects of DDL on non-English majors’ vocabulary learning in TOEIC reading Also, this study takes the form of action research and thus is significant in providing the present researcher with deeper insights into non-English majors’ vocabulary learning in her teaching context, NLU-CFS, and how the current problems are solved through DDL The research results may perhaps serve as a valuable reference source for those with similar teaching and learning issues in Vietnam’s context

1.5 Scope of the study

The study’s focus is on the effects of using DDL in the enhancement of English majors’ vocabulary learning in TOEIC Reading instead of TOEIC Listening even though TOEIC classes at NLU-HCM cover both The findings are confined to vocabulary learning rather than other language areas and skills Moreover, this study has only examined the students’ receptive vocabulary knowledge without productive knowledge because knowing how to speak and write the words is not relevant to TOEIC Reading

non-1.6 Organization of thesis chapters

The thesis consists of five chapters Chapter 1 presents an introduction to the study by stating its background and the problem statement This chapter also includes the aim, research questions, significance, and scope of the study Chapter 2 provides a review of related literature and the conceptual framework of the study Chapter 3 explains the methodology employed in the study and describes the research participants, learning materials and procedure, research tools, data collection, and data analysis procedure Chapter 4 discusses the findings from the data analysis Chapter 5 summarizes the main findings and the contributions of the study, as well as the present researcher’s self-reflections and a plan for her future actions Additionally, the limitations of the study and some recommendations for further research are offered in this chapter

CHAPTER 2 LITERATURE REVIEW

2.1 Overview of the TOEIC test and vocabulary acquisition

2.1.1 Overview of the TOEIC test

The TOEIC test is a standardized English proficiency test for non-native speakers It aims to assess the ability to comprehend and communicate in English as it

is used in daily life and working environments The results of the TOEIC test can be used for fulfilling school/university requirements and employment applications According to Ha and Gu (2020), the TOEIC test is the most common option of many Vietnamese universities for non-English major programs (p 4) Likewise, NLU-HCM also adopts the TOEIC benchmark as a requirement for students’ graduation While the original form of the TOEIC test only consists of Listening and Reading, there is a separate part of Speaking and Writing added by the Educational Testing Service (ETS) However, the scope of this study is confined to TOEIC Reading

Being the second part of the TOEIC Listening & Reading test, the TOEIC Reading test is 75 minutes long and consists of 100 questions presented in the multiple-choice format (ETS, 2015, p 3) Test scores can range from 5 to 495 points and can be interpreted into corresponding CEFR levels There are currently two versions of the TOEIC test, namely the old format before 2019 and the updated format since 2019, which differ in the number of questions in each sub-part of the test The old format is being employed in NLU-HCM and is broken down into three sub-parts as follows:

• Part 5: Incomplete Sentences – 40 questions

• Part 6: Text Completion – 12 questions

• Part 7: Single Passage: 28 questions; 7-10 reading texts with 2-5 questions each; Double Passages: 20 questions; 4 pairs of reading texts with 5 questions per pair

As previously mentioned in section 1.1, the role of vocabulary in language learning in general and in reading comprehension is indispensable Similarly, Trew

(2007), who is a trainer for the TOEIC test from Oxford University Press, points out that “vocabulary is arguably the single most significant factor in doing well on the test” (p 7) As such, learners should focus a lot on vocabulary learning to improve their TOEIC Reading scores

2.1.2 Vocabulary acquisition

Vocabulary acquisition is defined by González-Fernández and Schmitt (2017)

as “all the processes involved in learning lexical items (i.e., single words and formulaic language) in sufficient depth to be able to use them both productively and receptively,

by means of multiple incidental and intentional encounters with these items in varied contexts” (p 280) The critical issues of conceptualizing vocabulary knowledge are reviewed as follows

The first thing to look at is the distinction between vocabulary breadth and depth According to González-Fernández and Schmitt (2017), vocabulary breadth (or size) is the number of words a person knows and is often explored at the most basic knowledge

of form-meaning connection Meanwhile, vocabulary depth refers to how well those words are known It includes aspects such as collocations, associations, constraints on use, etc (pp 283-284) Many researchers have tried to describe the components of vocabulary depth, and the most comprehensive description so far comes from Nation (2013) (see section 2.1.2.2) As there are many aspects of word knowledge, all of them cannot be learned at the same time, but rather, each aspect may develop along a cline and may be gained in different degrees from each other This notion is identified by Schmitt (2000, 2010) as the incremental nature of vocabulary acquisition; that is, to fully acquire a word, learners need to be exposed to it at various times rather than in a single encounter These numerous encounters with target vocabulary can be possibly achieved through suitable recycling programs for consolidation of partially known words and enhancement of what is known about those words (Schmitt, 2010, p 38) This issue is also supported by Trew (2007) when he suggests TOEIC learners note the

“consolidation” as well as read as much as they can, including newspaper stories, business articles, even graded readers, to expand vocabulary knowledge Therefore, in this study, the DDL activities from the paper-based vocabulary tasks, the vocabulary practice exercises, and the vocabulary computer-based tasks to the consolidation games (see section 3.4.2) aim to provide multiple experiences with target vocabulary to the students

Secondly, extensive previous works on vocabulary have mentioned two approaches to vocabulary learning, particularly incidental learning and intentional learning As defined by Barclay and Schmitt (2019), the former “occurs when students are engaged in some other task and vocabulary learning is not one of the overt goals” while the latter is “the learning when (one of) the specified goal(s) is vocabulary acquisition itself” (p 807) They are complementary to each other’s weaknesses because intentional learning is focused and effective, but limited in terms of the number

of words (and word knowledge types) it can address while incidental learning is slow and untargeted, but beneficial for acquiring the ‘contextual’ types of word knowledge (Schmitt, 2010, p 40) For this reason, they should not be separated from each other but instead be included in a systematic approach In light of this issue, vocabulary in this study was taught through both incidental and intentional approaches More specifically, word knowledge was picked up when the students interacted and analyzed the concordance data, followed by the teacher’s correction of exercises after they fulfilled the tasks

2.1.2.1 Receptive and productive vocabulary

Vocabulary knowledge has been recognized to consist of receptive and productive vocabulary Receptive knowledge is “being able to understand a word is known as and is normally connected with listening and reading” while productive knowledge is “being able to produce a word of our own accord when speaking or writing” (Schmitt, 2000, p 4) Whether learners should learn vocabulary receptively or productively alone or combine these kinds of learning depends on what knowledge they

need to know Nation (2013) emphasizes that “it is more efficient to do receptive learning for receptive use, and productive learning for productive use,” and receptive and productive learning should not be combined unless learners need to use all the four skills of listening, speaking, reading, and writing (p 56) In fact, the DDL study by Boulton (2010) omits productive activities as they would have been less relevant to the learners’ primary concerns with the purely multiple-choice format of the TOEIC Reading test Therefore, as this study is confined to TOEIC Reading, vocabulary should only be learned receptively

2.1.2.2 Aspects of knowing a word

In Nation’s (2013) taxonomy, there are 18 different types of word knowledge categorized in three aspects: Form, Meaning, and Use, each of which contains both receptive and productive dimensions (pp 48-50) In detail, the Form aspect encompasses the word’s pronunciation (spoken form), spelling (written form), and any word parts that it is made of (such as prefix, root, and suffix) He explains that Meaning refers to the concept behind the word in different contexts and the related concepts that

it is associated with Use involves the grammatical functions of the word, collocations that frequently occur with it, and finally, any constraints on its use in terms of register, frequency, and so on As Barclay and Schmitt (2019) comment, Nation's listing of vocabulary knowledge components does not suggest that every dimension of all words known to the learners should be mastered Rather, it reminds learners that there are other potential aspects of knowledge apart from form-meaning connection and acts as

a pedagogical tool for setting the goals of a classroom activity or assessment (p 804)

In the current research context, the focus is on receptive knowledge, so a setting for vocabulary’ aspects is modified and presented in Table 2.1 below Given that the learners are not required to recognize the word form when they hear it and its constraints

on use in the TOEIC Reading test, the aspect Form – spoken and Use – constraints on use are excluded from this study The seven remaining seven vocabulary aspects are enhanced through DDL activities during the intervention

For assessment, although vocabulary knowledge is tested throughout three parts

of the TOEIC Reading test, it can be seen that Form, Meaning, and Use aspects are mainly focused on in Part 5 and Part 6 of the TOEIC Reading test Indeed, Trew (2007) elaborates that “success in these two sections is based on the depth and accuracy of the candidate's knowledge of the vocabulary, structure, and usage of English” (p 9) Additionally, in Part 7, such questions as “The word … in line … is closest in meaning to” aim to test learners’ knowledge of Meaning aspect and should be included for assessment In sum, questions of Part 5, Part 6, and vocabulary questions in Part 7 of the TOEIC Reading pretest and posttest are classified into Form, Meaning, and Use questions to compare the students’ vocabulary gains before and after the intervention (see section 3.5.1)

Table 2.1

Vocabulary aspects setting for the present study

Form written What does the word look like?

word parts What parts are recognizable in this word?

Meaning form and meaning What meaning does this word form signal?

concept and referents What is included in the concept?

associations What other words does this make us think of?

Use grammatical functions In what patterns does the word occur?

collocations What words or types of words occur with this one?

2.1.3 Vocabulary learning strategies

The last issue on vocabulary to be reviewed is vocabulary learning strategies It

is relevant to this study because the researcher would like to collect information on the students’ previously employed vocabulary learning strategies before the intervention Such a look at their learning strategies would suggest whether DDL is a new experience

to them and can be worth investing in

Schmitt (1997) compiled a detailed list of 58 different vocabulary learning strategies based on Oxford’s (1990) social, memory, cognitive and metacognitive categories The strategies are categorized according to two major stages of learning: (1) strategies for discovering a new word's meaning and (2) strategies for vocabulary consolidation once words have been encountered As DDL approach promotes discovery learning among the learners, it is definitely not associated with vocabulary consolidation strategies; thus, the second class of strategies is excluded Regarding discovery strategies, there are two forms of strategies, namely determination strategies and social strategies According to Schmitt (1997, 2000), the former are used when students discover the meaning of a new word without help from another person, while the latter are associated with learners’ interaction with other people to learn vocabulary However, this study only looks at determination strategies as they mention all kinds of data the learners can be driven to when encountering a new word

For determination strategies, as Schmitt (1997) comments, learners can discover

an unknown word’s meaning through bilingual dictionaries and monolingual dictionaries Besides, learners nowadays can learn from many sources of reference materials; for instance, lower-level learners in the current setting may find Google translate very familiar to their daily learning As a result, the researcher reworded the strategies into “using English definitions” and “using Vietnamese translation” as more general terms Learners may be able to learn the new word’s meaning by using pictures and gestures, and also from other sources such as objects and diagrams (Nation, 2013,

p 121) Additionally, structural analysis of its part of speech, root, or affixes can help learners gain knowledge of a new word Finally, another widely employed strategy is

to guess the new word’s meaning from context, in other words, to infer a word's meaning from the surrounding words From these descriptions, the researcher developed the list of vocabulary learning strategies in the pre-questionnaire (see Appendix 11A & 11B)

2.2 Data-driven learning (DDL)

2.2.1 What is DDL?

DDL is closely related to the concept “corpus,” so it is useful to understand what

a corpus is and what it actually looks like According to Schmitt (2000), “a corpus is a large collection or database of language, incorporating stretches of discourse ranging from a few words to entire books” (p 68) The language in a corpus, as stressed by Timmis (2015), is naturally occurring not in the form of isolated words or sentences randomly collected but instead consists of spoken and/or written texts (p 2) Traditionally, the user searches a corpus for a particular word or phrase via a concordancer (or a corpus analysis software), and the results are displayed in a list of concordance lines with the searched item in the middle of each line, which is known as KWIC (keyword in context) format To illustrate, Figure 2.1 presents a random selection of 20 concordance lines taken from the Corpus of Contemporary American English (COCA) for the search word “request” highlighted in green By examining these concordance lines, one can notice some useful information about the word Specifically, it has two parts of speech (verb and noun); however, the verb “request” can only be seen in the third and fourth concordance lines, meaning that this word is more frequently used as a noun Furthermore, on the left of the word, the noun ‘request’ collocates with a variety of verbs such as “refer, ask, have, voice, answer, turn down, forward, refuse, support,” as seen in concordance lines number 1, 6, 9, 11, 12, 13, 17,

18, 20 On the right, the noun “request” often occurs with prepositions “for, of, from” (concordance lines number 1, 2, 9, 12, 18, 20) In that manner, concordance data informs the actual patterns of language for learners to acquire

Figure 2.1 An example of corpus query results for the word “request” in COCA

Corpora have long been recognized by researchers and materials writers because

of their use as the basis for syllabus design, reference works, and teaching materials (Chambers, 2019, p 2; O’Keeffe, McCarthy, & Carter, 2007, p 21) The potential applications of corpora have then reached another level as an approach to language learning that promotes learners’ direct interaction with corpora when Johns put forward the term DDL in his studies (1986, 1991) The underlying feature of DDL, as Johns (1991) states, is inductive learning that requires the learner to discover the language rules from the examples and to move from data to generalization (pp 2-3) Hence, there

is a change in student-teacher roles, with the student now being more responsible for his or her learning and the teacher acting as research director and collaborator rather than a transmitter of knowledge DDL’s element of inductive learning (also referred to

as discovery learning by some researchers) makes learning motivating and fascinating,

and learners become autonomous “detectives” as in Johns’ motto “Every student a Sherlock Holmes.”

Since Johns’ pioneering work, several researchers have tried to define the term DDL, as summarized in Table 2.2

In early DDL studies, a popular technique has been to have learners autonomously conducting their queries through a corpus software to explore the target language features and patterns, as seen in the definitions by Gilquin and Granger (2010), McEnery and Hardie (2012), which stress on “using the tools and techniques of corpus linguistics” or “direct access to the corpus and a tool for searching it.” However,

in Johns’ (1991) words, the approach may not work well with “unintelligent, unsophisticated, and poorly-motivated” students In fact, not all learners, especially at lower levels of proficiency, can alone deal with the technological demands of traditional computer-based DDL since they already have to cope with new processes of sophisticated linguistic reasoning

Many researchers who advocate the DDL implementation among lower-level learners suggest that learners’ working autonomously with concordancers is not the primary concern (Boulton, 2015; Smart, 2014) As another option, teachers can prepare printed concordance data (known as paper-based materials) and offer more guidance (also called guided induction) to learners to make the process less autonomous and less challenging These two techniques are worth considering because the participants in this study are lower-level non-English majors (roughly pre-intermediate) and have little experience in using corpora Smart (2014) proposes two defining characteristics of DDL (Table 2.2); however, his definition cannot be applied in this study for two reasons First, his study aims to compare the effects of using corpus data as sources of language learning materials (inductive learning) and as reference resources (deductive corpus-based learning)1, while this study only employs inductive learning Second, the DDL activities in this study are sometimes teacher-led rather than student-centered all the time

Therefore, the present researcher would like to adopt the definition by Crosthwaite (2019), mentioning learners’ engagement with corpus data through paper-based and computer-based presentations of corpus data in the DDL implementation, based on which teacher-led and student-led activities are designed (see section 2.2.3) Also, this definition highlights the aim of such learners’ interaction with corpus data, which is to acquire statistical and contextual information about language in use

1 In inductive (also known as data-driven or discovery) learning, learners are asked to observe in corpus data the actual uses of lexical items and then discover or generalize the usage patterns of these items while in deductive corpus-based learning, learners often use corpus data to test or practice the lexicogrammatical rules or patterns

to boost noticing, namely input enrichment, which refers to learners’ repeated exposure

to the target structure over a period, and input enhancement which means using typographical methods such as bolding and color marking to emphasize the target structure Indeed, in the DDL implementation, input enrichment occurs when learners get access to concordance data and observe how the target vocabulary are used Also, when learners conduct corpus queries in a concordancer, the target vocabulary is highlighted and presented in the middle (see Figure 2.1) and that input enhancement is activated

Another important theory is constructivism, which views the acquisition of knowledge as “a dynamic process, with learners in the driving seat” (Flowerdew, 2015,

p 18) It is very consistent with DDL because learners have to discover and induce language rules independently while interacting with concordance data; thus, vocabulary knowledge is mainly acquired by learners’ active involvement in tasks instead of being passively transmitted from teachers Therefore, this kind of inductive learning helps learners develop cognitive skills in making hypotheses and inferences However, Kirschner et al (2006, as cited in Flowerdew, 2015, p 18) argue that constructivist learning may be too cognitively demanding to certain types of learners and suggest the adoption of a “scaffolding” approach based on a socio-cultural theory of learning Thus, the socio-cultural theory is brought into the discussion as the third theory related

to DDL

Flowerdew (2015) has summarized the socio-cultural theory developed by Vygotsky that knowledge is “co-constructed through collaborative dialogue and negotiation with guidance and support mediated by the teacher or students in the form

of scaffolding” (p 19) The scaffolding process relates to DDL because in some studies, especially when the participants are lower-level learners, teachers often offer guided induction to facilitate learners’ developing new understandings Moreover, the student-scaffolding can be recognized in pair or group discussions in many DDL classrooms

In such activities, students work together on interpreting corpus data, validating hypotheses, or formulating searches in a concordancer; thus, their acquisition of vocabulary knowledge is enhanced

The language learning theories discussed above have laid a theoretical foundation for the adoption of the DDL approach The design of DDL activities and materials in this study was informed by these theories and are thus discussed in the following section

2.2.3 Using DDL in EFL and vocabulary learning

In employing an approach to language learning, there is a need to consider its components to determine the design of the current study, which are conceptualized into the corpus data and DDL activities Thus, the following parts aim to discuss theoretical issues on these two components in EFL classroom and vocabulary learning

2.2.3.1 Corpus data

Types of corpus

In the DDL implementation, the first issue to look at is which corpus and concordancer are used to consult the language data There is a wide range of corpora available; however, the most common types of corpora being mentioned in language research are general corpora and specialized corpora According to Reppen and Simpson-Vlach (2020), general corpora (e.g., the BNC, the COCA, the Brown Corpus) represent language in its broadest sense and serve as widely available resources for baseline or comparative studies of general linguistic features (p 93) Such corpora are designed to be large, balanced, and composed of language samples from a wide range

of registers or genres; thus, users can obtain reliable and generalizable answers for

many queries on language In contrast, specialized corpora are often small and made for specific purposes They can be collections of texts from a particular field of expertise, produced in a certain setting, or produced by a particular group of language users (Römer, 2010, pp 21–22) Most general corpora are web-based concordancers, while specialized corpora can be accessed through offline corpus analysis software such

custom-as Antconc, Wordsmith Tools, and so on Although it is clear that each type of corpus and concordancer suits particular research purposes and contexts, they seem to have their weak points Braun (2007, as cited in Chambers, 2019, p 8) notes that general corpora are created as tools for linguistic research and not with pedagogical goals in mind, while language learners are usually not expert corpus users Specialized corpora may match learners’ needs and interests, but very few of them are made freely available for language teachers and learners

Many recent studies on DDL and vocabulary selected general corpora for their learners (Boulton, 2010; Geluso & Yamaguchi, 2014; Kaur & Hegelheimer, 2005), while several studies employed specialized corpora For instance, Karras (2016) and Allan (2010) self-compiled corpora of graded readers Chan and Liou (2005) had their learners consult bilingual Chinese-English corpora Binkai (2012) combined the BNC and a textbook corpus specially developed for non-English majors in China As there is

no available specialized corpus of TOEIC Reading for this study, it is reasonable to exploit corpus data from a general corpus to guide the students’ vocabulary learning

Corpus data presentation

The second critical issue is the presentation of the corpus data into based and paper-based format (hands-on DDL/hands-off DDL are alternative terms) Gabrielatos (2005) notes that learners have to directly access the corpus data via a concordancer in the first type, whereas the second type requires the teacher to consult the corpora and select appropriate concordance lines and prepare printouts for the students Apparently, to survive through computer-based DDL, which promotes “pure” inductive learning, learners need to have cognitive skills for linguistic reasoning and

computer-basic computer skills to conduct corpus queries Therefore, in the case of inexpert and lower-level learners, many researchers support using paper-based materials to reduce cognitive load for the students while they still benefit from interacting with authentic language data

When integrating DDL into vocabulary learning, most studies adopted computer-based presentation (Binkai, 2012; Chan & Liou, 2005; Geluso & Yamaguchi, 2014; Karras, 2016; Kaur & Hegelheimer, 2005), while printouts are chosen in few studies (Allan, 2010; Boulton, 2010) A comparative study by Boulton (2012) among A2-B1 level college students indicates that the difference in learning outcomes between these two formats is not significant in the short term Interestingly, he found that only computer-based work can make the most of long-term benefits such as noticing, language awareness, and autonomy because it requires students to formulate the queries themselves, sort the results, and analyze more data than paper-based activities Thus, in this study, it is practical to bring both paper-based and computer-based presentations into class

Manipulation

Additionally, the adoption of paper-based presentation often relates to the data manipulation performed by teachers According to Gabrielatos (2005), manipulation involves selecting particular concordance lines given to the learners because classroom time is not enough to analyze many examples (p 24) Several criteria for this selection have been proposed in the literature, including readability (the most difficult concordance lines are discarded); frequency (only the concordance lines illustrating the most frequent uses are kept), and usefulness (only those concordance lines that are judged useful are kept) (Kuo et al., 2001; Levy, 1990; Tribble, 1997, as cited in Gilquin

& Granger, 2010, p 4) Manipulation also consists of editing the concordance lines or removing too complicated sentences for the learners Nevertheless, pre-selection may cause the overusing of concordance data to teach the old textbook rule (Johns, 2002, p 110), while pre-edited data is criticized for undermining the authenticity advantage of

DDL and not preparing learners for the authentic language in the realities (Boulton, 2009)

Although data manipulation is still controversial, the above-mentioned based DDL studies involve manipulation in some ways In Allan’s (2010) study, the corpus data are drawn from her self-compiled corpus of Penguin graded reader texts to

paper-be given to B1 and B2 level groups of learners Similarly, Boulton (2010) selected the concordance lines from the BNC and put them in the printouts distributed to his students These examples indicate that manipulation is necessary for paper-based presentation, mainly to keep a clear focus and to provide learners with an accurate picture of the language patterns being taught (Gabrielatos, 2005, p 24); thus, it should

be applied in this study

2.2.3.2 DDL activities

DDL activities may be located along a wide spectrum ranging from teacher-led

to learner-led (Mukherjee, 2006, p 12) At the teacher-led end, the teacher designs relatively controlled concordance-based exercises such as cloze tests and gap-fills, which are more suitable for lower-level learners In contrast, learner-led activities associated with discovery learning are often for advanced learners as they have to browse a large amount of corpus data and explore the language patterns autonomously

It can be seen that some DDL studies on vocabulary are totally learner-led (Binkai, 2012; Geluso & Yamaguchi, 2014; Karras, 2016) as the students were free to choose which aspects of the target words to study (e.g., word parts, concepts and referents, collocations), then they presented their findings in speaking journals or portfolios and shared them in front of the class Other studies, namely Allan (2010), Boulton (2010), Chan and Liou (2005), Kaur and Hegelheimer (2005), involve characteristics of both teacher-led and learner-led end More specifically, the students were required to analyze the corpus data then answer questions on the target word’s part of speech, collocation, translation, and/or complete controlled exercises on a worksheet It seems that the choice of activities depends on learners’ learning goals and capabilities As the

participants in this study are lower-level learners, combining teacher-led and led learning is worth considering

As previously mentioned in sections 2.2.1 and 2.2.2, at the more teacher-led end, it is possible to create a less autonomous and less challenging inductive environment through a guided induction process in which the teacher employs scaffolding techniques as a facilitator or a consultant for their learners’ discovery learning The teacher does not present the language rules explicitly; rather, he/she directs the learners’ attention to relevant aspects in the input and asks guiding questions when they formulate their hypotheses about the language (Smart, 2014; Vyatkina, 2018) Guided induction can also take the form of student scaffolding (or peer-to-peer interaction) for mentoring and feedback in small groups, which has been shown beneficial in DDL (Flowerdew, 2015) As reported in Vyatkina’s (2016a, 2016b) studies on collocation learning, guided induction has contributed significantly to her students’ improved performance It is also mentioned in other studies by Binkai (2012) and Boulton (2010) in the form of teacher’s guiding questions and/or pairwork/group work, but not explicitly emphasized and discussed In light of this gap, both teacher scaffolding and student scaffolding are employed in this study to see what effects they would bring to the students’ vocabulary learning

2.3 Previous studies

Previous studies that answered research questions related to vocabulary learning are often comparative between using DDL and dictionary-based learning or rule-based learning (Allan, 2010; Karras, 2016; Kaur & Hegelheimer, 2005; Trang, 2019) Other studies were conducted on one or many experimental groups without control groups (Binkai, 2012; Boulton, 2010; Chan & Liou, 2005; Geluso & Yamaguchi, 2014)

The aims of these studies are mostly to find out (1) the effects of DDL on learners’ vocabulary achievement and (2) learners’ thoughts about the effects of DDL Concerning the effects of DDL, learners reported significant gains from DDL,

vocabulary selected from the existed coursebooks (Binkai, 2012; Karras, 2016; Trang, 2019), or target vocabulary prepared for international certificates such as TOEIC and Cambridge examinations (Allan, 2010; Boulton, 2010) In Kaur and Hegelheimer's (2005) study, the results show a not significant difference in the total score of academic vocabulary achievement between both groups

As for learners’ thoughts, data collected from questionnaires and interviews reveal mixed results A majority of participants from many studies found DDL activities helpful, meaningful and motivating for learning vocabulary (Binkai, 2012; Boulton, 2010; Chan & Liou, 2005; Geluso & Yamaguchi, 2014; Trang, 2019) In particular, students from Boulton (2010) rated DDL significantly higher overall than dictionary information when it came to usefulness and ease of use In addition, students from Bouton (2010), Chan and Liou (2005), Geluso and Yamamuchi (2014) expressed that they would continue to use DDL for their learning in the future In contrast, the treatment group in Kaur and Hegelheimer (2005) remained in their comfort zone and preferred to use the online dictionary instead of fully exploiting the concordance program In Binkai (2012), Geluso and Yamaguchi (2014), students responded that they would opt for a combination of concordancing and dictionaries Allan (2010) reports that higher-level learners generally appreciated the value of concordances more than lower-level learners, so their reactions depend on individual learning styles and preferences

Table 2.3 below summarizes the previous studies integrating DDL into vocabulary teaching and learning in terms of research issues, methodology, participants, and results

Table 2.3

Summary of previous studies

Allan (2010) Comparing the effects

of DDL's based tasks, dictionary tasks, and traditional learning tasks on lexical knowledge

concordance 3 concordance-based groups, 3 dictionary groups, 2 CGs -Tool: cloze test and word associates test (pretest and posttest), questionnaires to teachers and learners

58 adult students (level B1-C1) at

a language center in Ireland

-concordance-based tasks: deeper knowledge of the target words at higher level learners; dictionary tasks: a more positive effect among lower-level learners -Students’ reactions depend on individual learning styles and preferences

Binkai

(2012)

Students’ thoughts towards DDL in acquiring vocabulary taken from the textbook

-1 EG -Tool: post questionnaire (has open-ended questions)

89 non-English majors at a college in China

-Students’ thoughts were generally positive -some disadvantages of DDL: cannot fulfill the assigned task, unsatisfactory

concordancing software, hard to catch the word’s meaning

Boulton

(2010)

Comparing the effectiveness of DDL's paper-based materials, dictionary-based learning, and traditional teaching materials and practices

-1 EG (5 vocabulary items each for DDL, dictionary-based learning, and no treatment sessions)

-Tool: TOEIC Reading test (Part 5), questionnaire

62 non-English majors (level A2-B1) at an architecture school in France

-Items learned in DDL sessions showed the biggest improvement; paper-based materials can counter some DDL’s barriers

-Students were more favorable to DDL activities

Chan & Liou

(2005)

The effects of using Chinese-English bilingual concordancer

on verb-noun collocation learning

-1 EG (3 units with DDL, 2 units without DDL)

-Tool: pre-, post- and delayed retention tests; background &

evaluation questionnaire

32 non-English major at a university in Taiwan

-Students improved more on collocations taught with the concordancer; the retention effects took a longer time to demonstrate -Students held positive thoughts towards learning English V – N collocations via the concordancer despite having some

problems

Geluso &

Yamamuchi

(2014)

Students’ thoughtstowards the DDL approach to formulaic sequence learning

-1 EG -Tool: questionnaire, follow-up interview, reflection logs

30 students B2 level) at a private foreign language university in Japan

(A2 Students generally perceived DDL as having a positive effect on their collocation learning but had some complaints about DDL

Karras

(2016)

Comparing the effects

of using an online concordancer and an online dictionary in acquiring vocabulary taken from the textbook

-1 CG (online dictionary) and 1

EG (online dictionary and concordancer)

-Tool: pretest and posttest fill)

(gap-100 secondary students at an international school in Ho Chi Minh City, Vietnam

-A significantly overall higher results for the EG than the CG, a significantly marked increase in vocabulary observed

in the latter weeks of the experiment (long-term effect)

-1 CG (online dictionary) and 1

EG (online dictionary and concordancer)

-Tool: pretest and posttest (cloze, sentence-building, essay writing), pre- and post-

questionnaire

18 students (B2 level) at a Midwestern research university

-The EG did not significantly outperform the CG in the total score of the vocabulary activities

-The EG preferred to use the online dictionary instead of fully exploiting the concordancer

Trang (2019) The effects of DDL on

students’ learning of vocabulary taken from the textbook

-1 CG (textbook task-based instruction) and 1 EG (concordance task-based instruction)

-Tool: pre-, post- and delayed tests, 3 progress tests, interview

49 non-English majors (B1 level) at a language center

in Ho Chi Minh City, Vietnam

-The EG significantly outperformed the

CG in vocabulary learning and retention and showed more durable retention in the delayed test

-The EG had favorable thoughts towards DDL in increasing vocabulary awareness and learning autonomy; one student had a negative attitude

Note: CG: control group; EG: experimental group

Despite its theoretical advantages, the DDL implementation may cause many practical problems to the process of learning Students in previous DDL studies on vocabulary learning found analyzing the concordance lines demanding and time-consuming because the data contained many unfamiliar vocabularies, which made them unable to capture the precise meaning of a target phrase and use it in a pragmatically appropriate way It is hoped that such difficulties can be overcome by using printouts and providing more scaffolding from teachers While paper-based presentation and guided induction have not been widely adopted and emphasized in previous studies, as discussed

in section 2.2.3, this study hopes to fill this gap in the literature

Other gaps to be mentioned are related to the research setting and the language input Two studies were conducted in Vietnam’s context, namely Karras (2016) and Trang (2019), and only Boulton (2010) focuses on TOEIC vocabulary Thus, this study would be

a humble endeavor to report the use of DDL to improve non-English majors’ vocabulary learning in TOEIC Reading in Vietnam’s context

2.4 Conceptual framework

Figure 2.2 shows the conceptual framework of the present study After examining the students’ experience of vocabulary learning before the course, the researcher carried out an intervention to find out how much DDL improves non-English majors’ vocabulary learning in TOEIC Reading and their thoughts about the effects of DDL in vocabulary learning The study is grounded in three language theories: noticing hypothesis, constructivism, and socio-cultural theory The design of the DDL implementation involves two components, namely corpus data, and DDL activities For corpus data, the students worked with the concordance lines taken from a general corpus which were presented in paper-based and computer-based materials The researcher pre-selected these concordance lines so that only those frequent, useful examples suitable to the students’ proficiency levels were kept For DDL activities, both teacher-led and learner-led learning was combined in this study with the help of guided induction in terms of teacher scaffolding

and learner scaffolding These activities are intended to provide the students with multiple experiences with target vocabulary which were learned receptively throughout the intervention Seven vocabulary aspects from Nation’s (2013) conceptualization were set to

be the goals of DDL activities The students’ vocabulary performance was measured by conducting TOEIC Reading pretest and posttest, including Form, Meaning, and Use questions Finally, students’ thoughts about the effects of DDL were collected according

to three main themes adapted from previous studies (see section 3.5.2): the usefulness of DDL activities, the benefits of DDL to vocabulary aspects, and the drawbacks of DDL

Figure 2.2The conceptual framework of the study

by the qualitative results to help explain and develop a detailed picture of the students’ outcomes

3.2 Research site

NLU-HCM is a university that mainly focuses on agriculture, forestry, and fishery

in Vietnam By 2025, NLU-HCM’s vision is to strive for a multi-disciplinary university in education and research, with international standards in educational programs, making efficient contributions to national development

Regarding the undergraduate English program at NLU-HCM, non-English majors have to take a TOEIC placement test to be assigned to attend English 1 or English 2 classes

in their first year at university These are General English classes that equip students with general knowledge of English vocabulary, grammar, and skills to reach level B1 CEFR by the time of graduation As mentioned in section 1.2, non-English majors need to either