How input enhanced authentic videos support English listening comprehension: A discussion from an interactionist perspective

With the light that literature has shed on the merits of authentic videos, this paper aims to foreground two video input enhancement activities, namely annotating and captioning and argue that when embedded in authentic videos, annotations and captions aid EFL learners’ vocabulary acquisition and thus English listening comprehension.

1 INTRODUCTION

Technologies seem to have come to aid listening

skill earlier and with greater diversity than the

other three macro English skills Robin (2011) lists

broadcasts, tape recorders, and talking pictures as

some of the electronic mediators to the teaching of

listening skill before 1975 Since the 1980s, these

devices have been either substituted or made more

sophisticated, or naturally both, by an unending

source of authentic audiovisual materials, which

in turn came along with the advent of the Internet

and Web 2.0 However, as authentic materials

were historically created by and intended for

native speakers in contrast to those created for

pedagogical purposes, they are highly challenging

* Hanoi University of Science and Technology,  nghia.nx89@gmail.com

Received: 20/4/2019; Revised: 14/5/2019; Accepted: 17/5/2019

in terms of grammatical, lexical, genre, and cultural contents (Garrett, 2009) Authentic videos are thus dismissed as less appropriate for learners at levels other than intermediate or above (Guariento & Morley, 2001) One may consider this a valid account for language teachers’ attempts

to intentionally reduce the difficulty of authentic materials, but as Taylor (1994) puts it, materials can only be truly authentic when language is simplified by no means With this understanding, the paper highlights annotation and caption as two video input enhancement techniques that help create pedagogic videos without altering their authenticity First concepts around authenticity and its benefits for listening comprehension are summarized Then literature is reviewed on two

ABSTRACT

With the light that literature has shed on the merits of authentic videos, this paper aims to

foreground two video input enhancement activities, namely annotating and captioning and argue

that when embedded in authentic videos, annotations and captions aid EFL learners’ vocabulary

acquisition and thus English listening comprehension To this end, annotations and captions are

discussed on the theoretical grounds of Multimodality and the Interactionist Theory of Second

Language Acquisition (SLA) The paper concludes with implications for language teachers as to

the use of input-enhanced authentic videos for educational purposes in the listening classroom

Keywords: listening comprehension, authentic videos, input enhancement, multimodality,

interactionist theory

HOW INPUT-ENHANCED AUTHENTIC

VIDEOS SUPPORT ENGLISH LISTENING

COMPREHENSION: A DISCUSSION

FROM AN INTERACTIONIST PERSPECTIVE

theoretical tenets – multimodality and Interactionist

SLA – on which the discussion of annotation and

caption is based At the end, recommendations are

made for language teachers in regards to the use of

authentic videos for pedagogical purposes in their

listening classroom

2 AUTHENTICITY

Though authenticity was soon a subject of

ample discussion in the 1970s as a result of the

Communicative Language Teaching approach,

it has remained ambiguous in the field of

applied linguistics It is the concern over the

multifacetedness of authenticity – whether it is

associated with the text per se, the interaction

between the teacher and the learner, the tasks

chosen, etc (Gilmore, 2007) – that has led

scholars yet to address it fully For example,

Wallace focuses on the material aspect and

defines authentic materials as “real life texts, not

written for pedagogic purposes” (1992, p.145),

while Tomlinson accentuates the task, referring

to an authentic task as “one which involves the

learners in communicating to achieve an outcome,

rather than to practice the language” (2013, p.19)

Morrow’s (1977) definition is used in this paper to

summarize those varying lens: “An authentic text

is a stretch of real language, produced by a real

speaker or writer for a real audience and designed

to convey a real message of some sort” (p.13)

Authentic materials can be classified in

different ways On the grounds of modality,

Genhard (1996) classifies authentic materials into

three categories: authentic audiovisual materials,

which integrate both pictures and sounds;

authentic visual materials, which are image-based

and wordless; and authentic printed materials,

which are presented on paper Concerning mainly

authenticity level of the text, Campos (1992)

proposes three types: authentic text, adapted or

simplified text, and creative text An authentic

text is not simplified or modified to any degree,

and represent language used by native speakers in

genuine situations for the fulfilment of their social

needs An adapted or simplified text is characterized

by the simplification of language for the purpose

of grammar and vocabulary introduction and reinforcement And, a creative text is created on the basis of the sequence of contents covered in a textbook This view is shared by Geddes and White (1978), who align Campos’s (1992) paradigm

to only two categories of discourse: unmodified authentic discourse and simulated authentic discourse, with the former being “authentic text” while the latter encompassing “adapted text” and

“creative text” The diagram below represents types of authentic materials in light of the three mentioned approaches

Figure 1: Types of authentic materials (synthesized from Genhard, 1996; Campos, 1992;

and Geddes & White, 1978)

Authentic materials have been widely proved beneficial for L2 listening comprehension in a few ways First, learners yield constant exposure

to real-life situations that are unachievable inside the textbook-based classroom Scholars such as Gilmore (2004) highlight the cultural contextualization of learning as the unique value

of authentic materials, particularly authentic videos Second, learners have access to incremental linguistic gains This is enabled by frequent repetitions of lexical, grammatical and phonological features made by native speakers in

an unexaggerated fashion (Devitt, 1997), which

in turn strengthens the learner’s cognitive load and thus communicative competences (Sweller

& Chandler, 2007) Also, a body of research (e.g Melvin & Stout, 1987; Christopher & Ho, 1996)

has converged that authentic materials produce

an inspirational and motivational environment

for listening In brief, the advantages of authentic

materials provide an impetus for the replacement

of purpose-written materials In the sections below,

literature around two concepts foundational to

the discussion in this paper – Multimodality and

Interactionist SLA – is reviewed

3 MULTIMODALITY

Kress and Van Leeuwen define multimodality as

“the use of several semiotic modes in the design

of a semiotic product or event, together with the

particular way in which those modes are combined

– they may for instance reinforce each other, fulfil

complementary roles or be hierarchically ordered”

(2001, p.20) It is the number of representational

modes involved in presenting a piece of information

that differs multimodality from unimodality,

i.e the use of one single representational mode

(within the scope of this paper, bimodality, i.e the

use of two representational modes, is viewed a

form of multimodality) If the traditional listening

classroom relies mostly on aural texts, the cohesion

of aural and visual materials has been in vogue,

allowed by an abundant source of videos These

sorts of materials also mirror the complementary

nature of multimodality in that one semiotic

representation carries one set of meanings while

another a different set (Guichon & Cohen, 2016)

In the listening comprehension classroom,

multimodality is a notion that is inseparable

from audiovisual materials for the latter’s

representational essence, so the benefits of videos

are shared by multimodality In his 2014 study,

Woottipong stated that language teachers showed

a particular liking for videos since multiple

modalities were more appealing than audio-only

instructions This view is echoed in Oura (2001)

who found that audiovisuals engaged, motivated,

and captured the learners’ attention significantly

better than unimodal texts Another set of benefits of

multimodality concern paralinguistic nuances, that

is how intonation, pitch, prosody, and other non-verbal expressions align with the meanings being made This is because learners are immersed into

a whole range of communication situations from numerous contexts (Harmer, 2007) According

to Jones and Plass (2002), learners who were exposed to aural and visual listening outperformed counterparts given audio-only listening, and these individuals also bore a lower cognitive load

4 INTERACTIONIST SECOND LANGUAGE ACQUISITION

Amongst an array of theories underlying SLA, e.g Universal Grammar Theories, Cognitive Theories, Sociocultural Theories, Interactionist Theory is of paramount importance Originally, interaction is described by Ellis (1999) as

“the interpersonal activity that arises during face-to-face communication”, and also as

“the intrapersonal activity involved in mental processing” (p.3) The former type of interaction refers to direct conversational exchanges between two interlocutors, in which the less competent one (usually the learner) requests for the more competent counterpart’s (usually the native speaker) acts of repeating, explaining, modifying

or enhancing linguistic inputs whenever communication breakdowns occur (Gass & Selinker, 1994) The latter is essentially about the kind of cognitive activity within the learner’s mind that engages deep mental processing in the wake of episodes of meaning negotiation

or input modification requests Research shows that making linguistic inputs (lexis and syntax) and paralinguistic inputs (stress and intonation) salient during conversational interactions is how the more proficient interlocutor not only keeps the conversation going in a direction that mutual understanding is attained (Pica, 1994) but also facilitates delayed developmental effects on the learner’s cognition (Gass, 1997)

Within the domain of CALL, Chapelle (2005) has reshaped the concept of “interpersonal

interaction” by adding a “between person

and computer” facet to the existing “between

people” one She also notes three benefit groups

of interaction as opportunities for negotiation of

meaning, acquiring enhanced input, and focusing

attention on linguistic form, and aligns them to three

types of interaction as shown in the table below:

Table 1: Alignment of types of interaction to their

benefits (adapted from Chapelle, 2003)

Interpersonal

interaction

Between people (learner – native speaker) Negotiating meaning Between person and

computer (learner - computer)

Acquiring enhanced

or modified input

Intrapersonal

interaction Within the person (learner)’s mind Focusing attention on linguistic form in

the input

How interaction with computer promotes

L2 acquisition in light of Chapelle’s (2005)

conceptualization is indexed in the learner’s

exposure to enhanced input displayed on the

computer screen During this process, the learner

notices input forms and makes form-meaning

mappings, then gradually internalizes those

connections via a course of intrapersonal activity

This standpoint by Chapelle provides thrust for a

good deal of research For example, Borrás and

Lafayette (1994) found that accompanying aural

input with L2 subtitles in listening lessons offered

learners input of two modes, to the latter which

they referred to decode the former in terms of not

only general messages but also linguistic forms

and meanings In another study, Grace (1998)

put learners in contact with L1 translations and

annotations of different types (e.g written and

visual) to aid comprehension of vocabulary, and

concluded that those enhanced inputs remedied

miscomprehension and prompted noticing This

line of research has also been extended to the area

of reading comprehension, with Plass et al (1998)

providing multiple forms of annotations to help with

vocabulary and proving their encouraging effects

on learners’ acquisition These findings mirror

the need and also potential for the introduction

of Interactionist SLA into instructional design Given Long’s (1991) argument about the nature of the Interactionist Theory that only when noticed and comprehended can input become valid, instructional materials or CALL tasks should be designed in a way that key linguistic features are made salient, and enhancement of linguistic input

is offered (Chapelle, 2007) In the next parts, I provide definitions of the two input enhancement methods – annotation and caption, and discuss them on the grounds of Multimodality and Interactionist SLA to illuminate how they support listening comprehension

5 ANNOTATION AND CAPTION

Annotations are explanatory notes added to demystify the meaning of an unknown word and come in three fundamental forms: written, visual/ pictorial, and audiovisual (Jones, 2004) Written annotations can be L2 definitions or L1 translations

of the target word Visual/pictorial annotations, the representation in imagery, and audiovisual annotations, the support with both pictures and sounds, may potentially overload listeners if embedded in an existing audiovisual layer and seem to be a more appropriate for multimedia reading texts, so are not within the scope of the current paper

Captions refer to “on-screen text in a given language combined with a soundtrack in the same language” (Markham & Peter, 2003, p.332) It is important to distinguish captions from subtitles which refer to “on-screen text in the same language of the viewers that accompany the second language soundtrack of the video material” (ibid.) In alignment with the nature of authentic videos as previously mentioned, that is language input should not be simplified or modified, the term “captions” is employed to address the matter under examination in this study Guillory (1998) classifies captions into two types: keyword captions and full captions, with the former highlighting

only words that are either targeted or tough in

their meaning, pronunciation, or morphological

structure by themselves, and the latter displaying

the entire spoken text

6 DISCUSSION

6.1 Annotated and captioned videos are

multimodal

One of the arguments considered in the present

paper is that annotated and/or captioned videos are

multimodal In fact, a typical video is multimodal

for its aural and visual representations (with

the understanding that bimodality is a form of

multimodality as mentioned earlier) A written

annotation, when added to a video, makes it

threefold in modality, i.e aural, visual, and textual

In a similar vein, when captions, be they keyword

or full, are inserted into a video, they radically

switch it from a piece of two-modality material

into that of three Figures 2 and 3 below illustrate

this It is also important to note that annotations

and captions can appear synchronously in a same

video, and language teachers can use different

dynamics, depending on their specific aims For

example, instead of presenting the textual content

in form of either annotations or captions, they may

do so with both by providing keyword captioning

of a lexical item on the bottom of the screen and

its definition or translation in an annotation bubble

on the top

Integrating annotations and captions into videos clearly signifies the plurality of semiotic modes in accordance with Kress and Van Leeuwen’s (2001) definition of multimodal materials These modes consolidate one another and provide complementary and mutual support as

in the examples above: the annotated definition of

“the Celts” and the captioned keyword “immense” (the textual modality) synchronize with the speaker’s speech (the aural modality) This in turn generates an eminently interactive product that aids learning to a great extent as indicated by literature regarding multimodality, i.e linguistic gains, paralinguistic subtleties, and motivational factors (e.g Woottipong, 2014; Harmer, 2007; Jones & Plass, 2002)

6.2 Annotated and captioned videos promote listening comprehension from the perspective of Interactionist SLA

The other argument made about annotated and captioned videos in this paper is that they offer comprehensible input to facilitate listening comprehension According to researchers (e.g Kim, 2015), L2 listening comprehension is a complex process, involving two processes: a down process and a bottom-up process In the top-down process, learners mobilize their “schemata

or background knowledge” to gain the gist and main ideas of the aural text, while with bottom-up processing, they are drawn to discrete words and phrases to decipher the meaning and content (Kim,

2015, p.16)

Figure 2 Annotation of “the Celts” Figure 3 Keyword captioning of “immense”

Regarding top-down processing, annotated

videos serve well on this, in that the annotated

content goes beyond explanation of a lexical word to

the level that is directed at activation of background

knowledge and scaffolding of comprehension

To the best of my knowledge, this is lacking in

previous studies, so purposely made prominent

in this report For example, in a video comparing

the size of the Sun with that of other stars in the

universe, I created an annotation with the question

“Is the Sun the largest star in the universe?” to

elicit the learners’ prior knowledge about this

matter and prepare them for what was coming next

in the video (Figure 4) This conforms to Kim’s

(2015) analysis of the top-down listening process

that processing prior knowledge paves the way for

learners’ “grasp of incoming information” (p.16)

Bottom-up processing is catalyzed by

interactions with both annotations of individual

words/phrases and captions With annotations,

the matter of concern is whether definitions or

explanations should be provided in L1 as in Figure

5 or L2 as in Figure 2 In actuality, the speed of

an authentic video would not afford listeners time

and space to read L2 definitions, so L1 translations

seem to be the ideal option This can be elucidated

by a body of research about the inextricable link

between L2 words, L1 translations, and pictures

(e.g Paivio, 1986) Take a further look into Paivio’s

dual coding theory for an example The theory

postulates that there are two information channels

in the learners’ mind: an L1 verbal channel and

a nonverbal/imagery channel The L2 verbal channel is formed and developed owing to the fact that the L2 word is connected to the image and L1 verbal subsystem existing in the learner’s mind This can be further illustrated from Figure 5, that

is by the time listeners hear the word “the Celts” and the image depicting these tribes pop up on the screen, the L1 translation “người xen-tơ” allows them to assimilate “the Celts” with its meaning thanks to their consciousness of the L1 and visual cues Captions provide half of this route, in this regard, as far as keyword captions are concerned This means that when hearing and seeing the captioned word and the corresponding image on the screen, listeners make an interconnection and thus the form-meaning mapping In fact, literature has proved that watching videos with keyword captions made it easier for learners to understand the content for decreased cognitive load which would otherwise be heavy under full caption condition (Guillory, 1998)

Above all, the described forms of annotations and captions result in interactive experiences between the learners and the computer rather than between the learners and native speakers

as originally stipulated by Interactionist theory

of SLA Put it differently, they constitute a technological platform for the learners to interact with enhanced inputs and benefit from them for lexical acquisition and comprehension of aural texts (Chapelle, 2003)

Figure 4: Annotation of a question Figure 5: Annotation of “the Celts”

7 CONCLUSION AND IMPLICATIONS

The departure point for this paper is that authentic

videos are beneficial for second language learners

in listening comprehension, fundamentally

thanks to genuine situations and linguistic

and paralinguistic elements they provide On

this premise, the paper revisits annotation and

caption as two input enhancement activities

that can be performed on authentic videos, and

argues that they further aid learners’ linguistic

gains and thus listening comprehension abilities

From its discussion grounded on the theories of

Multimodality and Interactionist SLA, the paper

arrives at the following practical implications for

language teachers who intend to use authentic

videos in their listening classroom

[1] Videos should be opted for with the

perception that their authenticity is not reduced on

any level

[2] Annotations should expand from

explanations and/or definitions of individual words/

phrases to posing of questions for prior knowledge

activation and comprehension scaffolding as

well When annotations are intentionally aimed at

individual lexical items, they should be done so in

L1 as opposed to L2

[3] Captions should be supplied in key words

rather than in full for optimal learning experiences

Perhaps, an experimental study is needed in the future

to genuinely test the effect of these suggestions./

References:

Borrás, I., & Lafayette, R C (1994) Effects of multimedia

courseware subtitling on the speaking performance of

college students of French Modern Language Journal,

78(1), 61-75

Campos, R O (1992) Authenticity in listening and written texts

LETRAS, 1(25), 169-194

Chapelle, C A (2003) English language learning and

technology: Lectures on applied linguistics in the age of

information and communication technology Amsterdam:

John Benjamins Publishing

Chapelle, C A (2007) Technology and second language

acquisition Annual Review of Applied Linguistics, 27,

98-114

Chapelle, C A (2009) The relationship between second language acquisition theory and computer-assisted

language learning The Modern Language Journal, 93,

741-753.

Chapelle, C.A (2005) CALICO at center stage: Our emerging rights and responsibilities CALICO Journal, 23(1), 5-16 Christopher, E., & Ho, S (1996) Lights, camera, action:

exploring and exploiting films in self-access learning Taking Control: Autonomy in Language Learning, 185-200

Devitt, S (1997) Interacting with authentic texts: Multilayered

processes The Modern Language Journal, 81(4), 457-469.

Ellis, R (1999) Learning a second language through interaction

Amsterdam: John Benjamins.

Garrett, N (2009) Computer-assisted language learning

trends and issues revisited: Integrating innovation The Modern Language Journal, 93, 719-740.

Gass, S, M (1997) Input, interaction, and the second language learner Hillsdale, NJ: Erlbaum

Gass, S M., & Selinker, L (1994) Second language acquisition: An introductory course US: Lawrence

Erlbaum Associates

Geddes, M., & White, R (1978) The use of semi-scripted simulated authentic speech in listening comprehension

Audiovisual Language Journal, 16(3), 137-145

Genhard, J., G (1996) Teaching English as a foreign language: A teacher self-development and methodology Ann Abor: The university of Michigan Press

Gilmore, A (2004) A comparison of textbook and authentic

interactions ELT Journal, 58(4), 363-374

Grace, C (1998) Retention of word meanings inferred from context and sentence-level translations: Implications for

the design of beginning-level CALL software The Modern Language Journal, 82(4), 533-544

Guariento W., & Morley, J (2001) Text and task authenticity in

the EFL classroom ELT Journal, 55(4), 347-353

Guichon, N., & Cohen, C (2016) Multimodality and CALL In

F Farr, & L Murray (Eds.), The Routledge Handbook of Language Learning and Technology (509-521) Routledge

Guichon, N., & McLornan, S (2008) The effects of multimodality

on L2 learners: Implications for CALL resource design

System, 36(1), 85-93.

Harmer, J (2007) The practice of English language teaching

UK: Pearson Education

Jones, L (2004) Testing L2 vocabulary recognition and recall

using pictorial and written test items Language Learning &

Technology, 8(3), 122-143

Jones, L C., & Plass, J L (2002) Supporting listening

comprehension and vocabulary acquisition in French with

multimedia annotations The Modern Language Journal,

86(4), 546-561.

Kim H S (2015) Using authentic videos to improve EFL

students’ listening comprehension International Journal of

Contents, 11(4), 15-24

Kress, G., & Van Leeuwen, T (20015) Multimodal discourse:

The modes and media of contemporary communication

London: Oxford University Press.

Long, M (1991) Focus on Form: A Design Feature in Language

Teaching Methodology In K De Bot, R Ginsberg, & C

Kramsch (Eds.), Foreign Language Research in

Cross-Cultural Perspectives (39-52) Amsterdam: John Benjamins.

Markham, P., & Peter, L (2003) The influence of English

language and Spanish language captions on foreign

language listening/reading comprehension Journal of

Educational Technology Systems, 31(3), 331-341.

Melvin, B S., & Stout, D S (1987) Motivating language

learners through authentic materials In W Rivers (Ed.),

Interactive Language Teaching (44-56) New York:

Cambridge University Press.

Morrow, K (1977) Authentic texts and ESP In S Holden (Ed.),

English for Specific Purposes (13-17) London: Modern

English Publications

Oura, G K (2001) Authentic task-based materials: Bringing

the real world into the classroom Sophia Junior College

Faculty Bulletin, 21, 65-84

Paivio, A (1971) Imagery and verbal processes New York:

Holt, Rinehart and Winston

Pica, T (1994) Research on negotiation: What does it reveal about second language learning conditions, processes,

and outcomes? Language Learning, 44(3), 1-35

Plass, J L., Chun, D., Mayer, R., & Leutner, D (1998) Supporting visual and verbal learning preferences in

a second-language multimedia learning environment

Journal of Educational Psychology, 90(1), 25-36

Robin, R (2011) Listening comprehension in the age of Web

2.0 In N Arnold & L Ducate (Eds.), Present and Future Promises of CALL: From Theory and Research to New Directions in Language Teaching (93-108) CALICO.

Sweller, J., & Chandler, P A (2007) The effect of written text on comprehension of spoken English as a foreign language

American Journal of Psychology, 120(3), 237-261.

Taylor, D (1994) Inauthentic authenticity or authentic

authenticity? TESL-EJ, 1(2) Retrieved from http://www-

writing.berkeley.edu/TESL-EJ/ej02/a.1.html Tomlinson, B (2013) Introduction: Are materials developing?

In B Tomlinson (Ed.), Developing Materials for Language Teaching (2nd ed.) London: Bloomsbury Academic

Wallace, C (1992) Reading Oxford, UK: Oxford University Press.

Winke, P., Gass, S., & Sydorenko, T (2010) The effects of captioning videos used for foreign language listening

activities Language Learning & Technology, 14(1), 65-86.

Woottipong, K (2014) Effect of Using Video Materials in the Teaching of Listening Skills for University Students

International Journal of Linguistics, 6(4), 200-212

SỬ DỤNG VIDEO ĐỜI THỰC CÓ CHÚ THÍCH, PHỤ ĐỀ ĐỂ NÂNG CAO KỸ NĂNG

NGHE HIỂU TIẾNG ANH: THẢO LUẬN TỪ GÓC ĐỘ CỦA LÝ THUYẾT TƯƠNG TÁC

NGUYỄN XUÂN NGHĨA Tóm tắt: Với việc các nghiên cứu trước đã chỉ ra lợi ích của video đời thực đối với kỹ năng

nghe hiểu Tiếng Anh, bài viết này chứng minh rằng chú thích và phụ đề khi được đưa vào video

đời thực sẽ giúp người học cải thiện hơn nữa vốn từ vựng và kỹ năng nghe hiểu của mình Bài

viết cũng sẽ thảo luận tác dụng của hai hình thức này trên cơ sở lý thuyết về Đa phương thức và

Thuyết tương tác của Lý thuyết thụ đắc ngôn ngữ thứ hai Ở phần cuối là một số gợi ý cho giáo

viên khi sử dụng video đời thực có chú thích và phụ đề vào mục đích giảng dạy

Từ khoá: nghe hiểu, video đời thực, chú thích, phụ đề, đa phương thức, lý thuyết tương tác

Ngày nhận bài: 20/4/2019; ngày sửa chữa: 14/5/2019; ngày duyệt đăng: 17/5/2019