Interference and integration effects on auditory and visual information differences between chinese and english

2.2.1 Structural Differences between English and Chinese 2.2.2 Phonological Differences in Reading English and Chinese 2.2.2.1 Phonology in English 2.2.2.2 Phonology in Chinese 2.2.3 V

INTERFERENCE & INTEGRATION EFFECTS ON AUDITORY

& VISUAL INFORMATION:

DIFFERENCES BETWEEN CHINESE & ENGLISH

An Academic Exercise

Prepared for the

Faculty of Business Administration

In Partial Fulfillment of the Requirements for the

Master of Science (Management) Degree

By

TAN THOR LING

2002/2003

A BSTRACT

Written language is central to branding and marketing communications (Tavassoli & Han 2001) in modern society of consumerism, and the use of words and writing are key tenets in the development of brand identity for many products and services Brand identity may consist of language components such as a brand name itself, brand slogans and verbal advertising messages Majority of past advertising studies pertaining to memory have been conducted in an English-only context, implicitly assuming that advertising information is processed similarly across languages However, while Chinese logographic characters represent meanings, English script contains symbols that represent sounds (Tavassoli & Han 2001), and such structural contrasts between Asian and Western languages have been shown in separate studies to differentially affect how information is processed (Gathercole and Baddeley 1993; Zhou & Marslen-Wilson 1999)

Other factors that may potentially moderate the linguistic processing of a branding or advertising message also include the audio-visual representation of message and the interitem relevancy within a message Hence, the present study seeks to improvise on past research and reexamine linguistic script differences and their effect on auditory and visual processing under relevant identifier conditions

With China being increasingly recognized as a global economic force, it is an observation that many companies are keen to conduct commerce in the Asian region Knowledge about the discrepant impact of language on information-processing and memory representation is therefore

of practical interest to businesses interested in successfully penetrating the Chinese consumer market through effective branding and cross-language communication strategies

2.2.1 Structural Differences between English and Chinese

2.2.2 Phonological Differences in Reading English and Chinese

2.2.2.1 Phonology in English 2.2.2.2 Phonology in Chinese

2.2.3 Visual (Graphic) Differences in Reading English and Chinese

2.2.3.1 Visual Processes in Reading English

2.2.3.2 Visual Processes in Reading Chinese

2.2.4 Effects of Language Processing on Memory Representation

2.2.4.1 English Processing and Memory 2.2.4.2 Chinese Processing and Memory

2.3 Information Processing and Memory …….………

2.3.1 Integration

2.3.1.1 Integrating English with Auditory Information

2.3.1.2 Integrating Chinese with Auditory Information

2.3.2 Interference

2.3.2.1 Interference between English and Auditory Information

2.3.2.2 Interference between Chinese and Visual Information

2.3.3 Integration and Interference

2.4 Integration: Language and Visual & Audio Identifiers ………

2.5 Relevancy and Integration ………

2.5.1 Effects of Relevancy on Memory Representation 2.5.2 Effects of Relevancy on Information Processing 2.6 Interference: Language and Visual & Audio Identifiers ………

2.7 Summary ………

18 18 19 21 24 C HAPTER T HREE - M ETHODOLOGY 3.1 Introduction ………….………

3.2 Design ……… .………

3.3 Manipulation of Independent Variables (IVs) ……… ………

3.3.1 Selection of Attributes 3.3.2 Relatedness of Attribute-Identifier Pairs 3.4 Experimental Procedure … ………

3.5 Testing of Dependent Variables (DVs) ………

3.5.1 Retrieval Memory (Free Recall) 3.5.2 Relational Memory ( Fair Recognition) 3.6 Manipulation Check ………

27 27 27 28 30 34 37 37 37 38 C HAPTER F OUR – A NALYSIS 4.1 Introduction ………

4.2 Covariates ………

4.3 Integration & Relational Memory (Recognition) ………

4.4 Interference & Retrieval Memory (Recall) ………

40 40 40 45 C HAPTER F IVE – D ISCUSSION 5.1 Introduction ………

5.2 Findings ………

5.3 Theoretical Implications ………

50

50

51

Chapter One

Introduction

1 INTRODUCTION

1.1 MOTIVATION OF STUDY

Written language is central to branding and marketing communications (Tavassoli & Han 2001) in modern society of consumerism, and the use of words and writing are key tenets in the development of brand identity for many products and services For example, brand identity

commonly consists of language components such as a brand name itself (e.g Nike) and brand slogans (e.g Nike’s “Just Do It”)

With Asia emerging as a major economic force - in particular reference to China and its entry into the World Trade Orgnization (WTO) - at the turn of the century, many business organizations are waking up to the fact that approximately one-quarter of the world’s population communicates in a language that is starkly different from the modern, alphabetic English script that American-influenced branding is familiar with Many advertising studies pertaining to memory have been largely conducted in single-language – typically English – conditions, implicitly assuming that consumers around the globe process verbal brand information identically across tongues Such a presumption may be erroneous: while Chinese logographic characters represent meanings, modern English scripts contain symbols that represent sounds (Tavassoli & Han 2001), and such structural dissimilarities between Asian and Western languages have been shown in separate studies to differentially affect how information is processed (Gathercole and Baddeley 1993; Hung & Tzeng 1981; Zhou & Marslen-Wilson 1999) and represented in memory (Schmitt, Pan & Tavassoli 1994) Thus, knowledge on the discrepant impact of language on information-processing and memory representation is of practical interest

to foreign businesses keen on successfully penetrating the Chinese consumer market through

Naturally, language elements are not the only considerations in the creation of a successful brand identity Impactful brand identifiers also often include the use of logos – or

visual representations of a brand name (Shennan 1986), such as the Nike “SWOOSH” design – and jingles or auditory cues to a brand name (Croft 1999), such as the Nike “SWOOSH” sound

Advertisements – whether in print or in multimedia form – often consist of both verbal (i.e language) and visual (e.g pictures, logos, etc) or audio accompaniments (e.g voices, jingles, etc) Hence, investigation into the possible interactions between language and audiovisual brand identifiers is also critical to provide an accurate analysis of varying linguistic effects on memory representation

1.2 FURTHERING PAST RESEARCH

A couple of past studies have accounted for the influence of linguistic differences between alphabetic and logographic script on interactions with audio-visual stimuli and mnemonic performance In a cross-cultural experiment utilizing Chinese and American students from Beijing and Chicago respectively (Schmitt, Pan & Tavassoli 1994), it was showed that memory recognition will be differentially affected by whether Chinese or English words were learned auditorally or visually In another noteworthy experiment conducted on monolingual

Koreans - with language differences manipulated through the use of alphabetic Korean Hangul and logographic Korean Hancha (Tavassoli & Han 2001) – it was demonstrated that memory for

Chinese and English words will be affected by whether an audio or visual brand identifier was matched to the word

Although research findings have proved remarkable, this study aims to improve upon their inherent limitations that arise from the necessary use of monolingual subjects in the two

above-mentioned studies The first limitation is the fact that cross-cultural experiments may be somewhat limited by the host of factors that may differ between populations; differences that naturally occur when subjects in a single study consist of both Chinese speakers living in China and English speakers living in the United States These factors comprise of educational differences that can affect memory rehearsal strategies, cultural factors that can affect the evaluation of certain product characteristics as well as responses to nonverbal audiovisual

stimuli Secondly, single-language manipulations (e.g comparisons within Korean Hangul and

Hancha) may not prove as conclusive or as generalizable as across-language manipulations

In contrast to monolinguals, proficient bilinguals rely on different scripts to access a single conceptual system (Francis 1999; Chen & Leung 1989) Proficient bilinguals have also been found to comprehend written languages significantly faster than monolinguals, and to be more advanced in terms of memory representation (Bialystok 2003) The use of bilingual subjects’ will therefore produce a more accurate and true assessment of findings in any experiment involving language processing and memory representation In addition, the use of bilinguals’ living in the same country also simultaneously controls for cross-cultural differences while allowing for across-language manipulations In Singapore, Chinese students are educated

in both English and Chinese tongues from the time they are 6 years of age Hence, in this study, bilingual 16-year-old high school students (i.e have received a minimum of 10 years learning instruction in both the English and Chinese language) were roped in for participation in the experiment

Both the earlier studies mentioned also made use of “nonsensical” stimuli - such as odd shapes and meaningless sounds – in their experimental setup However, practically speaking, very rarely do business organizations and advertisers utilize (i) nonsensical, meaningless brand

identifiers, or (ii) irrelevant stimuli in their marketing communications Thus, while much of past research has partially-restricted real-world marketing and branding implications, this study seeks

to provide further business-managerial considerations through modified experimentation of past studies with the improvised use of relevant identifiers

1.3 RESEARCH OBJECTIVES

Marketers often struggle to find a dynamic combination of words and pictures and/or sounds that will convey their overall advertising message powerfully and effectively to their target audience Naturally, choice words and verbal messages are often paired with relevant,

meaning-laden information for greater impact For example, BMW car advertisements often seek

to communicate the idea of a BMW being “The Ultimate Driving Machine”, and their television advertisements typically pair this verbal message with both visual sights of a sleek BMW cruising

on the road and auditory sounds of an engine roaring powerfully as the BMW speeds past on the

screen

Hence, the present study seeks to reexamine linguistic script differences and their effect

on auditory and visual processing under relevant identifier conditions It is hoped that findings will produce a realistic representation of the true effectiveness of day-to-day marketing communications, and provide practical managerial recommendation in the development of cross-language advertising communication strategies

Chapter Two

Literature Review

2 LITERATURE REVIEW

2.1 INTRODUCTION

This study basically seeks to determine the impact that processing of marketing stimuli – with variations in language, brand identifiers, relevancy of attribute-identifiers and product category - will have on bilingual audiences; as such, an in-depth review of psycholinguistic and memory integration-interference literature will be discussed in this chapter

2.2 LANGUAGE PROCESSING

Due to inherent structural differences between English alphabetic script and Chinese logographic characters, there are contrasting degrees of phonological (audio) and orthographical (visual) processes involved in the reading of words in either language These disimilarities also affect the route by which English and Chinese words are represented in memory

2.2.1 S TRUCTURAL D IFFERENCES BETWEEN E NGLISH & C HINESE

Like most languages, English relies on an alphabetic script where letters represent sounds The English language only consists of 26 letters, which can be put together to create an

infinite number of words In such alphabetic writing systems, meaningless basic symbols (i.e

letters or alphabet) are made more or less corresponding to pronunciation units (i.e words) in speech (Chen, Flores d’Arcais and Cheung 1995) based on sounds

Chinese is a form of logographic writing system in which the characters represent basic units of meaning (Chen, Flores d’Arcais and Cheung 1995), with at least 7,000 distinct and unique logographic representations available (Tavassoli 1999) 82% of Chinese characters fall

two separable subcomponents (called radicals): the semantic radical usually provides a categorical cue to the meaning of a whole character, while the phonetic radical sometimes

provides clues to the pronunciation of the whole character (Zhou 1978; Xu, Pollatsek & Potter 1999)

2.2.2 P HONOLOGICAL D IFFERENCES IN R EADING E NGLISH AND C HINESE

Reading English is more dependent on phonological sound-based processes than in reading Chinese, largely because of the inherently pronunciation-linked composition of the English language:

2.2.2.1 Phonology in English

Due to the the linguistic structure of the English language, the written form and its pronunciation are linked by rules because the sequence of letters that make up an English word corresponds to specific sounds (Schmitt, Pan & Tavassoli 1994) As such, English spelling is inextricably coupled with phonemic identity (Schmitt, Pan & Tavassoli 1994), and the English alphabetic writing system thus allows quick access to phonological information from orthography (Feng, Miller, Shu & Zhang 2001) Many studies on English word recognition have shown early, automatic phonological activation when English words are presented to subjects in isolation (Frost 1998, for review) Hence, phonology plays a central role in the processing of English

2.2.2.2 Phonology in Chinese

Even though most Chinese chacters are prototypically picto-phonetic (Yin & Rohsenow 1994) – earlier decribed as consisting of one component that represents pronunciation and a second component that represents meaning - the validity of the phonological cues from phonetic radicals are frequently low (Feng, Miller, Shu & Zhang 2001) It has been estimated that only 26% to 39% of phonetic radicals accurately predict the pronunciation of the whole Chinese character (Zhou 1978; Gao, Fan & Fei 1993) This loose graphic-phonemic correspondence that

is characteristic in the Chinese language (Schmitt, Pan & Tavassoli 1994) means that orthographic, or structural, cues to phonology are ambiguous (Tavassoli 1999) and less useful in Chinese than in English (Feng et al 2001)

In addition, the Chinese language consists of a large number of homophones - or words that sound the same but have different meanings (Tavassoli & Han 2001) – such as to, too, two in

English As testimony to this, although Chinese contains 300 times as many character symbols as English, the former language utilizes only approximately 400 syllables compared to 4,000 in

English (Xindai Hanyu Pinyin Cidian / Modern Chinese Dictionary 1997) The extensive

number of homophones present in the Chinese vocabulary implies that many characters share the same pronunciation (Tan, Hoosain & Peng 1995) and that there is rarely a unique linkage from a single character to a syllable for pronunciation (Feng et al 2001), thus further suggesting that phonology is not very helpful in distinguishing Chinese characters (Aaronson and Ferres 1986)

2.2.3 V ISUAL (G RAPHIC ) D IFFERENCES IN R EADING E NGLISH AND C HINESE

Reading Chinese is more contingent on visual processes than in reading English, mainly due to the visually-logographic formation of basic Chinese units (i.e characters)

2.2.3.1 Visual Processes in Reading English

As earlier established, reading English is primarily a sound-based process In English writing, homophonic mistakes such as inserting the word “vein” for “vain” often go unnoticed by test subjects (Daneman & Stainton 1991), thus demonstrating that visual processes are of lesser importance than auditory processes in the reading of English script The role of visual processing

in English may be thought of as being limited to a shallow identification of English orthography, that after initial recognition allows a phonological process to convert written strings to phonemic strings (Feldman & Turvey 1983; Frost, Katz & Bentin 1987; Katz & Feldman 1983)

2.2.3.2 Visual Processes in Reading Chinese

The Chinese language comprises of basic units (i.e characters) that are composed via a visual arrangement of strokes with ideographic origins (Schmitt, Pan & Tavassoli 1994) The visual similarity between these Chinese characters are low (Feng et al 2001), hence Chinese readers tend to depend more on identification of the unique visual features of Chinese logographic characters in reading (Aaronson and Ferres 1986) With this inclination to reliance

on physical orthographic cues of the language, Chinese readers can build up strong graphic-form semantic associations which are quite deterministic (Perfetti & Tan 1998)

In addition, because syllable information is incomplete and unsystematic (i.e low phonetic prediction rate), a Chinese reader can - in principle - read words by going directly from graphic

writing units to meanings (Perfetti & Tan 1998) Reading Chinese has been established to be a

visual form-to-meaning process (Baluch & Besner 1991; Frost 1994), and this visual dependency

in reading Chinese has been demonstrated in past experiments that have shown Chinese consumers to be more sensitive to visual features of written Chinese words (Tavassoli 2001)

2.2.4 E FFECTS OF L ANGUAGE P ROCESSING ON M EMORY R EPRESENTATION

The structural and linguistic differences between English and Chinese will naturally affect the mental representations of information, and in turn, influence consumer memory (Schmitt, Pan & Tavassoli 1994)

2.2.4.1 English Processing & Memory

Due to the greater involvement of acoustics inherent in the characteristic of the English language, readers of English tend to sub-vocalize – or phonologically recode – written words (McCusker, Hillinger and Bias 1981) and rehearse words using a sound-based mental code in a subsystem of short-term memory known as the phonological loop (Baddeley 1986; Pavio 1986) Processing the alphabetic English script relies more heavily on the storage and serial rehersal properties of short-term memory’s phonological loop (Tavassoli and Han 2001) In contrast to Chinese processing, routine English language processing does not appear to reply on visual short-term memory, other than through mental imagery (Gathercole and Baddeley 1993)

2.2.4.2 Chinese Processing & Memory

Readers of Chinese are able to mentally access concepts unmediated by sub-vocalization, although the activation of pronunciation may be immediate and can also mediate the activation

of meaning (Perfetti and Zhang 1991) In other words, readers’ association with pronunciation when processing Chinese words is arbitrary, and unsystematically acquired via rote associative learning (Tavassoli and Han 2002) Hence, reading Chinese is not dominated by sound-based processes, and phonemic recording in processing Chinese is used much less, if at all (Sasanuma 1973)

In contrast to the sound-based dependence involved in the processing English, reading Chinese appears to rely on visual processes to a greater degree than reading English (Hung & Tzeng 1981, Zhou & Marslen-Wilson 1999) The processing of characters written in such logographic scripts relies more on the storage of and the spatial relational rehearsal properties of visual short-term memory (Tavassoli & Han 2001), and Chinese characters are encoded visually and mapped onto images directly

2.3 INFORMATION PROCESSING AND MEMORY

Depending on the degree of shared or separate mental resources required for various processing activities, an individual’s performance in undertaking two tasks simultaneously

would be affected: by whether common resources could be integrated for efficient processing (Wickens 1984), or whether competing demands on resources would interfere with performance

(Kinsbourne and Hicks 1978)

2.3.1 I NTEGRATION

There has been much research examining the effects of the integration of words with acoustic and visual information (Massaro 1996; Treisman 1988) To generalize, the degree to which efficient integration between features of an item can occur is when there is a shared

directness between the processes that are used to encode the features (McClelland 1996) For example, it has been shown that words that had been presented in the same modality (either spoken or written) were better memory cues than words that were presented in different modalities (Penney and Butt 1986) Similarly, consumers performed better at a pair-recognition task involving integration of brand names and product categories in memory, when these were learned in the same modality than in different modalities (Tavassoli 1998) In addition, associative memory in coordination tasks – like when a pilot needs to integrate visual, auditory and verbal information to make a cognitive decision – has been found to be greater when there is

a larger processing overlap (Yee, Hunt and Pellegrino 1991)

In summary, relational memory (memory which involves integration of two distinct items) should be stronger the more two stimuli rely on similar encoding processes (Tavassoli and Han 2001)

2.3.1.1 Integrating English with Auditory Information

As earlier discussed, reading English relies largely on phonological or accoustic mechanisms (Schmitt, Pan & Tavassoli 1994); hence, the cognitive resources required for the processing of English are similar to mental resources required for auditory processing It therefore follows that auditory stimuli should be integrated into memory more strongly with the alphabetic English (Tavassoli & Han 2001) For example, it has been shown that in English, mistakes and decrements have been observed in the recall of accoustically similar verbal stimuli, but not visually similar verbal stimuli (Baddeley 1981; Rubenstein, Lewis & Rubenstein 1971), and English consumers were found to be more sensitive to auditory cues such as a speakers’

voice (Pan & Schmitt 1996) Thus, the greater reliance on phonological resouces faciliates integration between two separate pieces of verbal-English and sound-based information

2.3.1.2 Integrating Chinese with Visual Information

Similarly, because reading Chinese depends heavily on visual processes, cognitive resources required for processing both Chinese and graphic information coincide with each other Hence, visual logos should be integrated into memory more strongly with logographic Chinese (Tavassoli and Han 2001) It has been found that memory for the print color of Chinese brand names was greater than in English (Tavassoli 1999), and that Chinese subjects were more sensitive to visual cues such as font type (Pan & Schmitt 1996) Past experiments have also shown that regardless of whether words were learned auditorily or visually, Chinese speakers were able to recollect them better by writing it down; researchers have suggested that the attempt

to write the Chinese characters serves as a prime for the Chinese logographs’ visually-integrated memory code (Schmitt et al 1994) Hence, Chinese characters are more easily integrated with

visual information in memory due to the overlap in visual processing resources involved

2.3.2 I NTERFERENCE

The reciprocal effect of information integration may be thought of as interference (Penney 1989; Tavassoli 1998), and interference may be regarded as whatever impedes an individual from effectively processing a stimulus (Wickens 1984) When stimuli compete for attention - such as in a dual task setting – the interference generated is greater the larger the overlap in cognitive resources required for effective processing (Tavassoli & Han 2001) For example, it has been shown that auditory ads interfere more with recall memory for competitive

auditory ads than with recall memory for visual ads (Unnava and Sirdeshmukh 1994) In contrast, it is easier to perform two competing tasks concurrently when one is auditory and the other is visual (i.e because separate mental resources are expended in the processing of audio and visual information), than when both are in the same modality (Tresiman and Davies 1973)

2.3.2.1 Interference between English and Auditory Information

Based on the assumptions underlying the described interference theory, the overlapping phonologically-based resources required for the processing of both audio information and English will result in greater memory interference between auditory stimuli and the English alphabetic script (Tavassoli & Han 2001) Past experiments have demonstrated that acoustic distractors in English stimuli will interfere more with the processing of verbal than pictorial information (Duncan 1980), and that auditory elements will interfere more with the learning of and cognitive responding to English than Chinese ad copy due to the modality-specific (i.e phonological) overlap in mental resource requirements (Unnava, Agarwal, and Haugtvedt 1996)

2.3.2.2 Interference between Chinese and Visual Information

Due to the greater dependence on visual resources in the processing of logographic scripts like Chinese, visual stimuli will show more interference in memory with Chinese than with English (Tavassoli & Han 2001) Research has found that interference in naming a words’ print color is greater if the conflicting color word (e.g., the word “BLUE” is visually printed in the ink-color red) is written in logographic Chinese than alphabetic English (e.g., Biederman & Tsao 1979), thus suggesting that perceptual or visual interference is higher in Chinese – as opposed to English – conditions Other supporting experiments in the same vein have shown that

the use of visual symbols produced more interference in the naming of Chinese characters than

in the naming of English words (Chen & Tsoi, 1990), and that Japanese native speakers also

showed more interference when color names were written with logographic Kanji characters than with phonemic Kana script (Morikawa 1981)

2.3.3 I NTEGRATION & I NTERFERENCE

In summary, memory integration may be thought of as a process that occurs when mnemonic encoding is facilitated by the presence of overlapping of features between two distinct items (McClelland 1996) Relational memory may be thought of as memory formed as an association between two separate pieces of information, and has been used in past research as an experimental measure of integration (Tavassoli & Han 2001) In the authors’ experiment, subjects were shown combinations of visual words, and a visual or auditory stimulus Relational memory was tested by soliciting subjects’ paired-recognition of word-stimulus pairs that were later shown to subjects again Hence, paired-recognition is directly proportional to integration; recognition is higher the more two stimuli rely on similar encoding processes (Du Plessis 1994)

As opposed to item recognition which does not depend on memory retrieval processes, individual capacity for free recall is lower when competitive information is presented in the same modality (Tavassoli 1998) Hence, recall may be considered to be inversely proportional to interference: the higher the occurring level of memory interference, the lower the memory recall capacity of an individual Recall tasks involve asking consumers to report what they have seen without any cues (Du Plessis 1994), and may therefore be used as an indirect measure of interference

2.4 INTEGRATION: LANGUAGE AND VISUAL & AUDIO IDENTIFIERS

Based on the literature reviewed above, it is apparent that alphabetic scripts are processed

in the phonological loop to a greater degree than logographic words are; hence, auditory cues should facilitate the encoding of memory relations between English words and auditory brand identifiers In contrast, the greater reliance on visual processing in the processing of logographs should make easier the coding of memory association between Chinese logographs and visual brand identifiers In previous research involving a monolingual Korean population (Tavassoli & Han 1999), it has been demonstrated that auditory identifiers are integrated in memory more

strongly with alphabetic script (i.e Korean Hangul script), while visual logos are integrated in memory more strongly with logographic words (i.e Korean Hancha characters)

In replication of the above findings on a bilingual Singaporean sample, it is therefore proposed that:

H1: When stimulus identifiers are neutral - or possess no related or meaningful association –

with respect to the target words,

(a): Relational memory between English words and auditory identifiers should be stronger than

with visual identifiers, and

(b): Relational memory between Chinese words and visual identifiers should be stronger than

with audio identifiers

F IGURE 2.1

R EPRESENTATION OF H YPOTHESIS H1

2.5 RELEVANCY & INTEGRATION

Relevancy may be defined as the extent to which material pertains directly to the meaning

of the theme and reflects how the information contained in the stimulus contributes to or detracts from the clear identification of the theme of primary message being communicated (Goodman 1980)

2.5.1 E FFECTS OF R ELEVANCY ON M EMORY R EPRESENTATION

Past research has shown that congruent items are easily comprehended and integrated with previous expectations (Heckler and Childers 1992), while the use of irrelevant items lead to minimal development of ‘inter-episode’ linkages, resulting in limited information-processing that

makes irrelevant items least likely to be integrated As such, meaningful associations can

enhance memory (Lutz and Lutz 1997; Schmitt, Tavassoli & Millard 1993) and facilitate the encoding of thematic information (Meyers-Levy 1991) For example, it has been demonstrated

Chinese (H1b)

Visual Identifier

Audio Identifier

Relational Memory

English (H1a)

Visual Identifier

Audio Identifier Relational

Memory

that ads containing relations among the ad elements – such as brand name, copy and pictures – were better recalled than those containing unrelated ad elements (Schmitt, et al 1993)

In short, when elements of a stimulus set complement other items within the set, memory representation is reinforced due to the facilitation of inter-item integration (Pomerantz 1981) This directly means that relevant stimuli can be easily associated with other information presented in the same message, and that relevant pairs can be more effortlessly recalled and recognized

2.5.2 E FFECTS OF R ELEVANCY ON I NFORMATION P ROCESSING

As the use of relevant pairs generally allows for enhanced integration in memory, it follows that when relevant identifiers are used in conjunction with Chinese logographs or English script, it should greatly facilitate encoding of target words in either language It is therefore predicted that the presence of relevant identifiers should reduce effects of processing advantages between English-Audio and Chinese-Visuals - over English-Visual and Chinese Audio respectively - that would otherwise be displayed in a neutral condition

In other words, due to the greater reliance on phonological-audio processing involved in the reading of English words (Schmitt, Pan & Tavassoli 1994), relational memory strength between English words and auditory stimuli can be assumed to be inherently superior to relational memory strength between the former and visual stimuli As relational memory for association between English words and audio objects is already naturally stronger, it is proposed that introducing relevant identifiers – or identifiers that possess related or meaningful

associations to the target word – should not significantly enhance relational memory between

English words and audio stimuli

On the other hand, relational memory for pairings between verbal English and visual objects – with the former relying on phonological processes and the latter on visual processes - is generally weak as there is no overlap in encoding processing (Tavassoli & Han 2001) It is therefore logical to hypothesize that relational memory will be significantly improved for target English words that are paired with relevant visual stimuli Hence:

H2: The incremental effects of relevancy on relational memory will be significantly higher under

English-Visual than English-Audio conditions

Likewise, with greater dependence on visual processing involved in the reading of Chinese words (Hung & Tzeng 1981), relational memory strength between Chinese words and visual stimuli can be given as innately better enhanced in comparison to relational memory strength between Chinese words and audio stimuli It is similarly suggested that introducing the

variable of relevancy will not facilitate relational memory between Chinese logographs and

visual identifiers, whereas relational memory has greater room for significant improvement for Chinese words that are paired with relevant visual stimuli

H3: The incremental effects of relevancy on relational memory will be significantly higher under

Chinese-Audio than Chinese-Visual conditions

F IGURE 2.2

R EPRESENTATION OF H YPOTHESIS H1, H2, H3

2.6 INTERFERENCE: LANGUAGE AND VISUAL & AUDIO IDENTIFIERS

Recall the earlier discussion on memory interference which in summary maintains that, the more the stimuli and the verbal target overlap in the cognitive resources required for their processing, the more interference the auditory or visual stimuli causes (Tavassoli and Han 2001)

As alphabetic scripts such as English are processed in the phonological loop to a greater degree than logographic characters (e.g Chinese), audio distractors should interfere more with the processing of English than that of Chinese Similarly, visual distractors should interfere more with logographic Chinese words as they are processed in visual short-term memory to a larger degree (Tavassoli and Han 2001)

As opposed to item recognition which does not depend on memory retrieval processes, individual capacity for free recall is lower when competitive information is presented in the same modality (Tavassoli 1998) Hence, it follows that memory retrieval - or free recall - for

Visual Identifier

Audio Identifier

Audio Identifier

Relational Memory

Neutral Relevant

Chinese

H3

H3

phonologically-dependent English words would be negatively affected by the presence of auditory identifiers, while retrieval for visual-based Chinese words would be adversely influenced by visual identifiers Therefore, it is proposed that:

H4: When stimulus identifiers are neutral - or possess no related or meaningful association –

with respect to the target words,

(a): Free recall of English words that are presented with auditory identifiers should be weaker

than when presented with visual identifiers, and

(b): Free recall of Chinese words that are presented with visual identifiers should be weaker than

when presented with audio identifiers

F IGURE 2.3

R EPRESENTATION OF H YPOTHESIS H4

English (H4a)

Visual Identifier

Audio Identifier

Free

Recall

Chinese (H4b)

Visual Identifier

Audio Identifier Free

Recall

Further recall that the presence of relevant stimuli typically has the advantage of enhancing memory (Schmitt, et al 1993) on the whole Hence, incorporating the element of relevancy should reduce the negative effects of memory interference between English-Audio and Chinese-Visual pairs In other words, any processing deficiencies under English-Audio and Chinese-Visual conditions should be offset by the processing facilitation that the use of relevant stimuli provides

With the adverse influence of memory interference on phonologically-based English with audio stimuli, retrieval memory for English words that are associated with audio objects is weak

at its base-level Thus, when English words are associated with relevant audio stimuli, there is greater room for significantly enhancing free recall for the relevant pair However, as performance on retrieval memory for English words that are paired with visual stimuli is inherently high even in a neutral, status quo condition, free recall of English words associated with relevant visual stimuli has lesser extent to produce any significant improvement in retrieval memory

H5: The incremental effects of relevancy on retrieval memory will be significantly higher under

English-Audio than English-Visual conditions

Likewise, as there is greater reliance placed on visual processing in the reading of Chinese words, the level of interference that occurs between Chinese words and visual stimuli at status quo is higher As relevant stimuli generally have the effect of improving memory performance, there is potential for significant facilitation of retrieval memory when Chinese logographs are associated with relevant visual identifiers In contrast, as the neutral-condition

level interference between Chinese words and audio stimuli is low (i.e retrieval memory is already high in a neutral condition), free recall performance for Chinese words has lesser room for improvement even when paired with relevant audio stimuli Hence:

H6: The incremental effects of relevancy on retrieval memory will be significantly higher under

Chinese-Visual than Chinese-Audio conditions

thus follows that while phonologically-based English script can be efficiently integrated with

Visual Identifier

Audio Identifier

Free

Recall

Neutral Relevant

English

H5

H5

Free Recall

Visual Identifier

Audio Identifier

Neutral Relevant

Chinese

H6 H6

acoustic information due to the coinciding of processing resources needed for information

encoding, audio information can also just as effectively interfere with the memory encoding of

English words due to the overlap of cognitive resources needed Similarly, the processing of Chinese characters and visual information is identically influenced in memory representation

Information integration may be experimentally measured by relational memory (or,

associative recognition of two distinct items shown simultaneously), and interference measured

by free recall based on pure retrieval memory Hence, relational memory or integration is

hypothesized to be higher for English-Audio and Chinese-Visual conditions (H1), while retrieval

memory (as an inverse reflection of interference) is predicted to be lower for English-Audio and

Chinese-Visual conditions (H5) As past research has shown that relevant conditions typically

produce results of general memory enhancement across the board, it is also suggested that the relevancy of language-stimulus pairings serves as a moderating factor that will vary the

language-identifier findings from a neutral condition (H2, H3, H5, and H6)

Chapter Three

Methodology

3 METHODOLOGY

3.1 INTRODUCTION

This section will discuss the manner in which the various independent variables (i.e language, identifier-modality, attribute-identifier relevancy and product category) were manipulated in this study, as well as the process by which the dependent variables of paired-recognition and free recall were measured Much of the general experimental setup and procedure was largely based on previous experiment methodology – with modifications to suit the purpose of this study – utilized in Tavassoli & Han 2001

3.2 DESIGN

The study used a 2 (Language: English versus Chinese) x 2 (Modality of Brand Identifier:

Visual versus Auditory) x 2 (Relevancy of Attribute-Identifier Pair: Relevant versus Neutral) x 2

(Product Category: ‘Car’ versus ‘Dinner’) mixed-subjects full-factorial design The first three treatments were operationalized as between-subjects factors, while Product Category was

included as a within-subjects replication factor

3.3 MANIPULATION OF INDEPENDENT VARIABLES (IVs)

The variable of Product Category was incorporated into the experiment as a replication factor in order to enhance generalizability of the potential findings The two categories – Dinner and Car – were chosen based on qualitative assumptions about their divergent categorical

characteristics, yet general familiarity to the target subjects This section thus describes the method in which the required verbal attributes, audiovisual identifiers and relevancy of attribute-

identifier pairs – corresponding to both Dinner and Car categories – were manipulated in the

study

3.3.1 Selection of Attributes

A set of 35 general attributes describing (i) a restaurant dinner experience, and (ii) features of a car was initially generated by a group of 5 bilingual Chinese undergraduate students A final list of positive and negative attributes was then selected from this larger set via pretest with 20 bilingual students, different from the initial group and the main experiment Pretest subjects were told to imagine a restaurant dinner experience (i.e the first product category) and to rate each attributes’ positivity (1 = Very negative; 7 = Very positive) and importance (1 = Not at all important; 7 = Very important) in influencing their judgment about the dinner experience

Using the pretest findings as a quantitative benchmark, a list of attributes that were considered relatively easy to portray with both visual and audio identifiers were qualitatively handpicked The final set of positive dinner attributes chosen was rated as equally positive in Chinese (X = 5.48) and English (X = 5.47), and as equally important in Chinese (X = 4.55) and English (X = 4.53) The set of negative dinner attributes was rated as equally negative in Chinese (X = 3.23) and English (X = 3.32), and as equally important in Chinese (X = 4.55) and English (X = 4.53)

Thereafter, this evaluation procedure was repeated by asking subjects to consider the features of a car The final set of positive and negative car attributes selected was rated as equally positive and negative in Chinese (X = 5.58; X = 3.34 respectively) and English (X = 5.56; X =

3.33 respectively), and both positive and negative attributes were rated as equally important in Chinese (X = 4.45; X = 4.44 respectively) and English (X = 4.43; X = 4.45 respectively)

Hence, both the Car and Dinner categories comprised of 3 to 4 positive (e.g enjoyable)

and negative (e.g unreliable) attributes each This mixed exposure of positive and negative attributes per category was intended to control for any potential favorability or impression-based bias or influence Table 3.1 displays the list of 7 English language attributes used in the experiment - together with their Chinese language equivalents - by corresponding product category

T ABLE 3.1

M ANIPULATION OF L ANGUAGE (E NGLISH VERSUS C HINESE) , P RODUCT C ATEGORY

(“C AR” VERSUS “D INNER”) & P RODUCT A TTRIBUTES

3.3.2 Relatedness of Attribute-Identifier Pairs

For the experiment, each of the final 7 selected attributes in both categories had to be paired with both a relevant audio and visual identifier in the experiment Four to five audio and visual identifiers that could be considered potentially relevant to each of the target attributes were first selected from an expansive source of ready-made stimuli The perceived relatedness of attribute-identifier pairings were then determined via a second pretest (n = 20) on a group of students, different from both the earlier pretest and the main experiment The second pretest group of subjects were exposed to each of the potential attribute-identifier pairs (both visual and audio), and asked to rate the degree of relevancy between each of the pairings on a 7-point Likert scale (1 = Very irrelevant; 7 = Very relevant) These students were also requested to rate the favorability or likeability of each of the attribute-identifier pairs on a similar 7-point Likert scale (1= Very unfavorable; 7 = Very favorable)

All the 14 final selected attribute-identifier pairs for use in the experiment (i.e 7

attributes-identifiers each for both the Dinner and Car category) scored a minimum

attribute-identifier relevancy mean of X > 5.80 There was also no significant difference between the selected attribute-identifier pairs in terms of favorability (X = 5.32; p > 0.5) Table 3.2 and Table 3.3 describe and display the relevant audio and visual stimuli paired with each of the 7 attributes

in both the Dinner and Car categories

Car engine being revved-up

by driver, like before a professional car race starts

6 FRUSTRATING

Many cars horning impatiently (like in a bad traffic situation)

7 SLUGGISH

Car engine igniting noisily in spatters and vibrations (giving the impression of a very old car)

Each of the 7 attributes in both categories was also paired with a neutral visual and audio identifier Neutral identifiers were chosen based on their use as visual and audio distracters in past research (Tavassoli & Han 2001) Visual identifiers were shape formations that had no

inherent meaning, while audio identifiers were “sound effects that could not be easily labeled verbally” (Tavassoli & Han 2001) Table 3.4 displays the neutral visual stimuli paired with each

of the 7 attributes in both the Dinner and Car categories The same neutral identifiers were used

across both categories

Neutral audio identifiers were

“sound effects that could not be easily labeled verbally”

These neutral audio identifiers were selected based on their use

in past research (Tavassoli & Han 2001)

3.4 EXPERIMENTAL PROCEDURE

One hundred and sixty secondary school students from the Holy Innocents’ High School participated in the study as part of a school fund-raising activity Subjects were randomly assigned to one of the 8 treatment groups, and were run in groups ranging in size from between

17 to 21 students over a two-day period There were 18 to 20 usable responses collected for each experimental condition

The study was conducted in a controlled computer laboratory experiment at the school itself Each subject was seated at an individual computer terminal - equipped with personal headsets - in the lab Each computer terminal was preset for manipulation of a different experimental condition, thus facilitating the randomization of subjects across experimental conditions (but within the same language condition) There were instructions for the subjects on the computer screen, and the experimenter also verbally reinforced these instructions in the same language, Chinese or English, as that being tested in those particular conditions

Subjects were told that they would be watching a slideshow in which they would be shown words describing aspects of a car or dinner (i.e product attributes), and that there would also be a visual image or auditory sound accompanying this presented word (i.e identifiers) They were then explicitly instructed to try to remember the words as well as the associated sounds or visual images as they would later on be asked to recognize both the verbal attribute and audiovisual identifiers presented to them in the slideshow

The students were informed that the purpose of the activity was to test the ability of secondary school students to remember new information presented to them In addition, subjects were cautioned that recognition of the attribute-identifier pairs would be a difficult task to perform as the words and identifiers that would be shown to them later in the object recogntion

task would be very similar to those viewed in the slideshow Such priming was intended to increase subjects’ interest in - and attention to - the activity at hand

After all instructions were dispensed, subjects were instructed to tap “Enter” on the keyboard to initiate the start of the slideshow Attribute-identifier pairs were presented on subjects’ computer screen as shown in Table 3.5 The attributes were displayed in bold, capital letters at the top-middle of the screen across all experimental conditions Visual identifiers were presented directly below the attribute printed on the screen Audio identifiers were played simultaneously – through individual students’ headphones - with the presentation of the attribute-word on the computer screen

Subjects were exposed to each of the attribute-identifier pairs for 3 seconds This length

of exposure was deemed to be sufficient based on pretest results that utilized experimental stimuli and a sampling frame similar to the main experiment In this pretest, subjects (n=20) were instructed to look at a stimulus presented on the screen before responding to a simple,

T ABLE 3.5

E XPERIMENTAL S TIMULI L AYOUT ON C OMPUTER S CREEN

ROMANTIC