A study on experiential meaning in “snow white and the seven dwarfs” from multimodal perspectiv

SystemicFunctional Grammar formulated by Halliday 2004 is used for analyzing theexperiential meaning of the verbal texts in the collected data resources, whileVisual Grammar originated b

QUY NHON UNIVERSITY

VÕ THỊ ĐĂNG NGỌC

A STUDY ON EXPERIENTIAL MEANING IN

“SNOW WHITE AND THE SEVEN DWARFS” FROM MULTIMODAL PERSPECTIVE

Field : The English Language Code : 8.22.02.01

Supervisor: Võ Duy Đức Ph.D.

TRƯỜNG ĐẠI HỌC QUY NHƠN

VÕ THỊ ĐĂNG NGỌC

NGUYÊN CỨU NGHĨA KINH NGHIỆM TRONG TRUYỆN TRANH “NÀNG BẠCH TUYẾT VÀ BẢY CHÚ LÙN” TỪ CÁCH NHÌN

In this study, I would like to express my gratitude toward my supervisor Dr

Võ Duy Đức with his enthusiasm, his encouragement as well as his patiencefor helping me finish the master thesis He spent a lot of time reading andcorrecting the mistakes in my thesis carefully although he was very busy Healways encouraged me to seek many articles, read many books, and studiedmany academic researches to conduct well the study

I would also like to express my sincere thanks to the board of directors of NgoMay High School, especially my headmaster - Châu Văn Phú for helping andfacilitating the distribution of teaching timetable during the time I took thecourse Moreover, I would like also to thank my close colleagues for teaching

my classes

I am extending my special thanks to all lecturers in the English master course.They have taught me a lot of useful knowledge during the two years I tookpart in this master course

Lastly, I would like to express my gratitude to my family for encouraging mementally and physically so that I could successfully complete this course

STATEMENT OF AUTHORSHIP

I merely confirm that the thesis titled ―A study on experiential meaning in

“Snow White and the Seven Dwarfs” from multimodal perspective” is

written only by me and describes entirely the result of my own work I choosemany references which have been quoted and all sources of information havebeen clearly cited The thesis has not been submitted for the award of anyother degree or diploma in any university or institution

Qui Nhon, 2020

VO THI DANG NGOC

An experimental investigation was conducted to explore theexperiential meaning of the verbal structures and the visual images through

multimodal perspective in the comic story Snow White and the Seven Dwarfs.

Two main frameworks of theory which are applied for the analysis of thisstudy are Systemic Functional Grammar and Visual Grammar SystemicFunctional Grammar formulated by Halliday (2004) is used for analyzing theexperiential meaning of the verbal texts in the collected data resources, whileVisual Grammar originated by Kress& Leeuwen (2006) is supported toidentify the experiential meaning of the visual language in the comic story

Snow White and the Seven Dwarfs The findings indicate that while texts in Snow White and the Seven Dwarfs comic show the actions and the events with

a large numbers of Material processes, the images mainly focus on thephysical actions of the represented participants with the Actional processes.Moreover, Relational processes in the verbal texts used to characterize oridentify the participants account for a proportion of equivalent rate withReactional processes in the visual image applied for identifying therepresented participants The results of the numerical simulation indicate that

circumstances both the verbal texts and the visual images in the comic Snow

White and the Seven Dwarfs rotate around the time and place in which the

activities or events take place with a large number of Location in texts and

Setting in images Implications of the results and future research directions

were drawn out

ABBREVIATIONS AND CONVENTIONS

[ I-207]

- Boldface is used to mark the first use of technical terms.

- Italics is used for examples, emphasis, and technical terms.

- Underlined words are used to mark the use of what is being illustrated

For example:

a One day my prince will come and rescuse me!

b Obey me, or I will kill you and your family!

=> The words ―come, rescue, and kill‖ above are underlined to depict the use of the verbal text in Material processes

CHAPTER 1 INTRODUCTION

1.1 RATIONALE

In the recent decades, comics have been one of the major interestingresearch subjects which we can use to tell our children about the wonderfullyimaginative story through the images and texts; they play an important role inour lives and they show a source of messages in creating the imaginary worldfor the children and sharing the dominant values of a society Throughoutreading comics and understanding the meanings of different comics, weactually experience the different cultures and characters around the world.Especially, each comic is usually structured with a happy ending for a goodfigure and the bad character being punished Moreover, a comic is a greatsource of additional material for the children or the people who do not knowhow to read and write via images It may help to enhance their creativeimagination which is regarded as an essential preparation for our futuresuccess Thus, comics constitute a significant instrument of entertainment.Colorful images, meaningful contents and the variety of characters makereading generally more congenial and create a profound impression on younggenerations as well; therefore, they can feel relaxed

Each image of a comic contains many kinds of various meanings when

we see and observe it from multimodal perspectives Jewitt (2007) argued that

a multimodal perspective on texts attends to linguistic resources (speech andwriting) as a part of a multimodal ensemble in which resources are organized,designed and orchestrated to realize curriculum knowledge and pedagogicrelations All of these perspectives are important and grow our understanding

of the comics The goal of our perspectives is to demonstrate the varied wayone can read, interpret and view a comic We want to collect the valuable

perspective on many different facets of comic If we look at the picture anddiscover the images containing many different meanings, our impressiveability about the pictures is completely different Images alone may not showclearly the intended meaning , so texts are a useful means for us to understandthe full meaning of the writer‘s message Kress and Van Leeuwen (2006)present that the common view of the function of images as ‗illustration‘ and

‗explanation‘ would be that the accompanying verbal text explains what is

not made clear visually.

Thus, each story has a different storyline and different images that the

author wants to express a national culture or that nation‘s customs, directingpeople to life‘s values through our own perception of the world around us,especially children One of wonderfully comic stories by simple words and

vivid image is Snow White and the Seven Dwarfs by which children may

approach the sense of experience from a multi-modal perspective with ease.Maybe, they perceive the plot in the experiential sense from a multimodalpoint of view via identifying the verbal components and visual languages in

Snow White and the Seven Dwarfs comic It could be a fact that the verbal text

and visual texts can be interpreted in various ways by different people In thecourse of my research into comics, I have found that there has not been much

experiential meaning study in the comic book Snow White and the Seven

Dwarfs Therefore, I decided to do the research on the topic ―A study on

experiential meaning in “Snow White and the Seven Dwarfs” from multimodal perspective”.

1.2 AIM AND OBJECTIVES OF THE STUDY

1.2.1 Aim of the study

The aim of the study was to investigate the experiential meaning in the

verbal texts and the visual images in the comic story ―Snow White and the

Seven Dwarfs‖ from multimodal perspective based on ―Systemic Functional

Grammar‖ originated by M.A.K Halliday (2004) and ―The Grammar of

visual design” formulated by Gunther Kress and Theo van Leeuwen (2006).

In order to fulfill the objectives above, the thesis focuses on two

research questions as follows:

1 How are the experiential meaning communicated verbally and

visually in ―Snow White and the Seven Dwarfs‖ comic?

2 What is the occurrence frequency of process types used in these verbal and visual communications?

1.4 SCOPE OF THE STUDY

This study particularly focuses on the only data ―Snow White and the

Seven Dwarfs” comic served for the work Moreover, the study is just on the

experiential meaning, leaving the other two meanings, the interpersonal andtextual, untouched

1.5 SIGNIFICANCE OF THE STUDY

In this paper, the great triumph in applying the two major theoriesenables us to understand experiential meaning in both texts and images via

multimodal perspective from the comic story Snow White and the Seven

Dwarfs The study provides the parents of the young with useful knowledge

to help their children build the life basic skills from the imaginary world tothe real one around reading and understanding visually clear meanings inimages In addition, the findings of the study about the experiential meaning

in text would be a valuable contribution for seeking out meaning of a comicstory which is not made clear in images Hopefully, the study is found to beuseful for those who need to do research on experiential meaning as well asthose who are interested in studying comics from multimodal perspectives

1.6 ORGANIZATION OF THE THESIS

The thesis consists of five chapters

- Chapter one, Introduction, presents rationale of the study, aim and

objectives of the study, research questions, scope of the study, significance of thestudy, and organization of the thesis

- Chapter two, Literature review, comprises the review of literature of

the established theories, the empirical studies and previous studies related to theresearch area

- Chapter three, Methodology and Procedures, outlines the research

methods and procedures of the study It also shows the methods utilized to collectthe data as well as to conduct the research

- Chapter four, Findings and Discussion, presents the author‘s findings

of the experiential meaning in the verbal texts as well as the visual images in

Snow White and the Seven Dwarfs comic.

- Chapter five, Conclusions, summarizes the main findings which

satisfy the research questions, presents implications and limitations of the study,and shows suggestions for further research of the study

CHAPTER 2 LITERATURE REVIEW

This chapter deals with Halliday‘s Systemic Functional Grammar andKress and Van Leeuwen‘s Visual Grammar as they are employed astheoretical framework for the study Furthermore, it shows an overview ofprevious research related to the present study

2.1 THEORETICAL BACKGROUND

2.1.1 Systemic Functional Grammar

Systemic Functional Grammar (henceforth SFG) is a form ofgrammatical description introduced by Halliday It is a linguistic, semiotic,descriptive theory of language According to Halliday (2004), language is amean of communication, which fulfills three distinct different modes ofmeaning or metafunctions They are ideational, interpersonal, and textualmetafunction The textual metafunction deals with organizing ideational andinterpersonal meaning as text in context The interpersonal one is used to talkabout language interaction between the speaker and listener The ideationalone is concerned with encoding our experience of the real world and of ourinner world As will be shown in the chapter, SFG theory ideational resourcesare related to particular elements of realization, construing experience ascomponents or constituents Experiential and logical function are two mainfunctions in the ideational meaning The experiential meaning is that languageexpresses our experiences in outside world such as things, events qualities andinside world as thoughts beliefs, feelings What logical function refers to isthat language expresses the logical relationship between two or more than twomeaning units Thus, when we utilize the theory association with theexperiential meaning, experiential function is chiefly embodied by the system

of transitivity

represents that transitivity system includes six processes: Material process,

Mental process, Relational process, Behavioral process, Verbal process, Existential process Figure 2.1 represents process types as a semiotic space,

with different regions representing different types

Figure 2.1 The grammar of experience: types of process in English

(Source: Halliday 2004, p.172)

These process types are clearly focused in the following parts:

Material process is a process of doing or happening Material Processes

express the notion that some entity ‗does‘ something- which may be done

‗to‘ some other entity The process is realized by the verbs such as form,

emerge, make, create, product, construct, build, design, write, compose, draw, paint, bake, etc.

Actor and Goal are two participant roles in material processes Agent is

also another participant who took part in causative material processes TheAgent causes an Actor to carry out the action For example:

1) Birds are flying in the sky

Actor process: material Circumstance

(Halliday 2004, p.175)

2) The lion caught the tourist.

(Halliday, 2004, p.180)

In addition to these two roles, there are other participant roles that may

be involved in the material processes such as Scope, Recipient, Client and

Attribute.

Scope is the participant that may construe an entity, which exists

independently of the processes but which indicates the domain over which theprocess takes places For example:

3) She crossed the room

Actor Process: material Scope (entity)

(Halliday 2004, p.189)

The two functions of Recipient and Client resemble one another in that

both construe a benefactive role; they represent a participant that benefiting

from performance of the process The Recipient is the participant that goods are given to; the Client is one that services are done for Here are some

illustrations

4) I gave my love a ring that has no end

Actor process Recipient Goal

(Halliday 2004, p.191)

5) Fred bought a present for his wife

Actor process goal Client

(Halliday 2004, p.191)

Moreover, Attribute is the additional participants of the material

process Halliday presents that Attribute may be used to construe the resultantqualitative state of the Actor or Goal For instance:

6) They washed the plates clean.

Actor process: material Goal Attribute

(Halliday 2004, p.189)

Mental processes are concerned with our experience of the world of

our own consciousness They are a process of thinking involving perception(see, taste), and reaction (like, fear) and cognition (know, believe, andunderstand), emotion (love, like) and so on Mental processes go with two

main participants: Senser and Phenomenon Senser refers to the person who

perceives and phenomenon is something that is perceived by the senser.Phenomenon included concrete person or objects, abstract things, happened

events and so on Table 2.1 shows the verbs serving as in mental processes.

Table 2.1 Verbs serving as Process in mental processes

(Source: Halliday 2004, p.210)

perceive, sense; see, notice, glimpse; (assail)

perceptive hear, overhear; feel; taste; smell

think, believe, suppose, expect, strike, occur to, convince; consider, know; understand, realize, remind, escape; puzzle, intrigue,

cognitive appreciate; imagine, dream, pretend; surprise

guess, reckon, conjecture, hypothesize; wonder, doubt;

remember, recall, forget; fear (think fearfully)

want, wish, would like, desire; hope (tempt) (for), long for, yearn for; intend,

desiderative plan; decide, resolve, determine;

agree, comply, refuse like, fancy, love, adore, dislike, hate, allure, attract, please, displease, detest, despise, loathe, abhor; rejoice, disgust, offend, repel, revolt;

emotive exult, grieve, mourn, bemoan, gladden, delight, gratify, sadden,

bewail, regret, deplore; fear, dread; depress, pain; alarm, startle, enjoy, relish, marvel frighten, scare, horrify, shock,

comfort, reassure, encourage; amuse, entertain, divert, interest, fascinate, bore, weary, worry

The following sentences present the mental processes

I can feel something on my foot

Senser Process: mental-perception Phenomenon

(Halliday 2004, p.199)

7) Mary liked the gift

Senser Process: mental-emotion Phenomenon

(Halliday 2004, p.200)

8) He knows the car

Senser Process: mental-cognition Phenomenon

(Halliday 2004, p.208)

9) He wants the car

Senser Process: mental desideration Phenomenon

(Halliday 2004, p.208)

Relational clauses are the processes of being and having They are used

for serving to characterize and to identify There are three main types of

relation: intensive, possessive, and circumstantial; each of those comes in two

distint modes of being – ‗attribute‘ and ‗identifying‘.

In the ‗attribute‘ mode, there are two elements recognized the

Attribute and Carrier The ‗attribute‘ is labelled by the class, the ‗carrier‘ is

ascribed by the entity Moreover, in the ‗identifying‘ mode, Value and Token

can be identified another in the intensive relational processes Value isidentified by Token and Token serves to the identity of Value The process is

realized by the verb ‗be‟ or synonym of it For instance:

Relational process of intensive attribution

Carrier Process: Relational: intensive attribution Attribute

(Halliday 2004, p.216)

11) Mice are timid creatures Carrier Relational: intensive attribution Attribute

(Halliday 2004, p.219)

Relational process of intensive identifying

12) Tom is the treasure Value Process: relational Token

(Halliday 2004, p.230)

13) Tom is the tall one Token Process: relational Value

(Halliday 2004, p.230)

On the other hand, relational process is possessive structures The

process can be realized by the present tense of the most typical popular verb

―have‖ and ―belong to‖, as exemplified below:

Relational process of possessive structures

14) Peter has a piano

Carrier Process: Relational: possessive attribution Attribute

(Halliday 2004, p.216)

15) Peter owns the piano

Token Relational: possessive identifying Value

(Halliday 2004, p.247)

Moreover, in Attributive circumstantial processes, the circumstantial

element was being ascribed to some Carriers For examples

16) The fair is on Tuesday

Process: intensive attribution Attribute: circumstantial Carrier

(Halliday 2004, p.216)

17) Tomorrow is the tenth

Token Process: intensive identifying Value

(Halliday 2004, p.244)

Behavioral process is a process of behaving These are processes of

physiological and psychological behavior, like breathing, smiling, dreaming

and staring The basic components of the process are Behaver and Behavior.

Behaver is the participant who is typically a conscious being Behavior is the

entity which exists independently of the process itself The examples show the

Behavioral processes.

18) People are laughing

Behaver Process: intensive Behavioral

(Halliday 2004, p.250)

19) He smile a broad smile

Behaver Process: behavioral Behavior

(Eggins:1994, p.250)

Verbal processes are processes of saying ‗Saying‘ has to be interpreted in

a rather broad sense It covers any kind of symbolic exchange of meaning

Sayer, Receiver, and Verbiage are three main participants in verbal processes.

A Sayer is being confined to speakers and writers who are responsible for the verbal process, such as asking or telling The Receiver is the one to whom the

saying is directed and is realized a conscious being (a potential speaker), a

collective or an institution Verbiage may construe the topic of what is said The Target occurs only in a sub-type of ‗verbal‘ process; this function

illustrates the entity that is targeted by the process of saying The set of verbs

serving as Verbal Process includes the common verbs „say, tell, and talk‟ and examples of verbs representing features of dialogic exchange, reply and

encounter.

The examples of this type can be clearly seen in (21) and (22)

20) You didn‟t tell me that last time Sayer Process: verbal Receiver Circumstance

Existential processes represent that something existed or happened In every

existential process, it must have an ―Existent‖, the Existent is recognized by the use of the word there as the subject The two examples below illustrate the

existential processes

22) There was a storm

Process: existential Existent

(Halliday 2004, p.259)

23) On the wall there hangs a picture.

process: existential Existent

(Halliday 2004, p.259)

Table 2.2 shows a summary of the types of process identified together with their general category meaning and the participants that are associated with each

Table 2.2 Summary of the process types

(Source: Halliday 2004, p.260)

PROCESS Category Participants, Participants, TYPE meaning directly involved obliquely involved Marterial: ‗doing‘ Actor, Goal Recipient, Client:

Action ‗doing‘ Scope; Initiator;

Event ‗happening‘ Attribute

behavioural ‗behaving‘ Behaver Behavior

Mental: ‗sensing‘ Senser, Phenomenon

Attribution ‗attributing‘ Carrier, Attribute Attributor,Beneficiary

Identification ‗identifying‘ Identified, identifier, Assigner

of processes and some special interpretations between the categories of

participant and circumstance Extent, location, manner, cause, contingency,

accompaniment, role, matter and angle are the main elements of

circumstances They are typically expressed in as adverbial groups orprepositional phrases

 Extent and Location

The circumstantails of Extent and Location construe the unfolding of the process in space and time Extent construes the extent of the unfolding of

the process in space-time: the distance in space over which the process

unfolds or the duration in time during which the process unfolds Meanwhile,

Location represents the location of the unfolding of the process in space-time:

the place where it unfolds or the time when it unfolds For example:

24) Jack was building a house throughout the year near the river…

Extent: time Location: place

(Halliday 2004, p.261)

25) The Dodo pointed to Alice with one finger

Actor Process Location: spatial manner

(Halliday 2004, p.281)

26) The whole party at once crowded round her

Actor Location: temporal Process:material Location: spatial

(Halliday 2004, p.281)

Manner construes the way in which the process is actualized Manner

comprises four subcategories: Means, Quality, Comparison, Degree Thesecover a considerable range; means is close to the participant role of Agent andcomparison is like a participant in a clause with the same kind of process,whereas Quality and Degree are like features of the process itself Means and

Comparison are realized by Prepositional phrases (by or with), whereas

Quality and Comparison tend to be realized by adverbial groups The

examples show the following circumstantial manners.

27) She beat the pig with the stick

Manner: means

(Halliday 2004, p.267)

28) He learned to walk in a certain way

Reason represents the reason for which a process takes place — what causes

it It is typically expressed by a prepositional phrase with through, from, for or

a complex preposition such as because of, as a result of, thanks to, due to; the negative for want of, as in (32).

31) Is it worse because of your asthma

Cause: reason

(Halliday 2004, p.270)

Cicumstantials of Purpose represent the purpose for which the process

is actualized (what has been called ‗final cause‘) They are typically

expressed by a prepositional phrase with for or with a complex preposition such as in the hope of, for the purpose of, for the sake of; for example:

32) President Bush is rallying the for a war against terrorism‟s attack on our

nation way of life.

Cause: purpose

(Halliday 2004, p.270)

+ Behalf

Expressions of Behalf represent the entity, typically a person, on whose

behalf or for whose sake the action is undertaken They are expressed by a

prepositional phrase with for or with a complex preposition such as for the

sake of, in favor of (negative: against), on behalf of For example:

33) Therefore, I urge you to against a CCC endorsement of the nuclear freeze vote initiative

Cause: purpose

(Halliday 2004, p.271)

Circumstances of Contingency specify an element on which the

actualization of the process depends There are three sub-types: Condition,

Concession, Default.

+ Condition

Circumstantial of Condition construes circumstances that have to obtain

in order for the process to be actualized; they have the sense of ‗if‘ They are

expressed by prepositional phrases with complex prepositions in case of, in

the event of, on condition of.

34) Get back to the bedroom and change clothes in case of bloodstains

they are expressed by prepositional phrases with the prepositions despite, not

with standings or the complex prepositions in spite of or regardless of; for

example:

35) In spite of its beacon , many ships have been wrecked on this rocky coast

during storms or in dense fog.

Contingency: Concession

(Halliday 2004, p.272)

+ Default

Default circumstantial has the sense of negative condition ‗if not,

unless‘; they are expressed by prepositional phrases with the complex

prepositions in the absence of, in default of; for example:

36) In the absence of any prior the amount is assessed, as a rule, by the agreement between the parties as to Admiralty Court.

the rate of salvage payable,

Contingency: Default

(Halliday 2004, p.272)

Accompaniment is a form of joint participation in the process and

represented the meanings ‗and‘, ‗or‘, ‗not‘ as circumstantial Two

subcategories of circumstantial Accompaniment are comitative and additive.

+ Comitative

The comitative represents the process as a single instance of a process,

although one in which two entities are involved

37) I was traveling up the with my father in a boat, west coast of Florida

Accompaniment: Comitative

(Halliday 2004, p.273)

+ Additive

The additive represents the process as two instances; here both entities

clearly share the same participant function, but one of them is representedcircumstantially for the purpose of contrast For example:

38) Instead of dingy velveteen he had brown fur, soft and shiny Accompaniment: Additive

(Halliday 2004, p.273)

Role construes the meanings ‗be‘ and ‗become‘ circumstantially Role

includes the subcategories of Guise (‗be‘) and Product (‗become‘)

39) He set out as a pauper

Actor Process: material Role: guise

(Halliday 2004, p.275)

40) Bend that rod into a straight line

Process: Goal Role: product

material

(Halliday 2004, p.275)

Matter is the circumstantial equivalent of the Verbiage, ‗that which

is described, referred to, narrated, as exemplified below:

41) Tell me about the Paris Review

interpersonal, textual and ideational function Ideational function is

concerned about our inner world, which is the major communication tool toencode and identify the action, relation, participants and circumstance based

on investigating Transitivity system The framework unfolds the experientialreality, the inner experience or the processes of consciousness and the outerexperience or the external world The system of transitivity is relevant to theprocess types with the experiential meaning in verbal modes The mainprocesses are Material, Relational, Verbal, Mental, Behavoiral and Existentialprocesses and Circumstantail elements are the additional participants, whichare focused on the experiential meaning Like verbal texts, the transitivitysystem about the experiential meaning is also performed in visual modeswhich are discussed in more details in the grammar of visual design in thenext section

2.1.2 The Grammar of Visual Design

Kress & Leeuwen (2006) present the ‗grammar of visual design‟ that

visual structures point to particular interpretations of experience and forms of

social interaction The way meanings are mapped across different semiotics

modes, the way can, for instance, be ‗said‘ either visually or verbally othersonly visually, again others only verbally, was also culturally and historicallyspecific

Kress and Leeuwen believe that visual design, like all semiotic modes,fulfills three different kinds of major functions which are named the

ideational, the interpersonal and textual as in Halliday‘s terms The

ideational function represents the world around and inside us as well as

express their experience about the real word via the entities, actions or eventsexisting in images or thought The interpersonal function is a function ofenacting social interactions as social relations, and textual function is function

of presenting the coherent words of the text

The experiential structures represent the aspects of the world visuallyvia the visual transitivity system Kress & Leeuwen (2006) recognize that the

structures can be either Conceptual or Narrative Following is the detailed description of Narrative and Conceptual processes Examples will be

provided for illustration and clarification

2.1.2.1 Narrative processes

When participants are connected by vectors, they are represented asdoing something to or for each other (Kress & Leeuwen 2006, p.59).Processes are realized by ‗vectors‘ and vectorial patterns are seen as

narrative Narrative processes represent unfolding actions and events,

processes of change, transitory spatial arrangement

The narrative structures in communication in the visual elementsconsist of process and circumstances Narrative processes have the major

components such as Actional and Reactional, Mental, Verbal, and

Conversion Circumstances are Setting, Means, and Accompaniment Figure

2.2 below is the summary of narrative structures As can be seen figure 2.2 inthe next page

 Actional processes

Actional processes are the processes of physical action related to the

represented participants realized by the vectors The vectors can be created bybodies, or limbs, or tools in action Actor and Goal are two main participants

in actional processes While Actor is the participant from which the vectoremanates, Goal is the participant from whom or which the vector is directed

Actional processes can be non-transactional or transactional.

Figure 2.2 Narrative Processes in visual communication

(Source: Kress & Leeuwen, 2006, p.74).

Kress & Leeuwen (2006) point out that when images or diagrams have

only one participant, this participant is usually an Actor (non– transactional)

or unidirectional transactional action When a narrative visual proposition hastwo participants, one is the Actor, and the other the Goal The Actor in such atransactional process is a bidirectional transactional action A vector is formed

by depicted elements and the participants are called Interactors to indicate

their double role

In Figure2.3, the Actor is a represented participant from which thevector emanates This structure is called non-transactional and had no Goal.This is a vector formed by an arrow

Figure 2.3 Unidirectional transactional process

(Source: Kress & Leeuwen, 2006, p 63)

Figure 2.4 illustrates a bidirectional transactional structure The twomen play the role as Actors and Goals They allow the participants toexchange with each other: each man plays the role of Actor and then becomes

a participant with the same Actor from the other participant The two men are

Interactors.

Figure 2.4 Bidirectional transactional process

(Source: Kress & Leeuwen, 2006, p 66)

whom or which the Reacter is looking Like actions, reaction can be

transactional and transactional In the case, the reaction in a

non-transactional process has no Phenomenon, but reactional non-transactionalprocess was Phenomenon Reacter obtains in transactional and non-transactional processes.

In Figure 2.5, for instance, the old man and his gesture form thePhenomenon, while the young boy is the Reacter who is looking at the gesture

of the old man This is called Reactional process as can be seen in Figure 2.5

Figure 2.5 Reactional process

(Source: Kress & Leeuwen, 2006, p 64)

Verbal processes are the processes connecting a participant with the

content of the speech Verbal process is one of the narrative processes dealingwith the oblique vector They are like the oblique protrusions between thoughtballoons and dialogue balloons There have two participants in verbal

processes, one is a Sayer and the other is an Utterance While Sayer is the

participant As presented above, Figure 2.6 presents that Sayer is a girl talking

on the telephone; Utterance is the ‗verbal‘ participant enclosed withinformation in the ‗dialogue balloon‘

Figure 2.6 Verbal process

(Source: Painter et al., 2012, p 69)

 Mental process

The processes created by the oblique vector of thought balloon orbubble which is connected by thinkers or speakers to their thought are

called Mental processes Senser and Phenomenon are two major

participants in the Mental structure Senser is the participant from whom the ‗thought bubble‘ vector emanates, whereas the Phenomenon is the

content of the speech of mental process The Mental process is depicted in

Figure 2.7

Figure 2.7 Mental process

(Source: Painter et al., 2012, p 69)

 Conversion processes

Conversion process is a process in connection with a participant The

participant is called Relay First is the Goal of one action and then it is the

Actor of another This involves a change of state in the participant This kind

of process is especially common in representations of natural events.Figure2.8 shows the conversion process

Figure 2.8 Conversion process

(Source: Kress and Leeuwen 2006, p 70)

2.1.2.2 Circumstances

Kress & Leeuwen (2006) represent that narrative images may containsecondary participants; the participants are associated with the mainparticipants, not by means of vectors, but in other ways These participants are

Circumstances Circumstances are made up of Setting, Means, or

Accompaniment.

Setting or Locative Circumstance is a term of a fact or condition of placeand time, and is relevant to an event or action The Setting of a process is animportant participant in a narrative process and used to refer to the socialmilieu in which the events of the participant take place For example,Figure2.9 depictes Circumstance of Setting They are the major participants innarrative processes which are the trees, grass, the moon and rocks describingthe beautiful landscape and the action of the two men holding gun

Figure 2.9 Circumstance of Setting

(Source: Kress & Leeuwen, 2006, p 45)

The tools used in action processes are often represented asCircumstances of Means If this is the case, there is no clear vector betweenthe tools and its user The tools themselves constitue the vector which realizethe action process Figure2.10 shows an example of Means The gesture of

the man and his hands below is a tool in actional process as a Circumstance of

Means (The old man addresses an unseen participant with his hands).

Figure 2.11 Accompaniment

(Source: Kress & Leeuwen, 2006, p 64)

2.1.2.3 Conceptual Processes

Kress & Leeuwen (2006) show that Conceptual processes represent theworld in terms of class, or structure or meaning Three major categories are

Classificational, Analytical, and Symbolic processes The simple framework

summarizes the types of Conceptual processes shown in Figure 2.12

Conceptual processes

Kress & Leeuwen (2006) noted that Classificational processes

representing the representational structures relate participants to each other interms of a ‗kind of‘ relation, a taxanomy Classificational processes arecomposed of two sub-types: covert and overt taxonomies

Covert Taxonomy, a taxonomy in which the Superordinate is inferred

from such similarities as the viewer may perceive to exist between the

Subordinates, or only indicate in the accompanying text, as in Figure 2.13.

The Subordinates can be visually realized by a symmetricalcomposition in the equal distance from each other They represent the samesize and the same orientation towards the horizontal and vertical axes Torealize the stable, timeless nature of the classification, the participants areoften shown in a more or less objective, decontextualized way

Covert taxonomies are often used in advertisements That represented

the variety of products marketed under a brand name, or arrangements ofdifferent people They all used the same product taxonomy represented by

formal diagrams with simple lines Meanwhile, Overt taxonomies are usually

‗chained‘, so that the ‗intermediate‘ participants indicate the term called

(Source: Kress & Leeuwen, 2006, p.81)

Analytical processes related participants in terms of a part–wholestructure There are two kinds of participants: Carrier and PossessiveAttributes One Carrier is the participant representing the whole of the

process, and Possessive Attributes are the parts of the process In this analytic

structure, they have no vectors It serves as a Carrier and allows viewers toscrutinize this Carrier‘s Possessive Attributes

Figure 2.15 shows Analytic process that the picture of Antarcticexplorer, which analyse the explorer in terms of his outfit, is one example ofthe analytical structure, the Antarctic explorer functions as a ‗Carrier‘, and

‗Possessive Attributes‘ are made up of the balaclava, the windproof top, andthe fur mittens, etc

Figure 2.15 Analytical process

(Source: Kress & Leeuwen, 2006, p.51)

represents the meaning or identity itself

If there is only participant, Carrier, the process is Symbolic Suggestive.

In Symbolic Suggestive processes, kind of images can be de-emphasized infavour of what can be called ‗mood‘ or ‗atmostphere‘ Figure 2.16 and 2.17below are Symbolic Attribute processes

Figure 2.16 shows that a scientist is studying on the fungi One fungus

is held in his hand by the scientist, and the fungi are in the foreground Thescientist is only Carrier and the fungi are Symbolic Attributes The Carrier

establishes his identity as an expert on fungi via the image of the fungi as theAttributes As shown in Figure 2.17 the scientist is researching information inhis book The Carrier is the scientist and the book and other objects in hisroom are the Attributes The viewers can recognize his identity and meaning

as a chemistry scientist

Figure 2.16 Symbolic Attribute process

Figure 2.17 Symbolic Attribute process

(Source: Kress & Leeuwen, 2006, p.106) (Source: Kress & Leeuwen, 2006, p.107)

2.1.3 Summary

To sum up, the research draws upon a theoretical framework from the

emergent field of ―Systemic Functional Grammar‖ by Halliday (2004) and

―Visual Grammar Design‖ by Kress & Leeuwen (2006) Transitivity system

is applied for analyzing the data composing of the main processes such asMaterial, Relational, Verbal, Mental, Behavoiral, and Existentail processesand circumstantail elements.They are usually used to encode the experientialmeaning in the verbal texts

The elements which are utilized to study grammartical forms asresources for encoding interpretations of experience and form of socialinteraction are Narrative processes and Conceptual processes in VisualGrammar developed by Kress & Leeuwen‘s theoretical framework InNarrative processes, the main processes are made up of Action, Reaction,Mental, Verbal, and Conversion In Conceptual processes, Classificational,Analytical, and Symbolic processes are involved with the experiential

meaning in the visual mode.

Moreover, the circumstantial elements are additional participants instudying the experiential meaning of the visual semiotics Thus, the features

of the framework of the theory are represented above The section whichfollows deals with previous studies related to the experiential meaning indifferent fields

2.2 PREVIOUS STUDIES

In recent years, several studies have focused on researching theexperiential meaning in the verbal and visual texts in different fields Somecan be listed here

Joy, Annamma, Sherry, John Venkatesh, Alladi Deschenes and Jonathan(2009) used many different multi-faceted ways that individuals want to buildtheir own meanings and their own experiences with the Internet via verbal andvisual analysis Lwin (2010) investigated the relationship between the spatialassignments that students may apply them for solving linear syllogisms and thelexical marking of the comparative adjectives in which students may use in thepremises Studies have found that experimental conditions certainly favor verbalprocesses rather than visual ones in problem solution

Similarly, Jaffe (2018) showed that graphic novels are used to teachvisual and verbal literacy He pointed out that while children's language ofcommunication outside the classroom is today acronyms and visual symbols,teachers are required to teach them how to listen, think, read, and writecomplex texts Graphic novels are perfect and unique vehicle that has helped

us to link students' communication skills and interests with compellingeducational goals The findings show that graphic novels are such powerfuleducational tools to help student's learners make interesting texts, to make thelesson more meaningful, and provide advice on conducting new literary

mediums in classroom Thus, graphic novels open up an exciting new world

to help children understand text contents via visual and verbal one

Moreover, in 2014, Sütterlin, Wulf Schiefenhövel, ChristianLehmannz, Johanna Forster 3, and Gerhard Apfelauer published an articleabout Art as Behavior related to visual and verbal art, music and architecture.The findings show that different aspects represented by art are focused onphylogenetic adaptations more than the other types that shaped our ancestors'artistic abilities Therefore, research focusing on visual and verbal arts, musicand architecture are constructed and designed to contribute to primitiveinterdisciplinary work in the field of evolutionary art research

Likewise, Unsworth (2014) searched into ‗Interfacing visual and

Verbal Narrative Art in Paper and Digital Media‘ The study illustrates that

visual and verbal aesthetic art about image stories, illustrations and movieadaptations is highly regarded as an important aspect of understandingliterature in art because of traditional stories and new cultures created andrecreated in many different art forms The findings suggest that subtlevariations in visual art of different versions of the same story may result insignificant changes in interpretability Thus, aesthetic understandingdevelopment and interpretability about cultural cohesion represented fromstories via implications for teaching and researching of new forms ofmultimodal perspectives are addressed

In addition, Demetra Pitta-Pantazi and Roza Leikin (2013) investigatedthe relationship between the creative process in mathematical tasks andspatial, object and verbal cognitive styles The findings show visual cognitivestyles (spatial and object imagery) were statistically significant predictors ofparticipants‘ creative abilities in mathematics more than verbal cognitiveones Therefore, this study tends to be towards different cognitive styles that

used different strategies in creative mathematical tasks.

Unsworth, Leonard, Cheirigh, and Chris (2009) expressed the ways inwhich images construct meanings The study show the intersemiotic semanticrelationships between images and language are explored by very littleresearcher through visual and verbal modes in interaction to construct theintegrated meanings of multimodal texts The findings suggest that visual andverbal interaction for building meanings can be one of the most crucial ways

to re-perceive pedagogy literacy from a multimodal perspective

Painter (2013) carried out analyzing images in children‘s picture books.This analysis provides a rich field for theoretical development and visualmeaning analysis and its relationship to the accompanying verbal text Thefindings show that previously socio-semiotic image analysis focussedattention on meanings at discourse level and on the semantic relationshipscreated by image sequences and extends current understandings of how imagebooks work through its explicit and systematic account of the visual meaningsand their integration with verbal aspects of the texts Thus, it is important toapply for discourse analysis, systemic-functional theory

Besides, Damanik (2017) examined the experiential meaning in poetrywritten by William Wordsworth Halliday Systemic Functional Grammar wasapplied to identify and analyze the occurrences of processes in five poetriestaken as the subject on this research The analysis shows that five of the sixprocesses in transitivity system such as material, mental, relational, behavioraland existential coded in the poetry of William Wordsworth The materialprocesses are the most dominant types in five poetries, it means that poetry tellsabout the poet‘s action in the expression of his idea, feeling and life This findingalso shows that William Wordsworth as the poet uses the material process tobuild the imagery such as personification, simile and metaphor and to be useful