Analysing visual imagery connecting semiotic and cognitive perspectives may l y wong

This article provides an exploratory study of a new analytical approach to examining visual imagery in relation to the underlying cognitive processes involved The analytical approach combines social semio.

Analysing Visual Imagery: Connecting Semiotic and Cognitive Perspectives

May L-Y Wong

School of English, University of Hong Kong

wlymay@gmail.com; maylywong@hku.hk

Abstract

This article provides an exploratory study of a new analytical approach to examining visual imagery in relation to the underlying cognitive processes involved The analytical approach combines social semiotic theory of representation with cognitive-linguistic studies on blending or conceptual integration The author’s thesis is that visual-analytic tools suggested by the social semiotic approach perfectly complements the inward cognition of an image-viewer, a synergy which has rarely been envisaged by scholars from both disciplines From this perspective, visual analysis is seen as both semioti-cally and cognitively relevant The interdisciplinary approach developed in the article hopes to present new perspectives on the ways images are analysed and interpreted

to fully comprehend visual structures Perhaps, the first question that comes

to our mind when we go about talking and thinking about what is actually communicated by images and by their visual design is: Do we have to engage

in a completely different set of cognitive processes for perceiving visual ages from what we normally do with verbal messages? In this paper I propose

im-a unified view of the relim-ations between conceptuim-al im-and visuim-al structures thim-at would have otherwise been considered quite disparate and dealt with rather separately in different domains as in cognitive linguistics and (social) semiot-ics I explore how image viewers harness both semiotic resources and concep-tual blending to construct meanings Although visual representation provides

a wealth of information about the way people make sense of a sign in ics, blending theory in cognitive linguistics goes further by accounting for the processes by which people create temporary and dynamic mental spaces and construct meanings within them

semiot-Although the terms ‘semiotics’ and ‘blending’ refer to different aspects of conceptualisation, they are in fact complementary to each other Coining the term ‘semiotic blends’, Bateman calls for a more integrative account of rela-tionships across visual and verbal modalities since “it is quite possible for there

to be … a growth of meaning drawing on both” (2014: 184; original emphasis),

hinting at the possibility of transferring meanings and integrating different kinds of information that has been seen in work in the cognitive linguistic tradition that has sought ways of characterising the conceptual integration of information of various kinds in the notion of a blend (Fauconnier 2001).The complementarity between cognitive linguistics and (social) semiotics

is not entirely new, though Brandt and Brandt hypothesise a general work for analysing metaphoric expressions in which all metaphors should be considered as signs – as understood in the Peircean tradition – and metaphori-cal meanings should “belong to the world of human experience” (2005: 244) and thus are inherently semiotic, i.e anchored in their contexts of commu-nication and engaging the imagination of both the speaker and the address-

frame-ee While Brandt and Brandt (op cit.) use only linguistic data and focus on

just the butcher-surgeon conceptual metaphor, Feng and O’Halloran’s (2013a

[2015], 2013b) examine visual realisations of metaphor to suggest that visual

images in films and tv commercials are constructed through combinations

of semiotic choices to cue different conceptual metaphors with which tive linguists have been preoccupied for decades ever since George Lakoff and

cogni-Mark Johnson’s seminal treatise Metaphors we live by (1980) By the same token,

Feng (2017) provides a social-semiotic account of conceptual metonymies in static and moving images, mapping out the types of metonymy in visual repre-sentation Alonso et al (2013) explore how the social semiotic visual grammar developed by Kress and van Leeuwen (2006) can contribute to the understand-ing of the global meaning of a narrative by combining social semiotic theo-

ry with some notions of mental spaces and Conceptual Integration Theory

(also known as Blending Theory; see Fauconnier 1994; Fauconnier and Turner

op cit.) The body of work on the potential benefits of both research traditions

is also reflected in the emergence of a new transdisciplinary field of ‘cognitive semiotics’ dedicated to the multifaceted study of meaning through combining

a range of research areas such as cognitive semantics, gesture studies, ics and the embodied mind “with the ultimate aim of providing new insights into the realm of human signification and its manifestations in cultural prac-tices” (Zlatev 2015: 1043; see Sonesson 2014 and Zlatev 2015 for a useful intro-duction to visual analysis within this stream of research)

semiot-Building on the findings proposed by these pioneering studies that most types of metaphorical thinking identified by cognitive linguists can indeed be explained within the social-semiotic framework, the present study sets out to

expand their scope of analysis by considering visual representations in general –

with or without instantiating any Lakoffian conceptual metaphor – and by extending the analytical framework for application in a broader spectrum of genres other than advertisements and storytelling, applying it as well to the study of paintings and web pages

The reminder of the paper is structured as follows Section 2 provides an overview of both the social-semiotic theory of representation and the concep-tual blending theory, addressing their common objective in creating meanings

at the moment of perception Section 3 briefly outlines the analytical approach and empirical examples under scrutiny Section 4 presents the central analy-sis of combining both semiotic and cognitive perspectives to examine visual images Section 5 discusses the major ideas underpinning the synergy and its implications for visual analysis

2 Social Semiotics Theory and Conceptual Blending: An Overview

In this section, social semiotics and its theoretical assumptions will first be cussed A ‘gap’ within the social semiotic theory that can be perfectly ‘filled’ by conceptual blending will then be considered Following on from this, blending theory will be presented

dis-2.1 Social Semiotics Theory

Social semiotics is a theoretical framework that develops out of Halliday’s theories of language as social semiotic and Systemic Functional Grammar

(Halliday 1978, Halliday and Matthiessen 2004) Language, as Halliday (op cit.:

21pp) argues, realises three types of social meaning/metafunctions (i.e the ideational, interpersonal and textual metafunctions) By extending Halliday’s

conceptualisation of language to the visual arena, Kress and van Leeuwen demonstrate how these meaning functions can be realised visually through the semiotic resources of images (Kress and van Leeuwen 2001, 2006) More specifically, they describe semiotic resources as having meaning potentials and can be used to communicate ideologies and discourses.

There are four interconnected theoretical assumptions that underpin the social semiotic theory The first assumption is that meaning-making is always

multimodal, drawing on a multiplicity of modes such as image, gesture, posture,

gaze, action, music, colour, 3D objects, alongside speech and writing, all of which have the potential to contribute equally to the meaning being expressed (Jewitt 2009a: 14) The role that multimodality plays in meaning-making, as Kress (2015: 62) puts it, lies precisely in the process of exploring “the different potentials for providing means of expressing views, positions, attitudes, facts; and to enable the production of what is best suited to a specific task or need”, which is inextricably related to the second assumption central to the theory

of social semiotics, i.e., that semiotic resources are used by people in a given

social context The social context shapes the resources available for

meaning-making and how these resources are selected and configured In social ics, therefore, sign-making is conceived of as a social process (Hodge and Kress 1988: 1); “[s]igns, modes and meaning-making are treated as relatively fluid, dynamic and open systems intimately connected to the social context of use”

semiot-(Jewitt 2009b: 30) Closely related to the notion of context is that of the

moti-vated sign, which forms the basis of the third assumption When making signs,

people bring together the available form that is most apt to express the ing they wish to represent at a given moment in a specific social context The connection between form (i.e signifier) and meaning (i.e signified) within the social semiotic multimodal analysis is, therefore, not arbitrary but motivated and transparent Finally, social semiotics is built on the assumption that it is

mean-the sign-maker’s interest that guides his or her selection of semiotic resources

Kress (1993: 174) defines interest as “the articulation and realisation of an dividual’s relation to an object or event, acting out of that social complex at

in-a pin-articulin-ar moment, in the context of in-an interin-action with other constitutive factors of the situation which are considered as relevant by the individual” Es-sentially, the sign-maker’s interest is an awareness of the social environment (and the configurations of power therein) for which sign production takes place; it refers to the sign-maker’s subjectivity which is “the momentary con-densation of all the (relevant) social experiences […] produced by the need for

a response to a prompt in and by the social environment in which a new sign

is made” (Jewitt et al 2016: 68)

Recently, Kress (2010: 26–27) has postulated a rhetorical approach to sentation and communication in the spirit of the social semiotic theory In the rhetorical approach, the social world is still regarded as a dominant feature Thus, there is a great deal of emphasis on the social environment and the so-cial relations which are enacted in it and the resources available for shaping the communicative event, in keeping with the social semiotic tradition What appears to be a ‘new’ feature in the model, however, is the primary focus on the rhetor and the interpreter The rhetor is the maker of a message Prior to sign-making, Kress believes that the rhetor has to make an assessment of all aspects of the communicational situation, i.e (i) his or her interest; (ii) the characteristics of the audience; (iii) the requirements of the communicative issue at stake; (iv) the resources available for making an apt representation; (v) the best means for dissemination Kress goes on to suggest that these aspects can fall into two broad categories, namely design and production Design gives shape to the interests of the rhetor, takes into account the audience and what the matter to be communicated demands as well as accessing the potentiali-ties and constraints of the semiotic resources which are available for the most transparent representation tailored towards the goal of the communicative event On the other hand, production is the implementation of design with the material resources available in the world in which the communication takes place In production, Kress (2010: 27) notes, “meaning is made material and becomes subject to review, comment, engagement and transformation”, which paves the way for the role of the interpreter.

repre-As Kress puts it, “the interpreter’s interest produces attention; attention shapes the form of the engagement; this leads to selections being made; the se- lections are framed; there is the subsequent transformation and transductions

of the elements in the frame; and, in that, the (‘inwardly made’) sign is

pro-duced” (Kress 2010: 42; original emphasis) This model of communication rests

on two central assumptions: “that communication is the response to a prompt; that communication happens only when there is ‘interpretation’” (Kress 2010: 35;

original emphasis) Communicational environments are always complex and multimodal Any aspects of the communicational situation can, potentially, act as prompts; whether they are or not depends on the interest of the inter-preter As can be seen from the ‘communicative sequence’ proposed by Kress, the interpreter’s interest directs his or her attention to a prompt in the com-municative event; the interpreter then engages with features of the prompt and forms his or her interpretation While the meanings of the semiotic processes of ‘selection’ (i.e the highlighting of the characteristics and the

‘shape’ of the prompt which constitute the ground on which the interpretation

proceeds), and of ‘transformation’ (i.e meaning change through re-ordering

of the elements in a text in the same mode within the same culture or across cultures) and ‘transduction’ (i.e meaning change resulting from a change in modes) can be understood with ease, the notion of framing needs further elab-oration Elsewhere, in Kress’s other publications on the social semiotic theory, framing is invariably referred to as the disconnection of elements of a visual composition by frame lines, pictorial framing devices (boundaries formed by the edge of a building, a tree, etc.), empty space, discontinuities of colour, and

so on (Kress and van Leeuwen 2001: 2, 2006: 203–204; see also van Leeuwen 2005: 6–14)

In the newly formulated social semiotic theory of communication sketched here, the term ‘framing’ is imbued with a different meaning Framing is con-sidered as a trace of a “punctuation of semiosis” (Kress 2010: 122); it entails “a momentary interruption of the ceaseless flow of semiosis, provoked by some event, action, process, brought about by a message taken as a prompt for a response or by some inner equivalent of that externally produced prompt” (Kress 2010: 145) In my view, this new sense of framing is of paramount impor-tance in the shaping of a response (i.e the visible, tangible, audible, material form given to an interpretation) as it serves as a bridge between the message-prompt and the response As mentioned above, a prompt can be taken as any aspect of the communicational environment – be it a spoken comment, a gaze,

a touch, a gesture or a change in position – which is of interest to the

interpret-er who then pays attention to and engages with it The intinterpret-erpretinterpret-er undinterpret-ergoes some kind of ‘inner’ semiosis, mulling over the characteristics and the shape of the prompt – it looks as if the ceaseless, ongoing meaning-making was ‘punc-tuated’ in some way, in response to the prompt and for the purposes of making

an interpretation The ‘inner’, unmaterialised meaning is subsequently given

‘outer’ material form; it is fixed modally – as speech, as image, as gesture, as gaze – either in the same mode as the initial message-prompt as in transfor-mation, or in another mode as in transduction The result of this framing and fixing is a new semiotic entity or a new sign, ready to be used as the basis for further semiotic work Figure 1 offers a sketch of the social-semiotic theory of representation and communication proposed in Kress (2010)

2.2 Proposed Link

We have learned from the preceding subsection that signs are made in

out-ward production – as in the process of making a text where a rhetor shapes

a message, and also in inward production – as in the activity loosely called

‘interpreting’ Both, together, are fundamental Yet, I am particularly

interest-ed in the latter because without the production of any ‘inner’, inwardly-made

sign, there is no point in considering any synergy between this social otic theory of communication and a cognitive approach to meaning making

semi-As noted above, communication has happened when attention by one or more of the participants in a communicative event has focussed on a prompt

of some kind and that prompt has been interpreted by that participant On that basis, the interpreter pauses momentarily to engage with the prompt in an attempt to shape a response, apt for the communicative event This is what Kress (2010: 42, 122) refers to as ‘framing’ which is used metaphorically to mean any ‘punctuation of semiosis’, an essential step before any orchestration of modes can be settled and shaped to ‘materialise’ the interpretation Yet we still know very little about what goes on in our head at the moment when our ceaseless flow of semiosis is ‘punctuated’ and a new ‘inner’ sign is produced

as a desirable outcome, even if this ‘inner’ semiosis is addressed very itly (albeit not very clearly) in Kress’s model of communication This evidently

explic-Kress’s (2010) social semiotic theory of representation and communication

from the message

Framing of these

elements

Transformation and transduction New (‘inner’) sign

Figure 1 The model of representation and communication in the social semiotic theory

recognises the need for some cognitive insights into the inward production

of signs

Communication, which is a constant process of interpreting and ating the world, does not only rest on attending to the visual resources and their meaning potential, but also on cognitive construal and sometimes cre-ative ad hoc conceptualisations operating at the moment of perception To

evalu-explain these, Fauconnier and Turner (op cit.) propose the theory of blending

Originally, it was tailored to explain metaphors in a more flexible manner than Lakoff and Johnson’s (1980) approach Gradually, however, the focus shifted

to non-linguistic phenomena Conceptual blending theory promises insights into “the general mental capacity of blending” and the human ability “to invent new concepts and to assemble new and dynamic mental patterns” (Fauconnier

and Turner op cit.: v) It is a general theory of online conceptual combination,

applicable to a wide range of areas including “art, science, religion, culture,

sophisticated tools, and language” (Fauconnier and Turner op cit.: v) In its

present state conceptual blending theory is sufficiently detailed to allow us

to catch a glimpse of otherwise hidden mechanisms of human thinking, ticularly how we conceptualise and construe experiences and perceptions in our minds In effect, the aim of this paper is to argue for a cognitive approach

par-to complement and assist current social semiotic approaches par-to the tion of meaning construction, representation and communication I believe that it can serve as a new angle from which to approach meaning-making, in combination with the social semiotic theory The present study therefore seeks

descrip-to demonstrate that the descriptive descrip-tools provided by the blending model are capable of contributing some useful answers to problems of meaning-making and understanding that are not satisfactorily solved in the social semiotic the-ory of communication alone

2.3 Blending Theory

Fauconnier and Turner (op cit.) describe conceptual blending (also known

as conceptual integration) as a dynamic process that occurs at the moment

of perception to create new meanings from existing ways of thinking ing is a common cognitive activity, closely related to analogy and metaphor (Fauconnier 2001) Lakoff and Johnson (1999) describe metaphor as constitut-ing systematic mappings from a source to a target domain For example, in

Blend-I spent some time in France this year, money and time are the two domains that

systematically map on to each other The assumption of directionality from source to target is problematic, however As Grady et al (1999) point out, meta-phor comprehension involves more than a set of directional mappings, and there is often an active combination and blending of information from target

and source concepts Despite the fact that conceptual blending also taps into the principle of mappings, it does so through the creation of input spaces and especially through the creation of a generic (third) space to engender a blend

(a fourth space) Fauconnier and Turner (op cit.: 5) describes mental spaces

as “small packets constructed as we think and talk, for purposes of local derstanding and action They are partial assemblies of elements, structures by frames and cognitive models” In other words, mental spaces are based on more general and stable knowledge structures associated with a particular domain; they are temporary spaces used in the construction of meaning (Coulson and Oakley 2000)

un-Fauconnier and Turner’s (op cit.) theoretical framework is rich with details

on the governing principles and mechanisms of blending Perhaps the most important theoretical construct established in blending that sets it apart from metaphor and analogy is that of emergent structure “The blend develops

emergent structure that is not in the inputs” (Fauconnier and Turner op cit.:

42) There are three mechanisms for the development of emergent structure: composition, completion and elaboration

• Composition: “Blending can compose elements from the input spaces to provide relations that do not exist in the separate inputs.”

• Completion: “We rarely realise the extent of background knowledge that we bring into a blend unconsciously Blends recruit great ranges of such back-ground meaning.”

• Elaboration: “We elaborate blends by treating them as simulations and ning them imaginatively according to the principles that have been estab-

run-lished for the blend.” (Fauconnier and Turner op cit.: 48)

It is worth noting that metaphor and blending are not restricted to language nor the field of linguistics To mention just one example which is particularly relevant to our discussion on visual imagery, consumer researchers have be-gun to pay attention to the cognitive mechanisms employed by consumers to

‘make sense’ of advertisements For instance, McQuarrie and Mick (1996, 1999) and Scott (1994) investigate figurative communication by drawing on analogy and metaphor Nonetheless, these studies contain no description of blending activities, restricting their arguments to meanings constructed through the typical unidirectional, two-space model – from source to target domains – used in analogy and metaphor Almost a decade later, Joy et al (2009) present the first study of consumer research from the perspective of blending theory One of the theoretical contributions they make is to highlight the intercon-nectedness of semantic memory that allows consumers to integrate different

semantic concepts they perceive in an advertisement with a vast array of ferent lived experiences through the use of blending processes They argue that conceptual integration of this sort advances the field of advertising rheto-ric “by means of the activation, interconnection, and manipulation of mental spaces Through composition, completion, and elaboration one can illustrate

dif-how complex processes of understanding occur” (Joy et al op cit.: 48).

Evidently, their research has shown how blending theory finds its way into non-linguistic research avenues What is particularly relevant to the present study is the fact that their research also heralds an integrated approach of se-miotics and blending in decoding advertisements, although this is not taken

up seriously in the article In one of their examples, Joy et al (op cit.: 46–47)

carefully analyse participant responses to the Hummer 2 ad in which a yellow Hummer 2 sits on top of a mountain, with a deep blue sky in the background and the two-word caption in small print in the top right hand corner that reads

‘Truck Shui’ Virtually all of the participants who were not well-versed in nese culture did not recognise the Feng Shui reference.1 Yet, most of the other participants who arrived at the message were able to see the American and Chinese orientations of the Hummer 2 ad, coupling the two seemingly incom-patibles mental spaces of ‘a truck from America’ (Input Space 1) and ‘Feng Shui from China’ (Input Space 2) The generic space projects the abstract structure that guides the blending process to convey the idea that correct alignment of objects within an environment produces harmony The blend has an emergent structure that fuses the incompatibilities in the input spaces into a meaningful outcome, implying that when you drive a Hummer, you enter the stress-free mental space of Feng Shui as well as the physical space of a powerful truck

Chi-that can navigate any terrain As Joy et al (op cit.: 47) rightly point out,

“partici-pants who were sufficiently acculturated to the Chinese rhetoric and semiotic systems could decode the text with ease” They also remark that only by pay-ing close attention to layout and colours can one grasp the full import of how such an understanding can be achieved The idea of harmony is evident in the placement of the truck against the plain background (lack of obstruction) The yellow colour of the truck is clearly associated with ancient Chinese emperors, who in turn are linked to the dragon, a symbol of wealth, whereas the blue co-lour of the background is reportedly the favourite colour in Western countries.Despite mentioning it in passing, Joy and her co-authors have stressed the need for reconceptualising visual images and verbal messages in advertising

1 Feng Shui (literally meaning wind and water) is an ancient Chinese ethnoscience used in modern times in positioning buildings, rooms, entrances and furniture to ensure health, wealth, family harmony and prosperity (Fretwell 2002).

rhetoric as constituting “culturally embedded forms or signs that combine in

a specific manner to communicate meanings about brands” (Joy et al op cit.:

39), and for seeking “the substance of cultural literacy that forms and informs

a community” (Joy et al op cit.: 48) This challenge is taken up in the present

study by drawing on blending theory and social semiotics to examine visual imagery in a small set of empirical data ranging from paintings to magazine advertisements and government leaflets (see Section 3), and most importantly, proposing a synergy between the two theoretical frameworks

3 Research Data and Analytical Approach

Data for this study were taken from a large variety of visual texts including paintings, magazine advertisements, government leaflets, and web pages.2 My purpose is to analyse these various empirical data from semiotic and cognitive

perspectives Kress and van Leeuwen’s Reading images: the grammar of visual

design (2006) provides a good analytical approach to the study of visual

imag-ery With a view to mapping out the meaning potentials offered by visual texts, Kress and van Leeuwen present a system of networks adapted from the work

of Halliday (op cit.), providing a means for visual analysis that is essentially

functionalist in nature Similar to the semiotic mode of language, the mode

of image also “represents the world (whether in abstract or concrete ways) [ideational/representational metafunction] … plays a part in some interaction [interpersonal/interactive metafunction] and, with or without accompanying text, constitutes a recogisable kind of text [textual/compositional metafunc-tion]” (Jewitt and Oyama 2001: 140) The theoretical constructs developed in this framework of visual analysis will be applied to the data to examine the design processes of the image-maker (or the rhetor) The same set of data will then be subject to close scrutiny under the framework of conceptual blend-ing from the perspective of a hypothetical image-viewer (or the interpreter)

My ultimate goal is to draw an analytical bridge between the semiotic and cognitive approaches, exploring how meaning construction is not restricted to either one of these approaches and can in fact benefit from a synergy between the two

2 For copyright reasons the images discussed in this paper are not reproduced here, only resented as diagrams.

rep-4 Social Semiotics Theory and Conceptual Blending: A Synergy

As explored in Section 2, social semiotics and conceptual blending are ing signs of complementarity as the latter lends itself well to explaining ‘inner’ semiosis that cannot be resolved satisfactorily in the former In fact, this com-plementarity is evident when we look at the meaning potentials of the basic units of analysis – namely semiotic resources and mental spaces

show-Van Leeuwen (op cit.: 4) makes a distinction between the theoretical and

actual meaning potential of a semiotic resource The former has to do with the

design of a resource and its previous and potential uses and functions, whereas the latter with what is considered relevant by users of the resource in specific contexts The theoretical meaning potential puts the focus on the power of normative discourses and the fact that the meaning potentials of semiotic re-sources are in no way indefinite History, culture and social life constantly re-strict, but also enable, the meaning potentials of resources Individuals cannot

do just anything with the available semiotic resources

This is, in virtually all manners, the same kind of relationship that

cogni-tive linguists advocate in blending Fauconnier and Turner (op cit.: 321) note

that people tend to form new mental spaces based on old ones They also gest that “the brain is constantly constructing very many blends, and that only some of them are selected out for further development and application”

sug-( Fauconnier and Turner op cit.: 321) Essentially, there can be numerous

com-binations of mental spaces, leading to a potentially rich repertoire of creative blends Even though human cognitive powers are unlimited, their innovation should be guided in some way to ensure that blends are created at human scale – i.e they can evoke familiar frames that are easily apprehended by hu-man beings This is done by both constitutive principles (partial cross-space mappings, selective projection to the blend, development of emergent struc-ture in the blend) and governing or optimality principles (typology, pattern completion, integration, heightening of relations, maintenance of connec-tions in networks, perspicuity of the blend, and relevance of structure in the blend for the entire network)

In keeping with the operational constraints of both of the theoretical works, in the following examples we will see how different semiotic resources are combined and orchestrated to create meaning and at the same time how mental spaces are set up and integrated into a conceptual network

frame-4.1 Simplex Networks

Example 1 is taken from a giftbook containing some famous paintings and wisest words written about important human values such as kindness,

perseverance, hope and courage (Exley 2002).3 The most salient visual element of the painting (see Figure 2) is a wooden house due to its huge size and its central placement It is supported and carried by a group of men The painting is an ‘analytical’ picture in Kress and van Leeuwen’s (2006: 87) termi-nology It serves to identify two Carriers and to allow viewers to scrutinise their

3 I wrote to Helen Exley Giftbooks for seeking copyright permission of reproducing in this

ar-ticle the painting in Example 1 and another one in Example 4 They do not, however, hold the copyright on the paintings I am grateful to them for their prompt reply and for supplying further information about the paintings, although the copyright holders could still not be tracked down Naturally I would be obliged to make proper acknowledgements at the first opportunity should anyone get in touch with valid information about the owners of these paintings.

Figure 2 ‘Bayanihan’ by Joselito E Barcelona (reproduced from Helen Exley 2002)

Possessive Attributes One Carrier (male labourers) has Possessive Attributes such as bare feet, head down, slightly bowed body posture and the other Car-rier has Possessive Attributes such as wearing a hat, holding a rod and having

an upright body posture

This painting is a good illustration of Fauconnier and Turner’s (op cit.:

120–122) simplex network, which is the simplest kind of integration network arising from human cultural and biological history A readily available frame of human culture is indeed the society and its hierarchical structure This frame prototypically applies to human beings The simplex network exhibited by Ex-ample 1 has one space containing only this frame with only two social groups, the privileged and the underclass, and the other space containing people with disparate attributes, broadly characterised by ‘wearing a hat’ and ‘wearing

no hat’ Cross-space mapping between the input spaces is a Frame-to-values connection – the role of the privileged connects to the value of people wearing hats and the role of the underclass connects to the value of ‘hatless’ people

In the blend, the roles and values are fused so that the privileged are those people who enjoy certain privileges (symbolically signified by wearing a hat) and have power over other people (symbolically signified by the wooden rod held by them in their hands) The fact that people with hats are outnumbered

by people without hats points to a kind of inequality; a majority of people are ruled by just a very small social group

As the blending theory stipulates that eight governing or optimality ples have to be (partially) fulfilled for any integration network, these principles are applied to this simplex network The compression principle is met because the cross-space role-value connections are compressed into the vital relation

princi-of property in the blend; the two social groups now acquire their essential tributes The typology principle is satisfied since the outer-space role-value re-lation is projected to the blend The pattern completion principle is invoked through the recruitment of the ‘social class’ frame The integration principle is present because the picture reflects the workings of our society in a representa-tive way The intensification of vital relations principle is fully satisfied through the carefully coordinated visual features; the working class people share Pos-sessive Attributes in terms of head position, body posture and clothing which are clearly distinct from the ruling class The inner-space vital relation of part-whole is therefore intensified The web principle is manifested by maintaining the frame-to-value mappings between the input spaces The unpacking princi-ple is apparent in that the incongruity of the Possessive Attributes prompts us

at-to construe this visual representation as something more than a simple picture

of concerted action of lifting a heavy object The relevance principle is bodied insofar as social inequality exists and thus people belong to different

em-social classes, albeit unwillingly In fact, the painting is ‘anchored’ by a literacy quotation to this effect.4

When indeed shall we learn that we are all related one to the other, that

we are all members of one body? Until the spirit of love for our fellow men, regardless of race, colour or creed, shall fill the world, making real

in our lives and our deeds the actuality of human brotherhood – until the great mass of the people shall be filled with the sense of responsibility for each other’s welfare, social justice can never be attained (Helen Keller (1880 –1968); cited in Exley 2002)

In Example 2, a general human kinship frame is recruited to apply to the entire network (the pattern completion principle) This example is a magazine ad-vertisement about Weetabix cereals, its brief history and its family of products and how they can cater for a wide spectrum of tastes (see Figure 3) The typo-logical relationship between parent and offspring appears in the input space

of roles and is mapped onto the values in the other input As mentioned above,

a conceptual strength of the simplex network is that it can compress roles as well as establishing the compression as a single new role in the blend In this advertisement, Weetabix itself is both the parent and one of the offspring and Alpen is the other offspring Both the roles and values are projected to the blend and compressed into a new role: parent of Alpen Similar observations can be made with the help of Kress and van Leeuwen (2006)’s model of vi-sual design; the roles of parent and offspring are referred to as ‘Superordinate’ and ‘Subordinates’ respectively in a classificational process of representational meaning The visual orientation of the Weetabix advert is mainly vertical; the Superordinate (parent Weetabix) is placed above the Subodinates (offspring Weetabix and Alpen) Such kind of covert taxonomy is often used in advertise-ments to “represent the variety of products marketed under a brand name” (Kress and van Leeuwen 2006:79) The blend is further elaborated by the simu-lated mental development of the parent-child relations in that children inherit some features from their parents The inherited feature is made explicit both verbally and visually The first heading of the advert reads ‘Weetabix leads you towards a royal life’ and the visual features such as the shape of the curtain,

4 Roland Barthes (1977), one of the pioneers in semiotics, distinguishes three types of

image-text relation: (i) when the verbal image-text comes before the image, the image forms an illustration

(see Example 4); (ii) when the image comes first, the text forms a more precise statement

of the image, a relation he calls anchorage (see Example 1); (iii) the verbal text extends the meaning of the image or vice versa, i.e they are complementary, a relation he calls relay.

the pattern of the fabric and the twinkles of stars sporadically placed in the advertisement also suggest a royal connotation.

Apart from the pattern completion, typology and compression principles, this ad also incorporates other optimising principles The unpacking principle

is satisfied in that the incongruity between the two brand names, Weetabix

Figure 3 The Weetabix advertisement (Elle magazine October 2011)

and Alpen, appearing in the same advertisement prompts us to take the space

as a blend and look for its inputs The ad also satisfies the web principle as the representation in the blend maintains its mappings to the input concepts – parent and offspring relations are present in Weetabix and Alpen relations The integrating principle is applicable since the interpreter can fabricate simi-lar structure in relation to other subjects and applications For example, it is common to talk about organisations as ‘parent companies’ The vital relation

of similarity in terms of inheritance is intensified and thus becomes shaper and stronger in the blend than in the inputs The relevance principle is met once Alpen is personified, it can inherit the royal quality from her parent.Example 3 is a screenshot of the McDonald’s Café web site for Hong Kong (see Figure 4) A new kind of coffee was on offer in McCafé in winter 2011 The layout of the web page is very simple (Figure 4a): the promise of the product

is given in the heading ‘An Italian indulgence’ and the bulk of the page is voted to promoting the new drinks on the left and the cakes on the right The left-right orientation is imbued with semiotic potential in visual analysis The elements placed on the left are presented as given, as something the viewer already knows, while the elements placed on the right are presented as new,

de-as something which is not yet known This meaning potential is not entirely apparent until the screenshot fades away and changes into another one The second screenshot (Figure 4b) has a ‘given-mediator-new’ structure in that the coffee is still presented as given (i.e something that the viewer now recognis-es) but the cake is presented as a mediator for the new message which reads

‘Buy any medium or big Tiramisu coffee and you can buy any slice of cake for just $15 (originally $24 for our brand new coffee chocolate cake)’ In fact, this visual analysis can be better understood if we integrate the visual resources into a blend The blend recruits a frame with eating and drinking by pattern completion and this relation serves in the network as compressed versions of important outer-space relations of role-value The abstract roles of food and drink in one input space are mapped onto more concrete values (in this case, cakes and tiramisu coffee) in another To fulfil the promise of ‘An Italian indul-gence’, the outer-space vital relations need to be intensified: the cake makes

an ideal accompaniment for the Tiramisu coffee and it can now be obtained

at a discounted price The end result of this cognitive work is an integration network with specified values mapped onto a general human cultural frame of food and drink and a new vital-relation of property (Italianness) in the blend

4.2 Single-scope Networks

Example 4 is another painting taken from the Helen Exley’s giftbook (Exley 2002) As can be seen from Figure 5, the painting portrays a sailing boat – being judged from its size and shape – in a storm, with a rough sea and a darkening