Effectiveness of Brainwriting Techniques: Comparing Nominal Groups to Real Teams 169 4 Metrics For the nominal group data, only quantity and quality were measured since only these two m
Trang 1Effectiveness of Brainwriting Techniques: Comparing Nominal Groups to Real Teams 169
4 Metrics
For the nominal group data, only quantity and quality
were measured since only these two metrics showed
few differences between the idea generation methods
The same process as before was used (Linsey, et al.,
accepted) A new evaluator scored the quantity and
quality data for the nominal groups Prior to scoring
the nominal groups, the evaluator was trained on two
teams’ results and then two additional teams’ were
scored by the evaluator to determine inter-rater
agreement Inter-rater agreement for quantity was
92% with a Pearson’s correlation of 0.91 This
indicates there is strong agreement between the two
evaluators
5 Results and Discussion
Interacting groups with appropriate idea generation
methods can be more effective than nominal groups
The results show that real teams in rotational
conditions develop a larger number of ideas than
equivalent nominal groups (Figure 4) This result is
consistent with the theory that one of the reasons for
the observed productivity losses in real interacting
groups as compared to nominal groups is due to
production blocking (Mullen, et al., 1991; Nijstad and
Stroebe, 1999) Production blocking occurs is when
one team member is talking (producing ideas) and
other teams members are listening This causes them
not to produce ideas This result is also consistent with
other hypothesized reasons for the productivity loss
includging performance matching (individuals see how
much their teammates are producing and adjust their
productivity to match), and evaluation aprehension
(Mullen, et al., 1991; Nijstad and Stroebe, 1999)
A clear interaction effective is observed in Figure 5
through the non-parallel lines An ANOVA shows that
there is a statistical interaction between the viewing
condition and the representation meaning that both are
statistically important and the effect of the viewing
condition depends on what representation is used
[Viewing Condition: F(1,48)=2.2, p=0.15,
Representation: F(2,48)=26.3, p<0.001, Interaction:
F(2,48)=9.0 p<0.01 and MSerror=30.7] The
representation implemented does not affect the
nominal groups (individual idea generation), but has a
substantial impact for the real groups In real groups,
the representation effects the communication between
group members, whereas with individuals, the
representation mainly serves to externalize internal
ideas
The statistical analysis in this paper does not
include data from any of the sketches only conditions
because the prior study (Linsey, et al., accepted) indicates that the results from sketches only conditions are likely significantly affected by the fact that US mechanical engineers are typically not taught to free-hand draw So only the data from Words Only and Word & Sketches is analyzed and compared
To maximize the number of ideas a team generates,
a team should use annotated sketches to communicate their ideas A hybrid 6-3-5/C-Sketch method that includes rotational viewing should be implemented For an individual working alone, it does not matter what representations is used
Fig 5 Interaction effects between words only and words &
sketches representations and the viewing condition
5.1 Quality
The representation has no effect on quality (Figure 6)
or the distribution of quality (Figure 7) for the individual idea generation (nominal groups) This is not particularly surprising since the individuals are not communicating their ideas to anyone else and the quality scale is rather coarse If the quality scale were finer, it might indicate differences between the representations
Fig 4 Average number of ideas per team Error bars are
+/- one standard error
Trang 2170 J Linsey and B Becker
The various conditions do have some effect on the
quality of the ideas generated and the quality
distribution (Figure 6 - 8) The prior study (Linsey, et
al., accepted) did indicate that sketches only conditions
tended to produce both higher quality ideas on average
and fewer low quality ideas (Figure 6 and Figure 7),
but this was likely due to the fact that many low
quality ideas like “chemically removing the peanut
shell” or “genetically engineering a peanut without a
shell” are difficult to draw and therefore would have
not been included by the participants
Fig 6 Quality results Each error bar is +/- one standard
error
0
5
10
15
20
25
30
35
Condition and Team
Quality Score Distribution
# of 2s
# of 1s
# of 0s
1a 1b 2a 2b 3a 3b 4a 4b 5a 5b 6a 6b
Fig 7 Distribution of team quality scores (Quality Scores
1=technically feasible, 2=feasible for the context)
The quality results indicate that words only should not
be used for a large number of quality ideas in a team setting The viewing condition (gallery verse rotational) had little effect on the average quality or the distribution
These results indicate that when teams implement
an effective method for idea generation, they can outperform the combined results of individuals (nominal groups) This result is in contrasted to results from Osborn’s Brainstorming method where nominal groups generally outperform real teams (Mullen, et al., 1991)
6 Conclusions
Brainwriting techniques that include a combination of sketches with annotations, such C-Sketch or Gallery, can assist a team in creating more ideas than the combined efforts of the same number of individuals working alone with redundant ideas removed, referred
to as “nominal groups” In contrast to this, prior experimental results from other studies on Osborn’s Brainstorming show that interacting groups are less effective than nominal groups These results indicate that designers should carefully select their group idea generation approach in order to obtain a successful process
To maximize the impact of a group idea generation, teams should sketch their ideas and add annotations to enhance interpretation Methods where individuals can all simultaneously work as opposed to methods where one person speaks at a time (e.g Osborn’s Brainstorming), will produce a greater number of ideas A hybrid 6-2-5/C-Sketch method, where teams sketch adding annotations and then rotate ideas, is best for group idea generation
This study compared nominal groups to real groups using techniques very similar to 6-3-5, C-Sketch and Gallery Nominal groups were compared to real groups
in a 3X3 factorial experiment The first factor was how the teams represented their ideas (words only, sketches only or words & sketches) and the second factor controlled how ideas were exchanged (rotational viewing, gallery style, or no exchange-nominal groups) This factorial design leads to teams generating ideas in conditions very similar to 6-3-5, C-Sketch and Gallery It was found that real teams using rotational viewing (e.g., 6-3-5, C-Sketch) created a greater number of ideas as compared to nominal groups In contrast to this, real teams using gallery viewing produced significantly fewer ideas than the nominal groups
Fig 8 Quality score distribution for individual idea
generation (combined to form nominal groups)
Trang 3Effectiveness of Brainwriting Techniques: Comparing Nominal Groups to Real Teams 171
References
Adams JL, (1986) Conceptual Blockbusting Cambridge,
MA: Perseus Books
Aiken M, Vanjani M, Paolillo J, (1996) A comparison of
two electronic idea generation techniques Information &
Management 30: 91–99
Baxter M, (1995), Product Design: A practical guide to
systematic methods of new product development
London: Chapman & Hall
Gryskiewicz SS, (1988) Trial by Fire in an Industrial Setting:
A Practical Evaluation of Three Creative
Problem-Solving Techniques In Innovation: A
Cross-Disciplinary Perspective, (Gronhaug K, Kaufmann G,
Eds.), 205–232 Oslo: Norwegian University Press
Higgins JM, (1994) 101 Creative Problem Solving
Techniques Winter Park, FL: The New Management
Publishing Company
Lewis AC, Sadosky TL, Connolly T, (1975) Effectiveness
of Group Brainstorming in Engineering Problem
Solving IEEE Transactions on Engineering
Management EM-22:119–124
Linsey JS, Clauss EF, Kurtoglu T, Murphy JT, Wood KL,
Markman AB, (accepted) An experimental study of
group idea generation rechniques: Understanding the
roles of idea representation and viewing methods ASME
Journal of Mechanical Design
Mullen B, Johnson C, Salas E, (1991) Productivity Loss in
Brainstorming Groups: A Meta-Analytic Integration
Basic and Applied Social Psychology 12:3–23
Nijstad BA, Stroebe W, (1999) Persistence of brainstorming
groups: How do people know when to stop? Journal of
Experimental Social Psychology 35(2):165–185
Otto K, Wood K, (2001) Product Design: Techniques in Reverse Engineering and New Product Development Upper Saddle River, NJ: Prentice Hall
Pahl G, Beitz W, (1996) Engineering Design – A Systematic Approach New York: Springer
Paulus PB, Yang HC, (2000) Idea Generation in Groups: A Basis for Creativity in Organizations Organizational Behavior and Human Decision Processes 82:76–87 Shah JJ, (1998) Experimental Investigation of Progressive Idea Generation Techniques in Engineering Design DETC’98, 1998 ASME Design Engineering Technical Conferences, Atlanta, GA
Shah JJ, Kulkarni SV, Vargas-Hernández N, (2000) Evaluation of Idea Generation Methods for Conceptual Design: Effectiveness Metrics and Design of Experiments Transactions of the ASME Journal of Mechanical Design 122:377–384
Shah JJ, Smith SM, Vargas-Hernandez N, (2003) Metrics for measuring ideation effectiveness Design Studies 24:111–134
Shah JJ, Vargas-Hernández N, Summers JS, Kulkarni S, (2001) Collaborative Sketching (C-Sketch) – An Idea Generation Technique for Engineering Design Journal
of Creative Behavior 35:168–198 Van der Lugt R., (2002) Brainsketching and how it differs from brainstorming Creativity and Innovation Management 11:43–54
VanGundy AB, (1988) Techniques of Structured Problem Solving New York: Van Nostrand Reinhold Company Vidal R, Mulet E, Gómez-Senent E, (2004) Effectiveness of the Means of Expression in Creative Problem-Solving in Design Groups Journal of Engineering Design 15:285–
298
Trang 5
Methods and Tools for Design Creativity
Front End Industrial Design (FE-ID) - Developing New Tools and Models for Industrial Designers to Operate at the Front End of New Product Development
Paul W Wormald
Virtuality – Offering Opportunities for Creativity?
Anthony Williams, Ning Gu and Hedda Haugen Askland
Thinking Inside the Box: Model, Tool, Team and Setting for Building Design
Wim Zeiler
Signs of Collaborative Ideation and the Hybrid Ideation Space
Tomás Dorta, Annemarie Lesage, Edgar Pérez and J.M Christian Bastien
Trang 7Front End Industrial Design (FE-ID) - Developing New Tools and Models for Industrial Designers to Operate at the Front End of New Product
Development
Paul W Wormald
National University of Singapore, Singapore
Abstract The front end of new product development is
often a focus for discussion about innovation This paper
presents a new model for how the role and place of industrial
design can be re-positioned so that it has influence and
impact at this front end The aim of this is to develop the
capability for industrial designers to generate creative ideas,
or opportunities, that have resonance and relevance within
the context of new product development, but that are made
explicit before a design brief exists A front end industrial
design process model, developed for undergraduate
designers and evaluated by industry, is described in detail
Keywords: industrial design, ideas, new product
development, front end innovation, undergraduate design
education
1 Introduction
Design creativity should be about ideas as well as
beautiful artefacts Ideas, as well as designed
outcomes, need to be 'beautiful' So a debate
surrounding design creativity can include some
discussion about ideas Beautiful ideas which are
successfully resolved into products are often judged as
attractive product innovation Creative product
innovation is a highly prized goal for most commercial
enterprises Von Stamm (2003 p2) states "One of
the big concerns for many companies is how to
generate more and better ideas - how to become more
creative."
This paper describes an approach which aims to
develop and enhance creative ideas by moving the
influence and impact industrial design has on new
product development (NPD) to before, as well as after,
a design brief is written This, and the desire to have
those ideas rooted in evidence and sound process,
requires new knowledge and abilities, and new modes
of working, including new tools and designerly
outputs
What has emerged, mainly from industrial design
education curriculum development, is a overall process
model which could be adopted in a re-think of how industrial design can better contribute to successful NPD (in commercial and other enterprise arenas) This process model has been refined over half a decade of university-level design education It has been reviewed
by large international companies, design consultancies and design research companies
The aims of this paper are:
To reveal and illustrate a process through which industrial designers can successfully impact on the front end of NPD, partly by generating creative and targetted ideas
To contribute and stimulate some debate and discussion concerning the future role of industrial design in NPD
2 The Front End of NPD
Figure 1 shows a basic diagram of commercial NPD
Fig 1 NPD diagram
The diagram identifies various stages and activities Obviously, this diagram is not to scale in the sense of time It shows that design (industrial, engineering etc.) typically starts after a brief is formulated Its major feature is the (pre-brief) area before a formal design brief emerges
It is well known that companies will attempt to utilise methods and approaches to enable successful innovation in the development of new products The
Trang 8176 P.W Wormald
methods and approaches used very early on in the new
product development process are often referred to a
'front-end' processes (Koen 2002 and Cagan and Vogel
2001) They can include such activities as
user/customer research, brand management, trend
surveys, and market analysis Additionally, these
methods and approaches are often seen as being less
strict, rigorous or even less well understood than some
of the 'downstream' processes such as product design,
manufacturing engineering, and product certification
Hence, this front end of NPD is sometimes referred to
as the 'fuzzy front end' (FFE)
With regard to industrial design activity in the
early NPD processes then Veryzer (2005) and
Jurotovac (2005) demonstrate how industrial designers
make successful contributions to the overall process
There has been limited pedagogic research into the
issue of how industrial design education can exploit
these new areas of opportunity Design students
pursuing user research activities early in project work
is discussed by Siu (2003, 2007), Lopes (2008) and
Lofthouse (2008) They all point to the potential
benefits of enhanced innovation and designer empathy
with users
Some of the background issues concerning FFE
thinking are covered by Wormald (2009) He provides
a full background to this paper, particularly the drivers
for change which provided the impetus for industrial
design educational curriculum development
3 A Process Model for Front End Industrial Design (FE-ID)
Figure 2 shows a process model for the front end of NPD activities undertaken by industrial designers, hence the term front end industrial design (FE-ID)
It is important to note that this model has been formulated following six annual cycles of pedagogic action research This action research was instigated to investigate and support changes to the author's curriculum development in the subject of industrial design at a UK university
The diagram is a more detailed view of how the role of industrial design can be modelled during that, notoriously 'fuzzy' period The diagram has been formulated to clarify and visualise the various stages and outputs of the front end investigation and synthesis processes
The following sections describe each of the major processes, with associated tools and output models Examples of work completed by industrial design undergraduates are presented Each stage or step in the process can be broken down into sub-areas for subsequent analysis and possible synthesis The generated outcomes can lead to further stages and further subsequent analysis
The process begins with a review, or revealing, of various contextual issues Within the broad, fuzzy, arena of very early new product thinking there will be
Fig 2 FE-ID process diagram
Trang 9Front End Industrial Design (FE-ID) - Developing New Tools and Models for Industrial Designers at the FE of NPD 177
a multitude of influences and pressures These are
indicated in the NPD diagram (fig 1)
There is, of course, no sense of what the nature of a
new product will be There is no 'big idea', there is no
design brief, there is only a sense, or urge, that a new
product is necessary The contextual issues are stated
as clearly as possible, but with enough flexibility to
allow for exploration and wide-ranging relevant
research
Four context areas are identified:
User/consumer
Scenario/theme
Global/PEEST
Company/brand
For 'user/consumer' a target user group is outlined
This usually entails identifying simple demographic
information such as age range, gender, and occupation
For 'scenario/theme' some broad user activity or
behaviour is outlined This would have relevance to
the target user group and the company/brand
Examples could be 'cooking', 'keeping fit', or 'local
travel' PEEST stands for Political, Economic,
Environmental, Social, Technology There will be a
'global/PEEST' context to be investigated, especially
relating to the overall theme being explored Each of
the Political, Economic etc areas can be reviewed
for possible insights Finally, the company and its
relevant brand is a necessary component of any sound
NPD thinking
Following the above basic clarification, the context
areas can be researched This research can be
conducted independently, in parallel, by a team of
design researchers Investigation can be overlapping
and related, but it is best to be able to have a view of
the separate areas initially The different contexts will
be researched using different methods and strategies
Different types and forms of data will be gathered
Possible research methods used and types of data
gathered for each context area are identified below
Context - user/consumer
Data gathering activities:
- Ethnographic observation
- Interviews, questioning
- Conversation, chatting
- Photography, video
Types of data gathered:
- User stories
- Detailed demographic information
- Photos of people and environments
- User attitudes and emotions
- User quotes
- Observational texts
Context - scenario/theme
Data gathering activities:
- Observation
- User interviews
- Photography, video
- Internet searching
Types of data collected:
- Photo essays of behaviour
- User stories of scenario experiences
- Observational texts
- Comment and opinion on theme
Context - global/PEEST
Data gathering activities:
- Internet searches
- Newspaper, trade magazine reading
- Media analysis
Types of data collected:
- News snippets/articles
- Technical briefing notes
- Cultural references (photos, notes)
Context - company/brand
Data gathering activities:
- Analysis of company publications and websites
- Discussion with company management
- Identification and analysis of competitor products/brands
- Consumer interviewing
Types of data collected:
- Product (company and competition) imagery
- Company financial and market information
- Brand values
- Forms of brand manifestations
As data gathering progresses the process of attempting
to make sense of it all begins As meaning and sense develops for the design researchers, increased understanding of the context areas will drive new avenues of investigation in an iterative way The design researchers will aim to synthesise specific areas
of 'meaning' from the various data sources Arising from the context areas these areas of meaning include:
User goals;
User lifestyle;
Insights into user behaviour;
The experience of the user relating to the scenario/theme;
Relevant PEEST insights;
Company market position - and subsequent insights;
Trang 10178 P.W Wormald
Brand territory;
Illustration of brand values - characteristics
In this work, an insight can be the answer to the
question "What do we now understand from the
evidence, that we did not know before?" Insights are
highly valued and valuable 'nuggets' of information
that relate strongly to the overall contextual issues
For the industrial designer an important part of the
process of making sense of the data takes the form of
communicating its meaning in particular and
specialised forms, namely 2D boards (printed or
digital) containing rich visual and textual information
The industrial designer uses standard graphical
techniques to present meaning and understanding in
easily accessible and engaging forms In the FE-ID
process (fig 2), these are identified as BRAND,
PERSONA, and EXPERIENCE These are separate
boards, and a more detailed explanation of each, with
examples, follows
BRAND
The aim of this board is to 'bring the brand to life' It
should reveal the brand territory (the metaphorical
ground the company's brand occupies in the market) It
should illuminate the underlying meaning of the brand
values It should successfully explain and illustrate any
of the brand messages (such as taglines and jargon
used in advertising and marketing) The content of the
BRAND board would typically include:
Company name;
Brand name;
Brand visuals (such as logotype, product visual
language);
Iconic products of the brand
Analogous, comparative products / brands /
competitors;
Explanatory texts
Fig 3 Example of BRAND board
PERSONA The aim of this board is to 'bring the user to life' A persona is a standard, accepted way of defining and visualising the target user group It is often used in marketing activities, and software design (Cooper 1999), and product design (Pruitt and Adlin 2006) There may be several personas for each project The content of each the PERSONA board would typically include:
Photograph of a person (with 'character');
Name;
Categorisation (a form of 'micro' description);
Basic demographic facts (such as age, occupation, location);
Lifestyle information (pictures and text), to paint a picture of the character;
Goals, needs or expectations which are specific
to the scenario/theme;
A short narrative, telling a story relevant to the user and theme
It is important to note that the details above are highly credible, but are actually fictional This is partly for ethical reasons
Fig 4 Example of PERSONA board
EXPERIENCE The aim of this board it to 'bring the user experience (of a specific scenario/theme) to life' The emphasis on this board is to reveal the actual behaviour of users, not simply to report what the users say that they do Stappers and Sleeswijk (2007) describe an approach they call "Context Mapping" which aims to reveal similar issues surrounding user experience The content of the EXPERIENCE board would typically include:
Photos of users in relevant environments and situations;
Photos of users actually doing tasks, jobs, activities;