Developing and testing a serious game to train creative thinking

Three cognitive components of creativity, divergent thinking, convergent thinking and insight problem solving of 27 university students were measured before and after their four-week gam

VU BICH PHUONG

DEVELOPING AND TESTING A SERIOUS GAME TO

TRAIN CREATIVE THINKING

MASTER SCIENCE THESIS IN BEHAVIOURAL SCIENCE

NIJMEGEN, THE NETHERLANDS – 2018

VU BICH PHUONG

DEVELOPING AND TESTING A SERIOUS GAME TO

TRAIN CREATIVE THINKING

MASTER SCIENCE THESIS IN BEHAVIOURAL SCIENCE

Supervised by: Simone Ritter & Frank Leone

Department: Social cognition

NIJMEGEN, THE NETHERLANDS – 2018

Developing and Testing a Serious Game to

Train Creative Thinking

Abstract

Creativity, the ability to generate new and useful ideas, is one of the most desirable employability skills in the twenty-first century Despite this need, creative ideas are scarce because we have a tendency to activate semantically related information and concepts and thus are constrained from producing new, unconventional ideas This research aims to develop and test a gamification training program that helps the practice of far conceptual combination In Study 1, a computerized game was developed and validated by correlating 49 participants’ game performances with several creativity measures The results suggest that divergent thinking is central and necessary for game performance Subsequently, the effectiveness of this game was examined using a pre- and post-training design in Study 2 Three cognitive components of creativity, divergent thinking, convergent thinking and insight problem solving of 27 university students were measured before and after their four-week game training It was found that divergent thinking and convergent thinking significantly improved following this creativity game training Potential for an effective gamification approach and possible improvements for the game training was finally discussed

Keywords: creative thinking, game training, divergent thinking, convergent thinking,

insight problem solving

Developing and Testing a Serious Game to Train Creative Thinking

In the fast-changing and uncertain economic outlook, creative thinking has been one of the top five employability skills of employees that organizations desire (Right Management, 2014) Not only in the workplaces of arts, business and sciences, creative thinking can also promote alternative perspectives and offer solutions for daily life’s problems (Tannenbaum, 1997) As such, researchers in various disciplines such as psychology, management and education have been exploring the ways to enhance creative thinking Previous studies have developed several training programs and suggested that creativity can be trained (for example, Feldhusen, Treffinger, & Bahlke, 1970; Im, Hokanson, & Johnson, 2015; McFadzean, 1998; Ritter & Mostert, 2016) Specifically, a meta-analysis on training effectiveness suggested that training programs targeting cognitive components of creativity often yield the best results (Scott, Leritz, & Mumford, 2004a) Given the importance of creativity, the objectives of this research were to develop and examine the effectiveness of a training program that targets the underlying cognitive mechanism of creative thinking In the following paragraphs, the cognitive foundation of this creativity training will be elaborated on Then, several elements and their advantages of this training program will be specified

Creativity commonly refers to the generation and recognition of new and useful ideas (Mumford, 2003) However, the fact that creative thoughts happen less often than routine, conventional thoughts can be explained by a cognitive structure called schema, which maps knowledge into domains and categories (Gentner, 1983) A conceptual structure theory proposes that, because of this cognitive mapping of knowledge, concepts (such as objects and events) and their compatible attributes are processed within a single semantic space (Simmons & Barsalou, 2003) This semantic relatedness explains, for example, why we often have the first thought about “apple” as an exemplar of fruit, instead of thinking about

technologies (Apple Inc.) Moreover, the more compatible properties the concepts share, the more likely they will be activated together For example, “dog” is closely related to “cat” because they are both pets, have fur and four legs This tendency to process information in schemas also explains why we often reach a common, conventional solution when solving problems However, once problems present information in an unexpected manner, such as in case of riddles, conventional thinking that is processed by schemas cannot provide alternative perspectives to solve these problems For example, in this riddle: “What goes up and down but does not move?”, “go” and “move” both describe changes in location Therefore, this challenging, unexpected situation requires a newer, more creative approach to solve It is when we realize that, instead of thinking about something that goes up and down in terms of positions, it will make more sense if we think about something that increases and decreases (and the answer is “temperature”)

Because existing knowledge is stored by cognitive schemas and cannot offer novel perspectives to solve problems, escaping the existing frameworks and proximity of information may facilitate the production of novel ideas, and possibly creative ideas (Sassenberg & Moskowitz, 2005) It is suggested that mental synthesis of previously unrelated concepts can help generate new properties, functions or associations for them (Ward, Smith, & Finke, 1999) This mental synthesis of concepts is known as conceptual combination The novelty of such combinations depend on how distant the concepts are in terms of semantic relatedness (Howard-Jones, Blakemore, Samuel, Summers, & Claxton, 2005), or the degree of incompatible attributes in the parent concepts (Wan & Chiu, 2002) The more distant, dissimilar they are, the more unconventional the conceptual combination is (Chan & Schunn, 2015) It has been found that conceptual combination facilitates divergent thinking, the ability to generate new ideas and a major cognitive process in creativity (Mumford, 2003) In the study of Wan and Chiu (2002), an experimental group who was

trained in novel conceptual combination showed significant improvements in the Torrance Tests of Creative Thinking (Torrance, 1974), a standard test of creativity involving divergent thinking skill, while a control group did not In another study, stories created by using a set of unrelated words were judged as more creative than by a set of related words, suggesting that semantic divergence in combining concepts is necessary for creative thought (Howard-Jones

et al., 2005) The importance of conceptual combination in creativity is also supported by other empirical studies (Baughman & Mumford, 1995; Finke, Ward, & Smith, 1992)

Given that cognitive training with a focus on unique requirements of creativity is more likely to be effective (Scott et al., 2004b), the training program developed in this research will incorporate the practice of conceptual combination to train creative thinking Given the four main accounts to be considered when designing a creativity training program (Scott et al., 2004b) including targeted cognitive process and skill of creativity, means of delivery, training technique and type of exercises, the following parts will describe our training program with respect to these elements Firstly, in terms of targeted cognitive process and skill, the training program in this research focuses on developing conceptual combination and divergent thinking skill This cognitive approach has been shown to have large effect size and high successful rate (Scott et al., 2004b) Secondly, the delivery of our training is computerized and gamified It has been found that game-based learning environment can be entertaining while also engaging the leaners and enhancing their intrinsic motivation (Azriel, Erthal, & Starr, 2005; Hamari et al., 2014; Kang & Tan, 2014) By means of a computerized game, learners can be autonomous and flexible in their own training because no formal, in-person lectures are incorporated Moreover, this computerized game can be programmed to have short training sessions to avoid lengthy duration, which is common in other cognitive training programs and disadvantageous for their real-life implementation (Ritter & Mostert, 2016; Scott et al., 2004b)

Thirdly, next to the practice of conceptual combination, our game training uses visualization as a creativity technique It is suggested that, because visual images can be scanned and mentally transformed, this will benefit the creative thoughts by allowing rapid detection of incompatible properties of objects or events (Finke, 1996) In the literature, the studies which target distant associations to enhance creative thinking have also applied imagery technique (Scott et al., 2004a) Moreover, using universal images can ensure that the training is language independent Therefore, the practice of conceptual combination in the game is based on visual, colored representations of a wide range of concepts, taken from different semantic categories For example, there are “dog”, “lion” and “fish”, which are typically known as animals, and there are “durian”, “cherry” and “orange”, which are typical fruits This collection of both related (“dog” and “lion”) and unrelated (“fish” and “durian”) concepts will encourage players to visualize and generate associations among them It also leads to the advantage of the final element, type of exercise, in our game training By using colored images of familiar, common concepts that can be encountered every day as training stimuli, the type of practice in this training can be domain-unspecific This means that the skill gained through the exercises can be applied in various settings without specific technical knowledge, making this game training appropriate for the general population

Taken together, this research aimed to develop a computerized game, which provides

an environment for practicing conceptual combination based on visual representations of concepts The game was programmed in PsychoPy 2 (Peirce, 2007) and several criteria were developed to measure game performance Subsequently, this research aimed to examine the relationship between game performance and creative thinking in two studies In Study 1, we validated the game by correlating game performance measures with several creativity tests scores We hypothesized that there were positive correlations between game performance and the creativity measures, suggesting that creative thinking is necessary to perform the game

Additional information about the participants’ personalities and their opinions about the game were also collected In Study 2, the effectiveness of this game in training creativity was investigated using a pre- and post-treatment design A group of young adults underwent the training by playing the game three times per week in four weeks We hypothesized that their performances in the creativity tests could be improved following the game training

STUDY 1

Method

The aim of this first study was to develop and validate a computerized game, named

“The Associates”, by correlating performance in validated creativity tests with performance

in the game

Participants A total of 49 undergraduate university students (40 females) from the

University of Social Sciences and Humanities Ho Chi Minh city (USSH), Vietnam, were

recruited with flyers (Appendix A), email and social media Their mean age was 20.71 (SD =

2.01) Most of them (94%) were majored in psychology Participant codes were used to ensure anonymity The experiment took approximately one hour to complete and the participants were rewarded with ₫30,000 (equivalent to €1.20) This study was approved by the Ethics Committee, Faculty of Social Sciences, Radboud University Nijmegen, the Netherlands, coded ECSW-2017-017R2, and by the Faculty of Psychology, USSH, Vietnam

Creativity measures Prior to the main testing, a pilot study was conducted to examine

how Vietnamese participants perceived and understood the instruments that were translated from English to Vietnamese (Appendix B) All instructions and materials used throughout this research were in Vietnamese language and programmed to present in PsychoPy 2 (Peirce, 2007)

Divergent thinking Researchers have recognized that creativity is not a single act but a

complex phenomenon, involving, but not limited to, generating many ideas and selecting the most creative, suitable idea (Mumford, 2003) Thus, one of the creative processes to be measured in this study was divergent thinking, the ability to generate multiple ideas or solutions The Alternative Uses Test (AUT, Guilford, 1967) is commonly used to evaluate divergent thinking skill In this test, the participants were required to list as many uses for a common object as possible in four minutes All participants were presented with two test versions, a brick and a newspaper (Appendix C) The order of the versions were counterbalanced across the participants Following the AUT scoring guidelines (Guilford, Christensen, Merrifield, & Wilson, 1960), performance in the AUT was judged based on how many ideas they could generate (fluency), how flexible the participants were in changing perspectives (flexibility), and how novel and useful the ideas were (creativity)

For the fluency criterion, the total number of complete, comprehendible ideas was counted For the flexibility criterion, the total categories of ideas was calculated A category

is a group of ideas that share similar properties For example, when asked to list alternative uses of a newspaper, “wrap a burger” and “wrap a bouquet flowers” can be grouped in a category “wrapping things”, whereas “beat a fly” and “beat someone” can be grouped into

“beating things” A list of all possible categories was recorded to assist the scoring For the creativity criterion, each listed idea was given a score from 1 (not at all creative) to 5 (very creative) based on the first impression of its novelty and usefulness Each idea and its given creativity score were then recorded in a list to ensure consistent scoring across participants Then, a sum score of all ideas for each participant was calculated Because this sum score is dependent on quantity, as having many less creative ideas may give a higher score than having a few highly creative ideas, a mean score of creativity was calculated by dividing the sum creativity score by the number of ideas, which is the fluency score, for each participant

Convergent thinking While divergent thinking is crucial for idea generation,

convergent thinking, the ability to select a single best, most correct answer to a problem, is also important for the creative process The Remote Associates Test (RAT, Mednick, 1962) was used to measure convergent thinking skill In this test, a combination of three seemingly

unrelated cue words were presented (for example, lapse - vivid - elephant) The participants

were asked to find a fourth word that can relate to all the cue words to form three meaningful

words (the answer is memory: memory lapse, vivid memory, elephant memory) Because the

original version of RAT was in English, a Vietnamese version of RAT was constructed and piloted (Appendix B) For Study 1, the participants were presented with ten three-word combinations (Table B4) They had 20 seconds to solve each combination, and the order of the combinations was randomized Each correct answer earned one point, while an incorrect score earned zero A sum score of total correct answers for RAT was calculated and the highest possible score was ten

Insight problems solving Besides idea generation and idea selection, creative thought

is also crucial when solving problems, especially insight problems that represent information

in an unexpected, challenging manner Although minimal technical knowledge is required, these insight problems can only be solved by overcoming functional fixedness, in which typical thought and perceptions about the problem block awareness of the solution (Batchelder & Alexander, 2012) In Study 1, after piloting the instruments, the Duncker’s candle problem (Duncker, 1945) and the Maier’s two-string problem (Maier, 1931) were used

to measure creative problem solving The order of the problems was randomized (Appendix D) The participants had three minutes to read and solve each problem Each correct answer received one point while an incorrect answer received a zero A sum score of total correct answers for insight problems was calculated and the highest possible score was two

Game instruction video Instructions on how to play “The Associates” game was

recorded by a video using the built-in screen recording function of Windows 10 In this video, the gaming screen was showed and accompanied with verbal explanations about how to play the game and how to earn scores

Demographic data and personalities A short demographic questionnaire in

Vietnamese was used to collect data about the participants’ age, gender and major subject Self-reported creativity was also measured on a Likert scale from 0 (not very creative) to 4 (very creative) A short, standardized version of the Big Five personality scale (Van Eijck, &

de Graaf, 2004) was translated into Vietnamese to collect subjective measures of extroversion, friendliness, conscientiousness, emotional stability and openness (Appendix E) This questionnaire includes 30 items, in which six items assess each personality trait on a Likert scale from 1 (strongly disagree) to 5 (strongly agree) Cronbach’s alpha indicated that this scale has high reliability (Van Eijck, & de Graaf, 2004) and correlates strongly with the Goldberg’s original Big Five scale of personality (Goldberg, 1992, Gerris et al., 1998)

Feedback for the game A questionnaire was used to assess whether the instructions for

the game were sufficient, and whether the participants enjoyed playing “The Associates” game (Appendix F) It also asks for suggestions and recommendations for improvements This questionnaire includes 10 multiple-choice and short-answer questions

Design This section specifies how “The Associates” game was developed with figures

illustrating “concept cards”, layout of the gaming screen, explanations of game rules, how to play the game, and how to score game performance

Development of the game To operate the game, an imagery database of concepts was

developed In total, there were 435 concepts, which were grouped into 21 categories based on their most common, typical properties (Table 1) For example, concept “banana” is grouped

into category “fruit”, concept “badminton ball” is grouped into category “sport”, and concept

“boots” is grouped into category “fashion” The images were presented in square-framed cards with a white background, showing clear real-life representations of each concept (Figure 1)

Figure 1 Examples of concept cards “banana”, “badminton ball”, and “boots”

The game was designed to have three rounds Each round presented 43 concepts cards

on the screen, and the participants had 10 minutes to form associations among the concept cards This timing was decided after trials of the game because it was not too short so the participants could familiarize with the game and not too long to avoid boredom The game was programmed to proceed automatically to the next round after 10 minutes In Study 1, we pre-defined three different sets of 43 concepts cards to ensure the same game condition across three rounds for all participants An equal number of cards from all categories were randomly selected This also ensured that the same concept card was not shown twice across three rounds The order and arrangement of these cards on the screen were fixed for all participants

To begin, 43 concept cards were displayed on a virtual screen in visually satisfactory size, with 42 cards arranged in a circle and the 43th card, called the starting card, presented in the middle of the screen At the bottom of the screen, the round number and a timer were showed At the left corner placed a “How to play” button, which the participants could click

on to read the instructions again while playing There was also a “Next round” button at the right corner, which could be used to proceed to the next round if the participants finished playing before 10 minutes (Figure 2)

Figure 2 Screenshots when the game starts (above) and when the game ends (below)

Game rules and how to play Before playing, the participants were asked and

encouraged to form as many creative associations between the concepts as possible within 10 minutes (Appendix G) The participants were required to begin with the starting card in the middle of the virtual screen (“snail” in Figure 2) To form an association, the participants used a computer mouse to click on a card that they could associate with “snail” and placed it next to this “snail” card The participants continued to form other associations with the two cards that had been placed in the middle screen, until all 42 cards were used or when the timing was finished For each association they made, a verbal explanation was required The

participants could freely associate the concepts using their properties and attributes, although associations based on the color of the images were not encouraged and deemed invalid as real-life concepts may have different colors These game rules were summarized and could be accessed by clicking the “How to play” button (Appendix H) The computer screen and the participants’ talks while playing were recorded for scoring and analysis Moreover, the game was programmed to record the screenshots at 0:00 minute, or when the players clicked on the

“Next round” button

Scoring game performance Five game performance measures were developed,

including fluency, flexibility, creativity, density and intersection Specifically, the criteria

fluency, flexibility and creativity were developed in accordance with the three scoring criteria

in the divergent thinking test (AUT) On the other hand, density and intersection were

game-specific (Appendix I)

The fluency score was the total number of the cards used to form associations Each

played card in the middle screen earned one point The possible highest point was 43

The flexibility score was the total number of associations with valid, comprehendible

explanations Each explanation for an association earned one point For example, participant

A earns one point by associating “fridge” with “eye glasses”, explaining that they are “things made of metal” (Figure 3) In case participant A continues to associate “eye glasses” with

“cctv” by the same “metal” explanation again, this does not earn a point If “eye glasses” and

“cctv” are associated by a new explanation, for example “serve to see”, participants A will earn another point A sum score of flexibility was calculated for all participants The higher the score was, the more flexible the participants were in changing perspectives and approaches when forming associations

Figure 3 Examples of associations “fridge – eye glasses” as “things made of metal”, and

“eye glasses – cctv” as “things serve to see”

Each explanation for the associations was then given a creativity score This was

judged based on the novelty and usefulness of the ideas, on a Likert scale from 0 (not at all creative) to 4 (very creative) To account for the dependence on quantity, a mean score of creativity for each participant was calculated by dividing the sum creativity score by the number of associations This means that the mean score of creativity equaled the sum score of creativity divided by the sum score of flexibility Importantly, 30% of the sample was scored

by three Vietnamese raters to analyze the consistency and agreement in judging creativity The inter-rater reliability of the creativity scores was calculated with a two-way random intra-class correlation coefficient (ICC) analysis and considered substantial (ICC = 0.82)

The density criteria was developed to capture the density of associations Back to the

example of “fridge – eye glasses” as “metal” and “eye glasses – cctv” as “serve to see”, the concept “eye glasses” has two neighboring cards, while “fridge” and “cctv” only have one each In this case, a density score of two is assigned to “eye glasses”, while a density score of one is be assigned to “fridge” and “cctv” Thus, a higher sum score of density indicated that the concept cards were placed densely next to each other (Figure 4)

Figure 4 An example of game density Each card gets a point depending on the number of its

neighboring cards

Finally, an intersection score was given when the participants associated three, four or

five concepts at the same time, instead of forming a usual two-concept association For example, participant A continues to place “computer” next to “fridge” and above “cctv” to associate them as “electric devices” (Figure 5) In this case, the concept card “computer” is called an intersection card To ensure valid explanations were given for all associations in these intersections, this criterion was scored step-wise according to the screen recordings of the participants An intersection score of four was assigned to the intersection card when three cards were associated at the same time (as in the example of concept card “computer”), nine points to the intersection cards when four cards were associated at the same time, and 16 points to the intersection card when five cards were associated at the same time (Figure 6)

Figure 5 An example of intersection card “computer” when three cards “fridge –computer –

cctv” are associated at the same time by an explanation “electric devices”.

1 4 1

1

1

1

Figure 6 Intersection scores for when participants associate three, four or five cards at the

same time (yellow cards are intersection cards)

Procedure Recruited participants were welcomed to a quiet lab room at the Faculty of

Psychology, USSH Upon their arrival, they were given verbal and written instructions (Appendix J) The participants gave their consents (Appendix K) and were assigned with a participant code to ensure anonymity Then, the participants sat in front of a computer The order of testing was fixed for all participants, starting with the AUT test, then the RAT test and finally the insight problems Two versions of the AUT were counterbalanced, while the order of the ten RAT combinations and the insight problems were randomized Next, before playing “The Associates” game, the participants watched the instruction video to understand the game rules and how to play Then, the screen recorder was turned on, and the participants played three rounds of the game in thirty minutes They finished the experiment by completing the demographic questionnaire, the short Big Five, and the game feedback questionnaire

Plan of data analysis Data was analyzed using the Statistical Package for Social

Scientists (SPSS, version 22) Divergent thinking measures consisted of three overall scores

of fluency, flexibility and creativity across the two versions of AUT Convergent thinking measure was a sum score of correct answers for ten RAT combinations Similarly, insight problem solving measure was a sum score of correct answers for two insight problems Game performance measures consisted of five overall scores of game fluency, flexibility, creativity, density and intersection across three rounds All variables in this study were continuous,

except for a discrete variable of insight problem solving with three groups of participants who scored zero, one or two

Five personality traits of extraversion, openness, friendliness, conscientiousness and emotional stability were computed following Van Eijck and de Graaf (2004) Additional analysis was run to examine whether openness, a trait often relates to creativity (Feist, 1999), was related to creativity measures and game performance

Results

Divergent thinking and game performance Prior to Pearson’s correlations, a

Shapiro-Wilk’s test of normality (Shapiro & Wilk, 1965) and visual inspection of histograms and normal Q-Q plots were performed The results showed that non-normal distributions

existed for AUT fluency (p = 006), game density (p = 016) and game intersection (p

< 001) These three variables were logarithmically transformed and used throughout the

analyses

Pearson’s bivariate correlations were performed to examine the relationship between three AUT scores and five game performance measures The results showed that AUT

fluency significantly correlated with game fluency (r = 315, p = 029) and game flexibility (r

= 390, p = 007) This means that people who generated more ideas in the divergent thinking

test also used more concept cards and formed more associations in the game No significant correlations were found between AUT creativity, flexibility and other game performance measures (Table 2)

Subsequently, multiple linear regressions were performed to investigate whether game fluency and flexibility can be predicted by AUT fluency Preliminary check showed that all assumptions of independence of errors, homoscedasticity of residuals, absence of multivariate

outliers and multicollinearity were met The results showed that AUT fluency positively

predicted game flexibility (b = 28.31, p = 013) (Table 3)

Additional analyses of each AUT version, the brick and the newspaper were performed

Two outliers were found and removed in brick fluency (z = 3.11) and newspaper fluency (z =

3.20) No significant correlations were found between the brick measures and game

performance However, newspaper fluency significantly correlated with game flexibility (r

= 367, p = 013) (Table 4) Regression analysis also showed that newspaper fluency was a significant predictor of game flexibility (b = 1.43, p = 013) This suggests that people who

generated more ideas to use a newspaper, but not a brick, also formed more associations in the game

Convergent thinking and game performance Pearson’s bivariate correlations were

computed between the total number of correct RAT answers and five game performance measures The results showed that RAT did not correlate with any game performance measures (Table 2), suggesting that conceptual combination practice in the game did not relate to convergent thinking skill

Insight problem solving and game performance A one-way multivariate analysis of

variance (MANOVA) was performed to examine whether game performance significantly

differ in the three groups of participants who solved zero (n = 19), one (n = 25) and two (n =

5) insight problems Preliminary check showed that the assumption of homogeneity of

variance-covariance matrixes was met, Box’s M = 32.40, F(15,4125.375) = 1.83, p =.026

The results showed no differences in all game performance measures between the three

groups, F(10,76) = 642, p = 774, Wilks’ Lambda = 850, partial η2 = 078, suggesting that game performance did not differ depending on insight problem solving

Additional analyses of each insight problem, the Candle and the Two-string problems were performed The results showed that no differences in all game performance measures existed between those who correctly answer the Candle problem and those who did not,

F(1,45) = 872, p = 508, partial η2 = 096 Similarly, there was no group difference in game

performance between the correct and incorrect groups, F(1, 45) = 550, p = 737, partial η2

= 063

Exploratory analyses for each game round The relationships between each round

performance measures and AUT, RAT and insight problem solving were explored Normality test showed that intersection scores in round 1, 2 and 3 were positively skewed and thus were

logarithmetically transformed One outlier was identified in Round 1 creativity (z = 3.19) and

removed

Pearson’s correlations showed that AUT fluency consistently related to game flexibility

across three rounds (round 1, r = 419, p = 003; round 2, r = 391, p = 007; round 3, r = 304,

p = 038) Moreover, AUT fluency significantly correlated with round 1 fluency (r = 346, p

= 015), while AUT creativity significantly correlated with round 2 creativity (r = 367, p

= 011)

With regards to convergent thinking, round 1 fluency positively correlated with RAT (r

= 300, p = 036) An unexpected result showed a negative correlation between round 1 creativity and RAT (r = -.344, p = 018) All game performance measures in rounds 2 and 3

did not correlate with RAT

Several one-way MANOVAs were performed to examine whether game performances

in each round differed depending on insight problem solving Multivariate tests showed no group differences in game performances of round 1, 2 and 3 However, a significant group

difference in round 3 creativity emerged in tests of between-subject effects, F(2,31) = 3.465,

p = 044, partial η2 = 183 Pairwise comparisons with the Bonferroni adjustment indicated

that the creative performance in round 3 of the zero insight group (M = 91, SD = 07) was significantly lower than that of the group who correctly solved one insight problem (M = 1.14,

2) However, game fluency negatively correlated with game creativity (r = -.39, p = 007),

suggesting that the more cards used to form associations, the less creative the associations were

Mean self-reported creativity of this sample was 2.51 on a scale from 0 to 4 (SD = 82), and mean openness was 3.61 on a scale from 1 to 5 (SD = 68), meaning that they perceived

themselves to be slightly above average in being creative and open to new experience These variables were negatively skewed and therefore raised to the second power to investigate their relationships with creativity measures and game performance There was a significant

positive correlation between openness and AUT creativity (r = 293, p = 041), suggesting

that those who reported to be more open had higher creative performance in the divergent thinking test No significant correlations between personality, RAT and game performance were found, however (Table 5) Group differences in self-reported creativity and openness

with regards to insight problem solving were insignificant, F(4, 90) = 528, p = 716, partial

η2 = 023 For descriptive statistics of each personality trait, see Table 6

Regarding feedback for the game, most participants (75.5%) agreed that the instructions for the game were very clear Only 14.3% indicated that the associations they formed in the

game were generated randomly, while the rest of the participants reported that they depended

on semantic and functional relatedness to form the associations Explanations for the associations made in the game included typical categories of concepts such as animals, technologies, clothes, and food; similarities in shapes, characteristics and functions; and making stories that involved different concepts (Table 7)

STUDY 2 Method

The aim of this second study was to examine whether performances in divergent thinking, convergent thinking and insight problem solving tests could be improved following

a four-week game training with “The Associates”

Participants A total of 29 undergraduate university students from USSH, Vietnam,

were recruited However, two participants withdrew from the experiment due to personal reasons Therefore, the final sample consisted of 27 participants (26 females) Their mean age

was 20.56 (SD = 1.16) About half of the participants (13) had joined Study 1 before and

continue participating in Study 2, while the rest of them joined Study 2 and played “The Associates” game for the first time Each participant was assigned with a code to ensure anonymity The majority of the participants (82.1%) were psychology students After completing the experiment, each participant was rewarded ₫350,000 (equivalent to €13.20) This study was approved by the Ethics Committee, Faculty of Social Sciences, Radboud University Nijmegen, the Netherlands, coded ECSW-2017-017R2, and by the Faculty of Psychology, USSH, Vietnam

Creativity measures All instruments used in this study was in Vietnamese language

and piloted (Appendix B)

Divergent thinking Because half of the sample in this study joined Study 1 before and

was tested with the brick and the newspaper, two new versions of the AUT, a paperclip and a plastic bottle, were used to measure divergent thinking before and after game training (Appendix C) The order of the versions were counterbalanced between the pre- and post-measure across the participants

Convergent thinking Ten three-word RAT combinations were used to measure

convergent thinking for Study 2 These stimuli was only selected for Study 2 after the pilot study (Table B5), which was not used in Study 1 It was then divided into set A and B with five combinations in each set, which were counterbalanced for pre- and post-training measures The items in the two sets were matched on difficulty levels (Table B3) Pre- and post-training convergent thinking measures were calculated as two sum scores of correct answers for RAT

Insight problems solving The two piloted insight problems for Study 2, the Reading in

dark and the Hole-in-card problems, were counterbalanced for pre- and post-training measurements These two insight problems had matching difficulty levels, which is suitable for a pre- and post-training design (Appendix B6)

“The Associates” Game At the beginning of the training, the instruction video as used

in Study 1 was shown to explain game rules The design of the game was identical to that of Study 1 However, concept cards in Study 2 were not pre-defined and fixed in their arrangements on the screen Instead, they were randomly generated from the database Moreover, each concept card was shown only once across three game rounds All outputs of the participants’ game training during four weeks were recorded, including screenshots and videos with verbal explanations during game play

Demographic data and personalities Participants who did not join Study 1 were asked

to complete the same demographic and personality questionnaires about age, gender, major subject, self-reported creativity and the short Big Five personality scale (Van Eijck, & de Graaf, 2004)

Procedure Following the first study, this second study was conducted at the Faculty of

Psychology, USSH, Vietnam Recruited participants were welcomed and assigned with a participant code During the first meeting, they were informed that they could withdraw from the study at any time After giving their consent (Appendix K), pre-training creativity of all participants were measured in the same order, starting with the AUT test, then the RAT test and finally the insight problems Additionally, the order of RAT combinations was randomized, in both pre- and post-training tests After the pre-training measures, the participants watched the game instruction video Then, they started the first game training session, playing three rounds of “The Associates” game in 30 minutes Demographic and personality questionnaires were completed afterwards The participants were then scheduled

to play the game three times a week in four weeks These meetings had at least one day interval On the last meeting in the fourth week, after playing the game for 30 minutes, the participants repeated the testing procedure with AUT, RAT and insight problems for post-training measures

Plan of data analysis Data was analyzed using the Statistical Package for Social

Scientists (SPSS, version 22) Divergent thinking measures included scores of fluency, flexibility and creativity for each version of the AUT test, the paperclip and the bottle Pre- and post-training convergent thinking measures were the sum scores of correct answers To

test the hypothesis of significant improvements in creativity tests following the game training,

a series of two-way mixed design analyses of variance (ANOVAs) were used to examine the

main effect of game training on creativity measures, while also tested whether the training effect differed between those who had experience with testing and “The Associates” game before and those who joined this study for the first time (participation) Overall game performance in the first and last training sessions were also computed for comparisons Exploratory analyses were performed to examine the relationship between creativity measures and game performance, as well as openness and self-reported creativity

Results

The effect of game training on creativity measures Preliminary analyses showed

that the assumptions of outliers, homogeneity of variances and normality were met A series

of 2 (pre- vs post-training) x 2 (first participation vs second participation) mixed design ANOVAs were performed The results suggested that the main effect of training on AUT

fluency remained insignificant, F(1,25) = 3.28, p = 082, partial η2 = 116, while a significant

interaction between AUT fluency and participation was found F(1, 25) = 5.97, p = 022,

partial η2 = 193 Specifically, those who had no experience improved their fluency in the AUT test after the training more than those who had joined both studies (Figure 7)

Figure 7 Significant interaction effect between training and participation on AUT fluency

6 6.5 7 7.5 8 8.5 9 9.5 10

AUT fluency pre AUT fluency post

Similarly, the training effect was insignificant for AUT flexibility, F(1,25) = 45, p

= 510, partial η2 = 018, although a significant interaction was found F(1,25) = 4.97, p = 035,

partial η2 = 166 Specifically, those who joined the second study for the first time improved their flexibility in the AUT test after the training more than those who had joined both studies (Figure 8)

Figure 8 Significant interaction effect between training and participation on AUT flexibility

In contrast, the main effect of training on AUT creativity was significant F(1,25) = 6.67,

p = 016, partial η2 = 211, with post-training AUT creativity (M = 3.01, SD = 28) was higher

than pre-training (M = 2.87, SD = 32) (Figure 9) The interaction between AUT creativity and participation was not significant, F(1, 25) = 1.05, p = 315, partial η2 = 040, meaning that the creative performance in the AUT was improved after the training, regardless of the participants’ experience with the AUT and “The Associates” game prior to the training

With regards to convergent thinking, the results showed a significant main effect of

game training on RAT performance, F(1,25) = 4.23, p = 048, partial η2 = 147, with

post-training RAT score (M = 3.19, SD = 1.04) was significantly higher than pre-post-training RAT score (M = 2.67, SD = 1.18) (Figure 9) No significant interaction was found between training and participation, F(1,25) = 138, p = 713, partial η2 = 006

3.2 3.4 3.6 3.8 4 4.2 4.4 4.6 4.8

AUT flexibility pre AUT flexibility post

Figure 9 Mean scores of AUT creativity and RAT before and after game training *p < 05

Finally, the main effect of game training on insight problem solving was not significant,

F(1,25) = 174, p = 680, partial η2 = 007, suggesting that insight problem solving did not

improve after the training The interaction effect between training and participation was also not significant for insight problem solving All statistics are summarized in Table 8

In addition, several 2 x 2 mixed ANOVAs were also performed to examine whether game performance in the last training session significantly differed from the first training session, and the interaction between game training and participation The results showed a

significant improvement in game fluency, F(1,25) = 7.405, p = 012, partial η2 = 229, with

fluency score in the last session (M = 33.88, SD = 8.45) was significantly higher than in the first session (M = 29.09, SD = 8.99) In contrast, a significant reduction in game intersection was found F(1,25) = 12.84, p = 001, partial η2 = 339, with intersection score in the last

session (M = 4.09, SD = 4.35) was significantly lower than in the first session (M = 11.65, SD

= 11.97) (Figure 10) No interaction effect was found on game fluency and game intersection The main effect of training on game flexibility, game creativity and game density was not significant The interaction effect on these measures was also not significant

0 0.5 1 1.5 2 2.5 3 3.5 4

* *

Figure 10 Mean scores of game fluency and game intersection in the first and last training

sessions *p < 05, *p < 01

The relationships between creativity measures, game performance and personality

Correlations between creativity measures, game performance and personality were examined once again In the first training session, Pearson’s correlations showed that game creativity

significantly correlated with pre-training AUT fluency (r = 425, p = 027), AUT creativity (r

= 548, p = 003) and openness (r = 404, p = 037) (Table 9) In the last training session, game creativity marginally correlated with post-training AUT fluency (r = 366, p = 061) Self-reported creativity marginally and negatively correlated with game fluency (r = -.377, p

= 052) and game density (r = -.375, p = 054) (Table 10) No correlations were found

between pre-training RAT, insight problem solving and game performance in the first training session Similarly, no correlations were found between post-training RAT, insight problem solving and game performance in the last training session

Sample characteristics This sample reported a mean score of 2.48 for their

self-reported creativity (SD = 75) and 3.63 for openness, meaning that they perceived themselves

to be slightly above average in being creative For descriptive statistics of each personality trait, see Table 11

0 5 10 15 20 25 30 35 40

* **

GENERAL DISCUSSION

Creativity is one the most desirable learning and employability skills (Trilling & Fadel, 2009) Given its importance, a number of training programs have been proposed to enhance creative thinking (Scott et al., 2004a) However, training programs involving instructors and formal in-person lectures are often costly and time-consuming (Baruah & Paulus, 2008) We propose that a gamification approach to creativity training can be an alternative In two studies, we firstly developed a computerized game training based on the cognitive theory of conceptual combination Subsequently, we examined the effect of this game training on several cognitive processes of creativity, including divergent thinking, convergent thinking and insight problem solving

The relationship between game performance and creativity measures

The results of this research support our hypothesis that divergent thinking, the ability to generate multiple ideas and solutions, is central and necessary for the performance in the computerized game Specifically, higher number of ideas (fluency) and creative level of those ideas (creativity) in the AUT test of divergent thinking are consistently associated with higher creative performance and flexibility in changing perspectives to combine concepts in the game (game creativity and flexibility) In contrast, inconsistent relationships were found between game performance and convergent thinking, the ability to select a single best idea, and insight problem solving, an interplay of divergent thinking and convergent thinking skills While performances in the RAT test of convergent thinking and insight problem solving did not show significant correlations with overall measures of game performance, additional analyses revealed that they had some associations with performance in one of the game rounds in the first study

Our research adds to the study of conceptual combination and creativity by developing

a computerized game that provides exercises for conceptual combination It has been suggested that new ideas can be generated for existing concepts using conceptual combination, a mental synthesizing process in which previously unrelated concepts are combined (Ward et al., 1999) As conceptual combination provides new perspectives to a problem and leads to the production of novel ideas, it is suggested that manipulations intended to encourage this practice can contribute to creative performance (Meador, 1994; Mumford, Baughman, Maher, Costanza, & Supinski, 1997; Phye, 1997) The exercise of conceptual combination in our game share some similarities with the paradigm of category-exemplar generation in the work of Mumford and colleagues (Mobley, Doares, & Mumford, 1992; Mumford et al., 1997) In these studies, participants were presented with a list of typical exemplars (e.g chair) from several taxonomic categories (e.g furniture) Labels of the categories were not shown to avoid verbal priming The participants were requested to generate a category label that can account for all presented exemplars before generating more exemplars for this new category In our game, several concepts from different categories were presented and the participants were asked to combine them in a meaningful, creative ways, hinting that they should avoid using the original categories of concepts to form the associations

Our finding of the positive correlation between game performance and performance in the AUT test supports the positive relationship between conceptual combination training and divergent thinking skill In line with Wan and Chiu (2002), our results show that those who have higher divergent thinking skill tend to be more successful, i.e making more creative associations, in the conceptual combination practice, although it can mean vice versa that those who are good at synthesizing existing concepts to create new ideas also have higher divergent thinking skill In contrast, the results indicate that convergent thinking and insight

problem solving are not consistently involved in the game performance We conjecture that, because conceptual combination is one of the early-cycle processing activities of creativity (together with problem finding, information gathering, and concept selection) while idea selection involving convergent thinking is often a late-cycle process (Scott et al., 2004a), convergent thinking skill was not necessarily required in the game that mostly relies on associating concepts without strict requirements to select the best ideas Alternatively, it could also mean that our game performance measures did not explicitly capture this ability

In case of insight problem solving, it is a complex, higher process cognition that can be difficult to understand scientifically (Batchelder & Alexander, 2012) The collection of classic insight problems, such that was used in our studies, is very heterogeneous in which each insight problem has its own character to provoke insights (Chu & MacGregor, 2011) Overall, the relationship between insight problem solving and game performance across three rounds was not found, although a significant finding emerged regarding the creative performance in one of the rounds in the game We conjecture that the low number of insight problems used in our studies might have limited our observation of the participants’ insight problem solving skill

The effect of game training on creativity

The results in our second study support the hypothesis that significant improvements in divergent thinking and convergent thinking performance could be observed following a four-week period of computerized game-based training Moreover, it is showed that a gamification approach to training creativity can be developed to facilitate some aspects of creative thinking Specifically, the creative performance in the AUT test and the number of correctly solved RAT items significantly increased post-training The results of this study are in line

with other research suggesting that divergent and convergent thinking can be enhanced by an intervention (for example, Im et al., 2015, Ritter & Mostert, 2017)

However, the significant interaction between game training and participation is unexpected with more profound training effect on AUT fluency and flexibility was found for those who did not experience our game and the creativity tests before We cannot reason why this is the case, although we want to note that the domains of concepts presented in the game can influence the flexible performance of the participants in the game In our game database, there is a mixture of man-made and natural concepts As found in the study of Mobley et al (1992), people are more flexible in manipulating and switching approaches when they deal with objects produced by human instead of natural, biological objects It might be because the group who had joined both of our studies were especially more interested in the biological objects than the group who participated for the first time, which subsequently affected their practice and resulted in less flexible performance in the AUT test However, given the significant main effect of training and its non-significant interaction with participation on the AUT creativity measure, it indicates that the practice of conceptual combination in the game exerts a consistent cross-domain effect on the participants’ creative performance overall

With regards to convergent thinking, the results of our second study show that it is improved after game training, while the results of our first study indicate a contradictory, inconsistent relationship between game performance and RAT scores One possible explanation for this might be that we used a Vietnamese RAT version that has not been validated and standardized, which might interfere with the true, consistent testing of convergent thinking However, in the second study, the RAT combinations used in the pre- and post-training measurements were matched on difficulty levels, which we think is appropriate and sufficient to retain the observed significant improvement Although

performance in the RAT test requires two capacities, exploiting associative components to produce new useful combinations and evaluating the best idea (Mednick, 1962), it is possible that the participants became more experienced in applying their associative skill to aid their convergent thinking process after the training, even when the ability to evaluate and derive the best idea was not trained in the game On the other hand, we did not find significant improvements in insight problem solving following the game training Again, we conjecture that using a low number of insight problem (i.e one) to measure insight problem solving in each measurement time points was not sufficient to yield notable results

Importantly, compared to the first training session, we did not find significant improvements in the creative and flexible game performance of the last session We suspect that because the number of concept cards was limited (435 concept cards in our database) while the training was repeated with twelve sessions and the participants played 36 rounds in total, each concept card was presented repeatedly throughout the training It is possible that the enjoyment and motivation towards the last game session were declined, although they still gained the benefits of the training to aid to their subsequent creativity testing

The relationship between game performance and personality

Exploratory analyses showed that, although openness to experience often relates to creativity (Feist, 1999), it does not correlate with the creative performance in the game This means that how creative and open-minded a person is does not relates to how creatively they will play the game and implies that individuals with either high or low creative divergent thinking can be encouraged to practice forming creative associations in the game

Feedback for the game, limitations and suggestions for future research

By collecting game feedback, it was found that most participants still depended on semantic relatedness to form associations This suggests that more explicit, stricter game

rules can be made to encourage breaking this pattern Moreover, analyses of game behavior help us to understand how participants perform in the game In contrast to the presumption of Osborn (1957) that more ideas generated will increase the average creativity of those ideas, our results showed that this is not the case within the game Specifically, the more concept cards used to form associations, the less creative those associations are This can be explained that, with a set of both related and unrelated concepts presented in the game (concepts from the same category versus concepts from different categories), a tendency to associate related concepts by using semantic relatedness will result in more cards played but less creative explanations for their associations Our finding of a negative relationship between game fluency and game creativity supports other studies (Baruah & Paulus, 2008; Rietzschel, Nijstad, & Stroebe, 2007), suggesting that fluency and creativity may not necessarily be complementary to each other This implies that clear and explicit instructions should be made

to encourage players focusing on the quality (creativity) instead of the quantity (fluency) of associations in the game

There are some other points that should be addressed in future research Firstly, because the second study employed a within-subjects design without a control group, it is uncertain whether the improvements in divergent thinking and convergent thinking skills were solely due to the game training It is probable that the participants naturally developed these skills through their courses and experiences in college However, we think that the results are unlikely due to a learning effect as we piloted and used different versions for all creativity tests with matched difficulty levels in pre- and post-training measurements, in line with other studies (for example, Ritter & Mostert, 2017) Nonetheless, a between-subjects design with a control group can be employed to interpret the effectiveness of the game training with more certainty To examine the long-term effect of this game training, a longitudinal study can also

be conducted and measure a third time point, which is a delayed post-training score, to

examine how long the game training effect holds

Secondly, the samples in both studies included mostly female psychology students and

thus may not be representative of other genders and populations Moreover, a gender effect

exists as creativity training often has greater effects on men (Scott et al., 2004a) It is also not

clear whether the game can be effective in creativity training for other cultural groups Future

research should take these into account to increase ecological validity, generalization and

provide more insights into the effectiveness of this computerized game-based training

Furthermore, future developments of the game should extend our concept card database

and consider adding more game elements to retain enjoyment and benefits to players with

scientific evidence A development of an automatic scoring system will also be beneficial for

the analysis of game performance

Strengths and contributions

By developing, validating and investigating the effectiveness of a computerized game

training for creativity, our research offers some values Firstly, we advocate a gamification

approach to design a less costly, time-consuming training program and demonstrate its

potential to be an effective alternative training Secondly, the computerized game

incorporates the practice of conceptual combination with cross-domain concepts, which is

suggested to lay a foundation for effective interventions of creativity (Mobley et al., 1992)

Specifically, the game relies on the imagery and visualization of concepts, instead of

terminology, to be independent on language and benefits the mental synthesis of concepts

(Anderson & Helstrup, 1993; Finke, 1990; Ward, 1994) Taken together, our research offers

some evidence for a potential gamification creativity training, which focuses on developing

conceptual combination practice, an important cognitive skill for creative thought

References

Anderson, R E., & Helstrup, T (1993) Visual discovery in mind and on paper Memory and

Psychophysics, 21, 283–293

Azriel, J A., Erthal, M J., & Starr, E (2005) Answers, questions, and deceptions: what is the

role of games in business education? Journal of Education for Business, 81 (1), 9-14

Baruah, J., & Paulus, P B (2008) Effects of training on idea generation in groups Small

Group Research, 39, 523–541

Batchelder, W.H & Alexander, G.E (2012) Insight problem solving: a critical examination

of the possibility of formal theory Journal of Problem Solving, 5 (1), 56-100

Baughman, W A., & Mumford, M D (1995) Process-analytic models of creative capacities:

operations influencing the combination-and-reorganization process Creativity Research

Thinking Skills and Creativity, 7, 177-186

Chu, Y., & MacGregor, J (2011) Human performance on insight problem solving: a

review The Journal of Problem Solving, 3 (2), 119-150

deBono, E (1969) The mechanism of mind New York: Penguin

Dominowski, R (1994) Insight and instructions Paper presented at 11th Annual Conference,

Cognitive Psychology, British Psychological Society New Hall, Cambridge

Duncker, K (1945; original in German 1935) On problem solving Psychological

Monographs, 58 (270), i113

Feist, G J (1999) The influence of personality on artistic and scientific creativity In R J

Sternberg (Ed.), Handbook of creativity (pp 273–296) Cambridge, England:

Cambridge University Press

Feldhusen, J F., Trepfinger, D J., & Bahlke, S (1970) Developing creative thinking: the

Purdue Creativity Program Journal of Creative Behavior, 4, 85-90

Finke, R A (1990) Creative imagery: discoveries and inventions in visualization Hillsdale,

NJ: Erlbaum

Finke, R.A (1996) Imagery, creativity, and emergent structure Consciousness and

Cognition, 5, 381–393

Finke, R.A., Ward, T.B., & Smith, S.M (1992) Creative cognition: theory, research, and

applications Cambridge, MA: MIT Press

Fixx, F F (1972) More games for the superintelligent New York: Popular Library

Gentner, D (1983) Structure-mapping: a theoretical framework for analogy Cognitive

Science, 7 (3), 155-170

Gerris, J R M., Houtmans, M J M., Kwaaitaal-Roosen, E M G., Schipper, J C., Vermulst,

A A., & Janssense, J M A M (1998) Parents, adolescents and young adults in Dutch

families: A longitudinal study Nijmegen: Institute of Family Studies, University of

Nijmegen

Gilhooly, K.J., & Murphy, P (2005) Differentiating insight from non-insight

problems Thinking & Reasoning, 11 (3), 279-302