lib case studies in knowledge management jennex

vii Section I: Knowledge Management in Support of Organizational Learning Chapter I.. England, Air Force Institute of Technology, USA Section IV: Knowledge Management in Support of Proje

IDEA GROUP PUBLISHING

Case Studies

in Knowledge

Management

Murray E Jennex

San Diego State University, USA

Acquisitions Editor: Renée Davies

Development Editor: Kristin Roth

Senior Managing Editor: Amanda Appicello

Managing Editor: Jennifer Neidig

Copy Editor: Joyce Li

Typesetter: Jennifer Neidig

Cover Design: Lisa Tosheff

Printed at: Integrated Book Technology

Published in the United States of America by

Idea Group Publishing (an imprint of Idea Group Inc.)

701 E Chocolate Avenue, Suite 200

Hershey PA 17033

Tel: 717-533-8845

Fax: 717-533-8661

E-mail: cust@idea-group.com

Web site: http://www.idea-group.com

and in the United Kingdom by

Idea Group Publishing (an imprint of Idea Group Inc.)

Web site: http://www.eurospan.co.uk

Copyright © 2005 by Idea Group Inc All rights reserved No part of this book may be duced, stored or distributed in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher.

repro-Product or company names used in this book are for identification purposes only Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI of the trademark or registered trademark.

Library of Congress Cataloging-in-Publication Data

Case studies in knowledge management / Murray Jennex, editor.

p cm.

Includes bibliographical references and index.

ISBN 1-59140-351-0 (hardcover) ISBN 1-59140-352-9 (softcover) ISBN 1-59140-353-7 (ebook)

1 Knowledge management Case studies I Jennex, Murray,

HD30.2.C378 2005

658.4'038 dc22

2005004515

British Cataloguing in Publication Data

A Cataloguing in Publication record for this book is available from the British Library.

All work contributed to this book is new, previously-unpublished material The views expressed in this book are those of the authors, but not necessarily of the publisher.

Knowledge Management Table of Contents

Preface vii

Section I:

Knowledge Management in Support of Organizational Learning

Chapter I.

Learning from Simple Systems: The Case of JPL 101 1

Lynne P Cooper, Jet Propulsion Laboratory, USA

Rebecca L Nash, Jet Propulsion Laboratory, USA

Tu-Anh T Phan, Jet Propulsion Laboratory, USA

Teresa R Bailey, Jet Propulsion Laboratory, USA

Hani Abdel-Aziz, Cairo University, Egypt

Khaled Wahba, Cairo University, Egypt

Afsoun Hatami, London School of Economics, UK

Robert D Galliers, Bentley College, USA

Chapter VI.

Governance of Strategies to Manage Organizational Knowledge: A Mechanism to Oversee Knowledge Needs 83

Suzanne Zyngier, Monash University, Australia

Frada Burstein, Monash University, Australia

Judy McKay, Swinburne University of Technology, Australia

Chapter VII.

Challenges in Developing a Knowledge Management Strategy for the Air Force Material Command 104

Summer E Bartczak, Air Force Institute of Technology, USA

Ellen C England, Air Force Institute of Technology, USA

Section IV:

Knowledge Management in Support of Projects Chapter VIII.

Knowledge Management in a Project Climate 130

Elayne Coakes, University of Westminster, UK

Anton Bradburn, University of Westminster, UK

Cathy Blake, Taylor Woodrow, UK

Chapter IX.

Where Knowledge Management Resides within Project Management 138

Jill Owen, Monash University, Australia

Frada Burstein, Monash University, Australia

Section V:

Knowledge Management is Support of Knowledge Transfer

Chapter X.

Organizational Knowledge Sharing Based on the ERP Implementation of

Yongxin Paper Co., Ltd 155

Zhang Li, Harbin Institute of Technology, China

Tian Yezhuang, Harbin Institute of Technology, China

Li Ping, Harbin Institute of Technology, China

Methods 165

Thomas Hahn, Profactor Produktionsforschungs GmbH, Austria

Bernhard Schmiedinger, Profactor Produktionsforschungs GmbH, Austria Elisabeth Stephan, Profactor Produktionsforschungs GmbH, Austria

Chapter XII.

Know-CoM: Decentralized Knowledge Management Systems for Cooperating

Die- and Mold-Making SMEs 186

Florian Bayer, Martin-Luther-University Halle-Wittenberg, Germany

Rafael Enparantza, Centro Technológico Tekniker, Spain

Ronald Maier, Martin-Luther-University Halle-Wittenberg, Germany

Franz Obermair, Profactor Produktionsforschungs GmbH, Austria

Bernhard Schmiedinger, Profactor Produktionsforschungs GmbH, Austria

Section VI:

Issues in Knowledge Management Chapter XIII.

Reserve Bank of New Zealand: Journey Toward Knowledge Management 211

Yogesh Anand, Reserve Bank of New Zealand, New Zealand

David J Pauleen, Victoria University of Wellington, New Zealand

Sally Dexter, Victoria University of Wellington, New Zealand

Chapter XIV.

A Comparative Case Study of Knowledge Resource Utilization to Model

Organizational Learning 235

Colin White, Deloitte Consulting, USA

David Croasdell, University of Nevada, Reno, USA

Chapter XV.

Implementing Knowledge-Enabled CRM Strategy in a Large Company:

A Case Study from a Developing Country 249

Minwir Al-Shammari, University of Bahrain, Bahrain

Chapter XVI.

Why Knowledge Management Fails: Lessons from a Case Study 279

Ivy Chan, The Chinese University of Hong Kong, Hong Kong

Patrick Y.K Chau, The University of Hong Kong, Hong Kong

Chapter XVII.

Infosys Technologies, Limited 289

Nikhil Mehta, Auburn University, USA

Anju Mehta, Auburn University, USA

Chapter XVIII.

Keeping the Flame Alive: Sustaining a Successful Knowledge Management

Program 315

Eliot Rich, University of Albany, USA

Peter Duchessi, University of Albany, USA

A.N Dwivedi, Coventry University, UK

Rajeev K Bali, Coventry University, UK

R.N.G Naguib, Coventry University, UK

Chapter XX.

Productivity Impacts from Using Knowledge 344

Murray E Jennex, San Diego State University, USA

About the Authors 358 Index 369

Knowledge Management (KM) has been growing in importance and popularity

as a research topic since the mid 1990s This is sufficient time for many organizations

to implement KM initiatives and KM systems (KMS) This book presents twenty casesinvestigating the implementation of KM in a number of business and industry settingsand a variety of global settings The purpose of this book is to fill a deficiency that I’veobserved while teaching KM KM is being taught in specialized courses and as a topicincluded in Decision Support Systems (DSS), Enterprise Information Systems (EIS),and Management Information Systems (MIS) issues courses The deficiency I’ve ob-served is in moving discussions of KM from a focus on theory to the more practicalfocus of how to implement KM to help organizations improve their performance Exist-ing course materials do include some short cases and/or vignettes discussing KM inbusiness settings, but I haven’t found any source that has multiple, detailed teachingcases This book is meant to fill that void

The cases contained in this book are presented as teaching cases All havediscussion questions and are written in a style that students can easily read and under-stand Also, additional sources and support materials are included where appropriate.The book includes cases from many different countries in an attempt to appeal to aswide an audience as possible Cases are included from Australia, Austria, Bahrain,China, Egypt, Germany, Great Britain, Hong Kong, India, New Zealand, and the UnitedStates Additionally, a variety of business situations are presented including banking,consulting, engineering, government agencies, manufacturing, military, project man-agement, software development, and public utilities Also, several different relatedprocesses and technologies are discussed Related processes include organizationallearning (OL) and organizational memory (OM) Technologies include Customer Rela-tionship Management (CRM), Enterprise Resource Planning (ERP), Data Warehousing,networking, and Intranets Finally, several issues are addressed including knowledgecapture, knowledge sharing, knowledge transfer, knowledge representation, organiza-tional culture, management support, KM/KMS success, KM sustainability, retainingworker knowledge, creating learning organizations, and management support

WHAT IS KM?

There are many definitions of KM but this book combines the KM and OM ture to define KM as the process of selectively applying knowledge from previousexperiences of decision-making to current and future decision making activities withthe express purpose of improving the organization’s effectiveness This definitionallows us to define the goals of KM as:

litera-• Identify Critical Knowledge

• Acquire Critical Knowledge in a Knowledge Base or Organizational Memory

• Share the stored Knowledge

• Apply the Knowledge to appropriate situations

• Determine the effectiveness of using the applied knowledge

• Adjust Knowledge use to improve effectiveness

WHY OM AND OL?

Why is OM, and OL included in a book on knowledge management? Jennex andOlfman (2002) found that the three areas are related and have an impact on organiza-tional effectiveness KM and OM are observed to be manifestations of the same pro-cess in different organizations User organizations ‘do’ knowledge management; theyidentify key knowledge artifacts for retention and establish processes for capturing it

OM is what IT support organizations ‘do’; they provide the infrastructure and supportfor storing, searching, and retrieving knowledge artifacts OL results when users utilizecaptured knowledge That OL may not always have a positive effect is examined by themonitoring of organizational effectiveness Effectiveness can improve, get worse, or

viii

Figure 1 The KM/OM/OL Model (Jennex & Olfman, 2002)

Learning

OMKM

Drives Users to put Information and Knowledge into their OMS

Monitor Organizational Effectiveness

and AdjustKnowledge Requirements

as needed

Identify and Acquire

Store, Retrieve, and Search

Evaluate Events for Use of Applicable Memory to perform actions that affect Organizational Performance

System Designers/IT Knowledge

Engineers

remain the same How effectiveness changes influences the feedback provided to theorganization using the knowledge.

WHAT IS A KMS?

The cases in this book address the implementation of Knowledge ManagementSystems (KMS) However, KMS is a term that does not have a consensus definition.Yes, we know what the initials KMS stand for and we have an understanding of what asystem is The IPO model: Inputs, Processes, Outputs, defines a basic system thatwhen we add feedback, is a fair description of a KMS in a learning organization We getfurther insight into what an information system is from Alter (1999) who defines aninformation system as humans or machines limited to processing information by per-forming six types of operations: capturing, transmitting, storing, retrieving, manipulat-ing, and displaying This is further refined by Churchman (1979, p 29) who defines asystem as “a set of parts coordinated to accomplish a set of goals;” and that there arefive basic considerations for determining the meaning of a system:

• system objectives, including performance measures

Alavi and Leidner (2001, p 114) defined a KMS as “IT-based systems developed

to support and enhance the organizational processes of knowledge creation, storage/retrieval, transfer, and application.” They observed that not all KM initiatives willimplement an IT solution, but they support IT as an enabler of KM Maier (2002)expanded on the IT concept for the KMS by calling it an ICT (Information and Commu-nication Technology) system that supported the functions of knowledge creation, con-struction, identification, capturing, acquisition, selection, valuation, organization, link-ing, structuring, formalization, visualization, distribution, retention, maintenance, re-finement, evolution, accessing, search, and application Stein and Zwass (1995) define

an Organizational Memory Information System (OMIS) as the processes and IT nents necessary to capture, store, and apply knowledge created in the past on deci-sions currently being made Jennex and Olfman (2004) expanded this definition byincorporating the OMIS into the KMS and adding strategy and service components tothe KMS

compo-Additionally, we have different ways of classifying the KMS and/or KMS nologies where KMS technologies are the specific IT/ICT tools being implemented inthe KMS Alavi and Leidner (2001) classify the KMS/KMS tools based on the Knowl-edge Life Cycle stage being predominantly supported This model has 4 stages, knowl-

tech-edge creation, knowltech-edge storage/retrieval, knowltech-edge transfer, and knowltech-edge cation and it is expected that the KMS will use technologies specific to supporting thestage for which the KMS was created to support Marwick (2001) classifies the KMS/KMS tools by the mode of Nonaka’s (1994) SECI model (Socialization, Externalization,Combination, and Internalization) being implemented Borghoff and Pareschi (1998)classify the KMS/KMS tools using their Knowledge Management Architecture Thisarchitecture has 4 classes of components: repositories and libraries, knowledge workercommunities, knowledge cartography/mapping, and knowledge flows; with classifica-tion being based on the predominant architecture component being supported Hahnand Subramani (2001) classify the KMS/KMS tools by the source of the knowledgebeing supported: structured artifact, structured individual, unstructured artifact, orunstructured individual Binney (2001) classifies the KMS/KMS tools using the Knowl-edge Spectrum The Knowledge Spectrum represents the ranges of purposes a KMScan have and include: transactional KM, analytical KM, asset management KM, pro-cess-based KM, developmental KM, and innovation and creation KM Binney (2001)does not limit a KMS/KMS tool to a single portion of the Knowledge Spectrum andallows for multi-purpose KMS/KMS tools Zack (1999) classifies KMS/KMS tools aseither Integrative or Interactive Integrative KMS/KMS tools support the transfer ofexplicit knowledge using some form of repository and support Interactive KMS/KMStools support the transfer of tacit knowledge by facilitating communication betweenthe knowledge source and the knowledge user Jennex and Olfman (2004) classify theKMS/KMS tools by the type of users being supported Users are separated into twogroups based on the amount of common context of understanding they have with eachother resulting in classifications of: process/task based KMS/KMS tools or generic/infrastructure KMS/KMS tools.

appli-While I tend to favor a more holistic/Churchmanian view of systems and the KMSand like to classify the KMS by the amount of context needed by the users to effec-tively use knowledge, others are equally happy with these other KMS definitions andclassification schemes It is not the point of this book to settle the debate; in fact, many

of the enclosed cases use definitions different than the holistic KM is a young pline and it will have multiple definitions of key terms for a while as we go throughgrowing pains in establishing our definitions That is okay, but for us to mature weneed to settle on some of our fundamental definitions Defining a KMS is one of thosefundamental definitions we need to agree on This is needed for our practitioners, and

disci-to some degree, our researchers Practitioners need disci-to speak a common language disci-toeach other and to their clients The KMS is one of those concepts that clients expect us

to understand It is hoped that the cases in this book, when taken as a whole, providesupport for the holistic definition as the KMS discussed are varied in their componentsand purpose

x

employees learn about the organizational culture The second case is from BrigetteMcGregor-MacDonald and describes the KMS used in Marsh, Inc to help employeeslearn and pass on their knowledge to other employees Both cases look at key issuesand discuss the importance of management support in sustaining the KM effort.Section 2 explores using KM to support the retention of organizational knowl-edge in organizations where the work forces are in transition Hani Abdel-Aziz, andKhaled Wahba discuss the use of OM to capture knowledge in an Egyptian Profes-sional Services company that had a high rate of employee turnover Gail Corbitt dis-cusses the issues affecting knowledge loss and the creation of two financial divisionswhen HP split into HP and Agilent These papers find that the processes used tocapture knowledge are critical Additionally, issues such as corporate culture, techni-cal infrastructure, and training are discussed

Section 3 discusses the importance of a KM strategy in the implementation of a

KM initiative Afsoun Hatami and Robert D Galliers look at the long term impacts ofstrategy on the success of an OM system used to support decision making SuzanneZyngier, Frada Burstein, and Judy McKay discuss the use of corporate governance as

a method of implementing KM strategy in Australia’s Science and Technology opment Organization Summer E Bartczak and Ellen C England discuss the issuesinvolved in developing a KM strategy for the United States’ Air Force MaterialCommand’s KM initiative These cases also explore the impact of leadership and theuse of a strategic framework in the development of a KM strategy

Devel-Section 4 discusses the use of KM in the support of projects and project ment Elayne Coakes, Anton Bradburn, and Cathy Blake, discuss the use of KM tocapture and use best practices in the British construction firm Taylor Woodrow toimprove project performance Jill Owen and Frada Burstein look at where knowledgeresides in an Australian consulting firm and how the firm uses this knowledge to im-prove project performance Both cases discuss the importance of understanding knowl-edge transfer dynamics to improve the flow of knowledge within a project team.Section 5 discusses KM in support of knowledge transfer Zhang Li, TianYezhuang, and Li Ping, discuss the dynamics of using a Enterprise Resource Planningsystem to capture and transfer knowledge in a Chinese manufacturing firm ThomasHahn, Bernhard Schmiedinger, and Elisabeth Stephan look at the use of communities ofpractice and other techniques to improve the transfer of knowledge in and betweenAustrian small and medium sized manufacturing firms Florian Bayer, Rafael Enparantza,Ronald Maier, Franz Obermair, and Bernhard Schmiedinger discuss the use of KnowCom to facilitate the decentralized control of the flow of knowledge between small andmedium sized German die and mould makers

manage-Section 6 discusses a variety of issues associated with the implementation of KMand a KMS Yogesh Anand, David J Pauleen, and Sally Dexter discuss the develop-ment and sustainability of the KM initiative in the New Zealand Reserve Bank ColinWhite and David Croasdell discuss issues in representing knowledge in EnterpriseResource Planning Systems at Nestle USA, Colgate-Palmolive, Xerox, and Chevron-Texaco Minwir Al-Shammari discusses issues in using a Data Warehouse and a Cus-tomer Relationship Management system to capture and transfer knowledge in a MiddleEastern telecommunications company Ivy Chan and Patrick Y.K Chau explore why a

KM initiative failed in a Hong Kong manufacturing and export firm Nikhil Mehta andAnju Mehta discuss issues faced by India’s Infosys Technologies, Limited Eliot Rich

and Peter Duchessi discuss the issues involved in sustaining the KM initiative at theUnited States’ System Management Solutions International.

Section 7 discusses how to determine KM outcomes A.N Dwivedi, Rajeev K.Bali, and R.N.G Naguib discuss a general KM framework for the British healthcareindustry and how to manage KM successfully Murray E Jennex discusses how theuse of knowledge can impact individual and organizational productivity

REFERENCES

Alavi, M.& Leidner, D.E (2001) Review: Knowledge management and knowledge agement systems: Conceptual foundations and research issues MIS Quarterly,

man-25(1), 107-136.

Alter, S (1999) A general, yet useful theory of information systems Communications

of the Association for Information Systems, 1(13).

Binney, D (2001) The knowledge management spectrum: Understanding the KM scape The Journal of Knowledge Management, 5(1), 33-42

land-Borghoff, U.M & Pareschi, R (1998) Information technology for knowledge

manage-ment Berlin: Springer-Verlag.

Churchman, C W (1979) The systems approach (revised and updated) New York: DellPublishing

Hahn, J & Subramani, M.R (2000) A framework of knowledge management systems:Issues and challenges for theory and practice Proceedings of the Twenty-first

International Conference on Information Systems, Association for Information Systems, (pp 302-312).

Jennex, M.E., Croasdell, D., Olfman, L & Morrison, J (2005) Knowledge management,organizational memory, and organizational learning at the Hawaii InternationalConference on System Sciences International Journal of Knowledge Manage-

ment, 1(1), 1-7.

Jennex, M E & Olfman, L (2004) Modeling knowledge management success

Confer-ence on Information SciConfer-ence and Technology Management, CISTM.

Maier, R (2002) Knowledge management systems: Information and communication

technologies for knowledge management Berlin: Springer-Verlag.

Marwick, A.D (2001) Knowledge management technology IBM Systems Journal,

Section I

Knowledge Management

in Support of

Organizational Learning

Chapter I

Learning from Simple Systems:

it was deployed within the organization This case demonstrates that a relatively

“simple” system can effectively support learning or organizational knowledge, while still presenting a variety of challenges during the implementation process.

BACKGROUND

The Jet Propulsion Laboratory (JPL) is a federally funded research and developmentcenter (FFRDC) operated for the National Aeronautics and Space Administration(NASA) by the California Institute of Technology (Caltech) JPL’s history dates to the

2 Cooper, Nash, Phan, and Bailey

1930s and Caltech’s pioneering work in rocket propulsion After two decades of support

to the Army, JPL was transferred to NASA in December 1958 JPL brought with itexperience in building and flying spacecraft, an extensive background in solid and liquidrocket propulsion systems, guidance, control, systems integration, broad testing capa-bility, and expertise in telecommunications using low-power spacecraft transmitters andvery sensitive Earth-based antennas and receivers

Following the success of Sputnik, JPL developed the first U.S satellite, Explorer 1

In the 1960s, JPL began to conceive and execute robotic spacecraft to explore otherworlds Ranger and Surveyor missions were launched to the moon, and Mariner missionsvisited Mercury, Venus, and Mars JPL has since achieved stunning successes with anarmada of missions such as Voyager, Galileo, Magellan, Deep Space 1, and MarsPathfinder It also had to deal with highly publicized failures such as the Mars ClimateOrbiter and Mars Polar Lander missions JPL is currently operating several missions (e.g.,Cassini mission to Saturn, the Stardust comet sample return, Spitzer space observatory,and the twin Mars Exploration Rovers, Spirit and Opportunity), with many new missions

in various stages of development

As a major national research and development (R&D) laboratory, JPL’s mission is

1 to explore our own and neighboring planetary systems;

2 to search for life outside the Earth’s confine;

3 to further our understanding of the origins and evolution of the universe and thelaws that govern it;

4 to make critical measurements to understand our home planet and help protect itsenvironment;

5 to apply JPL’s unique skills to address problems of national significance andsecurity;

6 to enable a virtual presence throughout the solar system by creating the etary Network; and

Interplan-7 to inspire the next generation of explorers

In pursuit of this mission, JPL has a rich program of technology development,science, and mission development (the three “value-adding” processes of the Labora-tory)

To enable the mission of the Laboratory, JPL boasts an extensive infrastructure ofresearch, fabrication, test and design facilities and tools Employees make use of a robustand extensive intranet, serviced by high-speed networks, internal and public accessportals, and a multitude of Web-based systems, for example, to support accounting,human resources, document management, and internal communications functions.Hundreds of thousands of Web pages are published by individuals, teams, and organi-zations, and are accessible through directory and search utilities

JPL covers 177 acres north of Pasadena, California The university-like campus ishome to about 5,500 employees and on-site contractors Nearly three quarters of theworkforce are involved in R&D activities in support of the three value-adding processes

Of the R&D personnel, roughly one third have PhDs, and an additional one third havemaster’s or professional degrees JPL has an annual operating budget of approximately

$1.4 billion Additional information about JPL can be found at www.jpl.nasa.gov

SETTING THE STAGE

The system described in this article, JPL 101, is a series of Web-accessible quizzesbuilt upon a database of general organizational knowledge, which is encoded asquestions and has answers annotated with connections to related information andresources JPL 101 was conceived of by the Knowledge Capture (KC) team, a subset ofJPL’s Knowledge Management (KM) project This four-person team consisted of alibrarian, two Web and database system designers, and an engineer who alternatedbetween KM-related projects and working on Mars missions

The motivation for the system was twofold First, there was a growing concern by

KC team members that the KM project in general was isolated from the value-addingprocesses that formed the mainstream work of the Laboratory This isolation wasbelieved to lead to products and services that did not fully address users’ needs.The second motivation was a desire to share valuable knowledge gained through

a previous knowledge capture task Prior to his retirement in fall 2001, the Deputy Director

of the Laboratory agreed to do a series of retrospective interviews During his tenure,JPL went through a decade of sweeping changes that fundamentally altered the way JPLconducted business The primary purpose of the interviews was to collect informationfor the incoming deputy director who was new to the organization However, it was feltthat the insights gained during the interviews were of potential value to the greaterLaboratory population In particular, discussion about stakeholder relations and theinterplay between NASA, Caltech, and JPL served to make sense of the changes thatoccurred throughout the 1990s

This combination of motives led to the concept for JPL 101 It was felt that by callingattention to work related to the value-adding processes, the system could help improvethe connection of the KM team to the rest of the Laboratory In addition, by incorporatinginformation gained through the interviews with the deputy director, valuable insightsinto stakeholder issues and basic operations could be shared with the Laboratorypopulation

Although inspired by events local to the KC team, the circumstances correspond

to a broader organizational issue To perform the planetary exploration mission and “dowhat no one has done before,” large numbers of technical and professional disciplinesmust be integrated to support innovation (the value-adding process) In addition,infrastructure and support services are required to perform routine organizationalfunctions (the enabling processes) While cross-functional project teams have become

a common approach to integrating multidisciplinary knowledge in support of productdevelopment (Brown & Eisenhardt, 1995), less attention has been paid to bridging gapsbetween value-adding and enabling processes

In established firms, emergent knowledge processes (EKPs) (Markus, Majchrzak,

& Gasser, 2002), such as product development, take place within the context of theorganization’s bureaucracy The clash between those tasked with operating the bureau-cracy and those who must work within it can be viewed as another flavor of “thoughtworld.” Dougherty (1992) describes thought world differences between members fromthe marketing, engineering, and manufacturing functions in new product developmentteams Areas such as human resources, contracting, accounting, and informationtechnology also draw from different professional disciplines, focus on different criticalissues, and use different approaches to define and solve problems While cross-

4 Cooper, Nash, Phan, and Bailey

functional teams serve to bridge thought worlds by creating a shared vision of asuccessful, marketable product, there are few resources (e.g., mission statements) thatare effective at providing the same sort of actionable focus for the organization as awhole

Thought world-related problems, such as conflict and miscommunication, can bemitigated by helping people to learn about other domains and to recognize and exploitdifferences (Dougherty, 1992) Knowledge management systems (KMS) have the poten-tial to support this type of learning Knowledge-based approaches have been used tosupport transfer of best practices (Markus, 2001), knowledge reuse for innovation(Majchrzak, Cooper, & Neece, 2004), identifying experts, and a variety of businessprocesses (Davenport, Jarvenpaa, & Beers, 1996)

Therefore, JPL 101 was envisioned as an educational resource for Laboratorypersonnel, and a way to assist them in exploring the abundance of electronic and otherresources available to them The orienting question that guided development was “How

do you help people to make sense of the ‘big picture’ given that direct work-relatedexposure may be minimal (or nonexistent)?”

CASE DESCRIPTION

This case describes the 11-month evolution of JPL 101 from initial concept to fullyoperational system There were three distinct stages: (1) beta test of initial concept, (2)

feasibility analysis for use as a contest, and (3) implementation Each of these phases

is addressed in the following sections

Beta Test

The goal of the beta test phase was to quickly assess whether it was worth pursuingimplementation Due to the structure of the KM project, there was flexibility to exploreinteresting concepts, but implementation required explicit approval and sponsorship bythe KM project From the very beginning, JPL 101 was conceived of as a quiz The namewas chosen as a tongue-in-cheek reference to beginners’ classes in college to emphasizethe educational nature of the resource, and to convey that much of the content is basicmaterial that employees should know The quiz metaphor seemed like a natural approach

in an organization that values education as highly as JPL does

The beta test version consisted of a paper prototype Over the course of one week,the team brainstormed questions; experimented with different formats, difficulty, andwording of questions; and had much fun creating wrong answers The resulting 81questions were divided into three roughly equal groups Participants were given thethree sets of questions in different orders to make sure that all the test questions wouldhave at least a subset of the group looking at them Timed tests were then conductedwhere people worked their way through the quizzes As expected, there were theoccasional chuckles as people viewed the more humorous entries

Reaction to the quiz from the KM project team was generally positive but skeptical

as to the potential value of the system While this beta testing did not garner enthusiasticsupport from the KM project, it did provide feedback used to determine the rough size

of the quiz, appropriate mix of questions, and what constituted a reasonable level ofdifficulty for the questions

Beta testing of content provided insight into the types of questions that had thepotential to be controversial — primarily those that asked about absolutes such as

“firsts,” “only,” or “bests.” This led to standards for structuring a “good” question andguidelines for a reasonable amount of material to include in the answer

Following the internal beta test, organizations within JPL that were perceived aspotential stakeholders of the eventual system — Internal Communications, HumanResources, and the Ethics Office — were contacted Additionally, a shortened, improvedset of questions was tested as a demonstration model on actual work groups from theteam’s home organizations The response was overwhelmingly enthusiastic People wereanxious to share the information with their colleagues, contribute questions andanswers, and considered it both valuable and fun Everyone, including people who hadbeen with the organization for a number of decades, indicated that they learnedsomething either through the questions or the supporting information given in theanswers In addition to encouraging proceeding with development, people also begansuggesting questions that they thought would be good to include

The beta test phase ended in a surprising way with the serendipitous opportunity

to show one of the Laboratory’s highest-ranking executives the paper prototype He wasinstantly interested in the concept, brought his staff members in to have them take thequiz, and formulated the idea of using the JPL 101 concept as a Laboratory-wide contest

as part of the 40th anniversary of planetary exploration being celebrated that year Giventhis level of advocacy, the go-ahead from the KM project was quickly granted andimmediately began our second phase of development, the feasibility analysis of usingJPL 101 for a contest

By the end of the beta test phase, the following was achieved:

• Confirmation that the basic concept was sound and likely to be positively received

by the Laboratory population

• A cadre of stakeholders interested in seeing the system implemented

• A clear understanding of what constituted a well-formulated question: clear,concise, and simple structure; cautious use of absolutes; and humorous wording

• A practical approach to ensure correctness of the question by either triangulating

an answer (two-sources to confirm) or verification through an unimpeachablesource

• A requirement from the Knowledge Management project that the system age employees to explore the JPL intranet

encour-Feasibility Assessment

The direction to evaluate if and how JPL 101 could be used to support a wide contest led to a detailed requirements analysis and resulting design decisionsdescribed in the following At the same time, the team was also involved in a separateeffort investigating how to capture decision rationale It was decided to test some of ideasfrom that effort internally using informal decision-mapping techniques to capture therequirements generation process These decision maps form the basis for the followingdiscussion

Laboratory-Answering the question “Is a contest feasible?” first required answering a set ofkey questions, as shown in Figure 1 An assessment was conducted by methodicallyworking through each of these questions, identifying additional constraints, and

6 Cooper, Nash, Phan, and Bailey

incorporating stakeholder concerns The decision maps were used to track open issues,options, assumptions, information sources, and resulting requirements and designdecisions Even for a simple system such as JPL 101, the decision map quickly became

a tangled web of interactions and information that did not easily fit into single categories.The decision maps presented in the following sections are simplified for illustrationpurposes

How Do You Deliver the Quiz?

This turned out to be the easiest question to answer Two potential methods wereconsidered as shown in Figure 2 The first was to use a paper system, by sending a hardcopy to all participants This option was quickly eliminated as too burdensome due tothe need for hand scoring of the quizzes, no ability to time-tag responses, and thereproduction and mailing costs This option also was contrary to the KM requirement

to promote exploration of the intranet

The second option was to use a Web-based delivery mechanism via the JPL internalintranet In addition to being the area of expertise for our team members, this optioneliminated the negatives from the first option and contributed to a reasonable definition

of our participants After making this decision, the team immediately began prototypingactivities so that we would have a system to experiment on during the rest of theassessment period

Who are the Participants?

The delivery mechanism decision effectively limited participation to those who hadroutine access to the JPL intranet Four categories of potential participants wereidentified based on the derived requirement that participants have a JPL-issued badgeand identification number: current JPL employees, current JPL contractors, JPL retirees,and others resident at JPL but not falling into the previous categories From within thesecategories, several issues were identified:

How to determine the winner(s)?

What should the quiz look like?

How to deliver the quiz?

Who are the Participants?

What is the prize?

Figure 1 High level decision map

1 Timekeeping constraints: How much time could people in the individual

catego-ries commit to participating before we needed to provide them with an accountcode? This was resolved through the Ethics Office and resulted in a requirementthat each individual quiz takes 15 minutes or less Also, our Ethics Office confirmedthat JPL personnel could participate, but that the Human Resources departmentwould have to determine whether contractors could participate

2 Contractor constraints: Could contractors participate, and if so, were there any

timekeeping constraints, and were they eligible for prizes? These issues remainedopen during the feasibility analysis

3 Retiree participation: Would we actively work to enable retiree participation, and

if so, were they eligible for prizes? It was decided that our system should not

preclude participation from retirees as long as they had intranet access (we would not provide external access) and they had a JPL badge However, they would not

be eligible for prizes

As shown in Figure 3, these decisions led to the following:

• System must be capable of supporting an upper bound of 8,000 participants

• The individual quizzes must be sized to keep participation under 15 minutes

• Participants must have a current JPL badge and intranet access

• Only current JPL employees are eligible for prizes

What Should the Quiz Look Like?

Beta testing determined how to construct good individual questions The next set

of decisions addressed how to construct the quizzes Figure 4 shows the decision mapfor the quiz design In addition to requirements to keep participation under 15 minutes

How to deliver the quiz?

Paper?

Negatives

• Hand scoring

• No time tag capability

• Doesn’t support intranet goal

• Electronic, instant scoring

• Time tag capability

• Supports intranet goal

• Continued availability after contest is over

Figure 2 Delivery mechanism decision map

8 Cooper, Nash, Phan, and Bailey

and be able to differentiate 8,000 participants, specific goals that we set for the systemwere as follows:

• Make the quizzes challenging but not burdensome

• Ensure that the number of questions we have to generate is reasonable

• Have a broad mix of questions that include some aspect from all areas of theLaboratory community

The driving factor in the quiz design was the need to differentiate 8,000 participants

to determine the winner(s) We knew that there was limited project support for this effortand therefore felt that we would have resources to reliably develop only 100 to 120questions This is too small a number of questions to be able to distinguish the number

of potential participants solely based on score, so we determined that a time componentwas also needed

Several options were considered for incorporating a time-based component to thescore Our executive advocate had suggested a “fastest finger” approach where whoevergot the most right in the shortest time would win This approach, however, brought tobear issues of network latency (which is a function of the number of users) and wouldrequire that the system time tag all quizzes, leading to concerns about server load

A technically feasible approach to the quiz design was not possible until weanswered the question of how to determine the winner However, it was determined that

Participants?

Current JPL Employees

Issue: What is the time limit for participation before required to provide a charge number?

Contractors

Constraints:

•Need badges for identification

•No changes to individuals’ intranet access

Retirees Other

Issue: Are resident contractors allowed

to participate?

Decisions:

• Require <15 minutes (so no charge number needed)

• Upper bound of participants ~8000

• Contractors can participate

• Retirees and other can participate, but not eligible for prizes and no special effort made to enable participation

Figure 3 Participation decision map

we were looking at a four-week contest, consisting of four quizzes at 25 questions each,and that it was highly unlikely that we would be able to identify a single winner based

on this design

How Do You Determine the Winner?

One way to work around the inability to reliably create a single winner is to createmultiple categories of winners We assumed that it would be harder for newer employeesthan more experienced employees, and that different categories of employees wouldshine in different subject areas Based on these assumptions, participants would begrouped based on number of years of tenure, with three categories of under five years,five to 20 years, and more than 20 years, driven by the demographics of the Laboratoryand a desire for fair groupings

A multitiered approach was chosen, with weekly results feeding into identification

of grand winners The weekly results would be based on a score computed as the number

of right answers minus a fraction of the number of wrong answers, similar to the ScholasticAptitude Tests (SATs) Options for handling ties, which were highly likely on a weeklybasis, were a tie-breaker quiz, drawing names from the pool of highest scores, or simply

Quiz Design

Issue: Differentiate

~8000 potential participants

Constraints:

•Keep time to 10-15 minutes

•Reasonable number of questions for KC team to develop

•Fairness

Results:

•4 quizzes

•25 questions per quiz

•Score based on number right/wrong

•Don’t use time tag as component of score

•Avoid negative scores by including 5 easy questions

•Unlikely to identify winner based solely on score

Base score

Options:

• Number right

• Number right – ¼ * number wrong Issues:

• Avoid negative scores

• Keep number challenging without being burdensome

10 Cooper, Nash, Phan, and Bailey

accepting multiple winners The best choice for these options would depend on whetherprizes were given at the weekly level One consequence of our scoring scheme, whichwas chosen to increase differentiation, is that it would be possible to actually obtain anegative score To reduce this possibility, we decided to include at least five easyquestions per quiz

The multiweek format also led to multiple options for determining the grand winners,

as shown in Figure 5 These options, based on cumulative results, imply that winnersparticipate each week, which in turn raises issues of fairness (because people may be ontravel or vacation), and could result in a drop in participation due to people feeling thatthey were out of the running

Inherent in the determination of winners is the ability to identify the participantsand associate them with their results The multiweek format meant that we also needed

to correlate participation by the same individuals across weeks Therefore, our systemhad to

• have a method to handle ties;

• ensure a fair opportunity for participation;

• provide a fair and culturally acceptable method for determining winners;

Determining the Winner(s)

Create categories

Constraints:

•Fairness

Assumptions:

•Harder for newer vs more experienced employees

•Employees from different areas would do well on different types of questions

Results:

•Recommend categories based on tenure and possibly organization

•Need process for handling ties

•Possible big public event

•Recommend sponsor make final decision

• Based on tenure:

Under 5 years, 5-20, and over 20 years

• Based on organization

• Based on job classification

Determine Grand Winner

Multiple options, e.g.,

• Top cumulative winners in run off

• Top [n] from each week in run of

• Calculate based on best 3

• provide a method for differentiating between different categories of participants;

• have the technical capability to implement a scoring scheme based on both scoreand time;

• have a reasonable approach to addressing attempts to “game” the system; and

• reduce the probability of negative scores

What is the Prize?

Due to JPL’s status as an FFRDC, there were a number of constraints on the prizes.While the Ethics Office confirmed that it was all right to give prizes, our ContractsManagement Office ruled that prizes were an “unallowable” cost based on JPL’s contractwith NASA, and therefore would have to be paid for from discretionary funds and remainunder a specified value Our executive-level advocate said that his office would provide

or obtain the discretionary funds to cover the costs of the prizes Figure 6 provides anoverview of our decision process for determining prizes

Because the structure of the contest was two tiered with weekly quizzes leading to

a grand prize winner, we looked at a combination approach For weekly winners, a number

of ties were expected Rather than a prize, we evaluated different recognition mechanisms,for example, a system-generated e-mail for those obtaining a perfect score, or those with

Determining the Prize(s)

•Grand prize determined by random drawing

•Recommend sponsor make final decision

• List by category on website

• Certificate, coffee mug?

• Larger prize for top winner

• Gift certificate for dinner, trophy, JPL merchandise

Figure 6 Prize decision map

12 Cooper, Nash, Phan, and Bailey

the top scores Alternatively, we considered listing the top scores by category on a Website

The prize options included merchandise from the JPL gift shop, trophies, orcertificates, with the possibility of a higher-valued grand prize such as a gift certificatefor dinner We decided to leave the decision on the number of prizes to award, how toacknowledge weekly winners, and how to select the grand-prize winner up to the contestsponsor

Summary

Despite the relatively simple nature of JPL 101, the decision space quickly becamecomplicated with multiple interacting requirements and constraints Management waspresented the following options:

Option 1: Use the JPL 101 quiz for a Laboratory-wide contest Winners in each category

would be chosen based on best score over the four weeks of the contest Tokenprizes, as permitted, would be given to the finalists, with the ultimate grand-prizewinner(s) chosen from a random drawing of the finalists This option requiredadditional software development and coordination across multiple departments,but had the potential to generate significant interest and participation Additionaldetails would be worked out with the internal contest sponsor

Option 2: Proceed with JPL 101 as originally conceived without the contest element.

This option required minimal software development, kept the focus on the contentand therefore the goals of the KM project to promote intranet capabilities, and wasconsidered less risky However, it would not benefit from the executive-levelattention and did not have prize incentives as a way of gaining interest

After several months of debate, cost considerations won out, and Option 2 waschosen

Implementation

JPL 101 is a Web-accessible database of general organizational knowledge edge is encoded as questions, answers, and connections to related information andresources (see Cooper, 2003a for a detailed discussion of the use of the quiz interface).The system is organized into quizzes each containing five to 10 multiple-choice andmatching questions The deployment of the system took place over 12 weeks, after which

Knowl-it entered steady-state operation During each of the first 12 weeks, a new quiz was added.Following the 12-week initial deployment of the content, the system provided access tothe full set of past quizzes

The implementation of JPL 101 was relatively simple, with a minimal amount of userfunctions Due to rapidly dwindling support from the KM project, low maintenance costswere essential and the questions and answers needed to be robust with regard toobsolescence In addition to question and answer fields, the JPL 101 database alsoincluded administrative fields for identifying the category, originator, quiz, and valida-tion date for each question

During the initial 12-week deployment, the entry page for JPL 101 featured a directlink to the current week’s quiz Access to previous quizzes, background information, and

feedback mechanisms were provided through pull-down menus After the 12-weekdeployment period, the entry page provided a direct link to the list of previous quizzes

as well as the menu-accessible items

Design Considerations

JPL 101 was designed based on the assumptions that the general JPL populationhad access to a computer, was able to effectively use a Web interface, and would findthe use of a quiz-based model for the knowledge acceptable The first two are reasonableassumptions given the proliferation of Web-based institutional applications for generalexchange of information, support of business and administrative functions, and organi-zational communications The third assumption was validated during preliminary betatesting of the concept

Based on the assessment of the organization and with guidance from the Ethics,Human Resources, and Internal Communications offices, several constraints wereincorporated into the design process First, the overall set of quizzes were maderepresentative of concerns across the wide range of disciplines in the Laboratory so that

no group would feel “ignored” in the process and to ensure that the thought-world issueswere addressed Second, in order to avoid potential problems with time-keeping rules,the quizzes were kept short Third, we had to ensure that people could participate at theirconvenience, and that pragmatics, such as individuals being on travel, would not limitparticipation Fourth, since participation would be voluntary, there had to be motivations

to use the system Fifth, the goal of the system was learning, therefore it was critical thatthere were mechanisms for assessing whether people actually benefited from the system.Finally, it was important that people not feel that they were being graded or assessed inany way Therefore it was necessary to ensure that participants could take the quizzeswithout fear of violating their privacy This limited the type of performance andparticipation data that could be collected

Content

The heart of JPL 101 is the content The content categories were carefully chosen

to emphasize areas important to the Laboratory, essentially representing the differentthought worlds Table 1 provides a description of the different categories, the rationalefor including them, and an example of each

Over the course of the 12 weeks, a total of 66 questions were presented Eachquestion went through a rigorous quality check to ensure accuracy and that it met thestandards for a well-formulated question The distribution of questions across catego-ries is also provided in Table 1

Two areas received special attention in developing the questions: JPL Basics andStakeholders The 21 questions in the Basics category covered material ranging from how

to get help with computer problems to knowledge on new institutional resources and localrestaurants available after hours This is the type of knowledge that generally does notreceive high visibility, but contributes to the overall work environment The Stakeholdercategory consisted of 10 questions that covered the multiple constituencies to which JPL

is responsible Because JPL is a National Laboratory operated for NASA by the Caltech,there is a wide spectrum of stakeholders who influence the operations of the Laboratory.Understanding the nature of these stakeholder relationships and the various legal,

14 Cooper, Nash, Phan, and Bailey

contractual, and public trust concerns of the Laboratory is important for efficientoperation

Table 1 JPL 101 question categories

Area Description Rationale Example

Basics (n=22)

General knowledge about how JPL operates at and below the level of published procedures

Make it easier for employees to learn about things that make it easier to get their job done (and correct misconceptions)

What is the number to call if you're having computer hardware or software-related problems?

(A: x4-HELP) History

(n=6)

Knowledge of key accomplishments and

of individuals who contributed greatly to the Lab

Who was the director of GALCIT, and co- founder of JPL?

(A: Theodore von Kármán) Missions

(n=10)

Knowledge about missions, which are the primary product of the Laboratory and the focus of our work

Establish a connection to the past and share accomplishments that contribute to a sense of pride

Share the excitement of space exploration, which

is the reason for existence for the Lab

What is the name of the rover that explored the surface of Mars in 1997?

(A: Sojourner)

Product Developmen

t (n=9)

Knowledge about how the Laboratory builds and operates space missions and instruments

Where could you go at JPL to evaluate your spacecraft under environmental conditions that are similar to those found in space?

(A: 25-foot Space Simulator) Science

(n=5)

Knowledge about key scientific principles of importance in space exploration

What is the most active volcanic body currently known in the solar system?

(A: Jupiter’s moon, Io) Technology

(n=4)

Knowledge about the development of technology of importance in space exploration

The three JPL core processes represent the reason the Lab exists: our mission

of space exploration

All work at the Laboratory contributes either directly to one of these three areas, or

is responsible for supporting these processes

What is the name of the substance nicknamed

“frozen smoke”?

(A: Aerogel) Stakeholders

(n=10)

Knowledge about external entities that impact or are impacted

by JPL

JPL is answerable to multiple constituencies and is often highly constrained in the way it can operate

It is critical for JPL personnel to understand these factors and how they impact their work

Who is the President of Caltech?

(A: Dr David Baltimore)

In addition, the quiz database recorded the answers submitted each time someone took

a quiz

The online survey was used to collect basic organizational demographics (tenure,organizational unit, job category, and whether a manager or not) and responses to twoquestions: “Did you learn anything from the questions?” and “Did you learn anythingfrom the answers?” Taking the survey was voluntary, as was responding to thedemographic questions The second anonymous response method was an onlinefeedback form Users could submit comments, problems, feedback, and candidatequestions for the system While most users decided to remain anonymous, some madethe effort to include their names and contact information Finally, the e-mail basedfeedback form was available to contact the development team directly This was notanonymous and was the least-used form of feedback

Results

JPL 101 premiered on January 13, 2003, and ran for 12 weeks ending its initialdeployment on April 6 It remains in operation, although new content is not currentlybeing developed Results are presented based on analysis of the data collected duringthe initial 12 weeks, and extending through Week 19 of operations relative to thefollowing: design considerations, usage, motivation for use, learning results, and generalreaction

Design Considerations

Background usage and database data were analyzed to assess how well the designconsiderations were met Background usage data indicated success in meeting theparticipation time goals of the system The average time spent in the system eachworkday ranged from 2:01 minutes to 8:21 minutes, with the mean being 3:53, which arewithin the limits recommended by JPL Ethics and Human Resources offices

A second consideration was that the quizzes needed to be challenging but not toohard Figure 7 shows the average quiz scores for the 12 quizzes, based on data from theentire operational period With the exceptions of weeks five and eight, the average quizscores stayed between 70% and 90%, meeting the goal

0 20 40 60 80 100

16 Cooper, Nash, Phan, and Bailey

Additionally, there was a concern with question quality Because the JPL culture

is such that participants would readily point out any errors in the questions, evaluation

of question quality was based on the number of corrections required Two inputsregarding the accuracy of questions were received, one of which resulted in a minorchange (attributing an additional source for information in an answer) Given the volume

of material in 66 questions plus all the associated ancillary information, two minorcomments were well within the range for acceptable performance

Participation

Ultimately, a measure of success for a system is the number of people who use it.Given that this is a voluntary-use resource and not required for anyone’s job, participa-tion statistics are critical for gauging overall success Background usage statistics werecollected including hit rates and unique visitors based on IP addresses, modified to filterout members of the development team and automated Web crawlers During the 19 weeks

of operation covered in this study, a total of 2,144 employees participated, roughly 40%

of the Laboratory population Figure 8 shows the usage statistics over time for the 19weeks

In addition to reaching a large audience, the goal was to reach a broad audience.Although privacy and user-burden concerns prevented automatic collection of organi-zational demographics on general participants, a voluntary survey instrument was used

to collect some data Five hundred and thirty-three surveys were received over the course

of 19 weeks, representing a participation rate of just under 25% The organizational tenurefor participants ranged from brand new (zero years) to a maximum of 47 years, with anaverage of 15.3 years and a standard deviation of 10.5 years Users spanned the entireLaboratory, with participation concentrated most heavily in the Technical and Admin-istrative divisions, where the majority of Laboratory personnel are assigned Participantswere distributed across technical, administrative, and science disciplines, and includedboth managers and nonmanagers Taken in total, the data collected via the online surveyindicates a broad and substantial audience

0 200 400 600 800 1000 1200 1400

Email

End of Roll-Out

Figure 8 Participation by week, annotated to show key communication activities

Impact of Communication Mechanisms

Because JPL 101 is a voluntary-use system, providing general rather than specific knowledge, a number of institutional communication mechanisms were em-ployed to let people know this resource existed These mechanisms were as follows:

job-• JPL Universe: a traditional, biweekly organizational “newspaper” distributed to

personnel through interoffice mail There was a multicolumn story about JPL 101plus a sample quiz the week before rollout

• Cafeteria Monitors: closed-circuit television screens in the cafeterias that

broad-cast announcements Consisted of “teaser” questions — shorter versions of quizquestions, plus the URL for the site — for three days prior to rollout

• Daily Planet: electronic daily “newspaper” for JPL personnel Accessible via

intranet Publicity was via an a small graphic posted on the sidebar of the page thatlinked to JPL 101, starting the first day of rollout and continuing through the 12-week rollout period In addition, a short informational article was placed in centercolumn “news item” area during Week 5 of rollout

• Inside JPL Portal: Web portal that provides central access to JPL Web space for

internal users A link to JPL 101 was included in sections for new employees andinstitutional knowledge management during the first week

• This Week: electronically distributed (e-mail announcement with link to Web

page) weekly newsletter that highlights personnel announcements, organizationalchanges, and upcoming talks and events A one-paragraph blurb about JPL 101plus access information was included several times throughout the 12-week rollout

• All.Personnel e-mail: a tightly controlled list that sends e-mail to the entire

Laboratory population A single all.personnel e-mail was sent during Week 9.Publicity for JPL 101 began 1 week prior to its rollout Prerelease publicity included

an article in the JPL Universe and announcements on the JPL monitors In partnershipwith the Internal Communications Office, the primary entry point for JPL 101 was the

Daily Planet Unfortunately higher priority events limited entry to a single sidebar icon

during the initial weeks This icon remained until the end of the initial 12-week run Laterduring the first week, access was added via the Inside JPL portal These links continuedthroughout the entire period

The impact of each of these devices can be seen in the usage statistics shown inFigure 8 The first spike in the graph occurs during Week 5 and corresponds to thepublication of the Daily Planet article Additionally, a smaller increase, not visible in theweekly statistics but present in the daily statistics, occurred when links were added tothe Inside JPL portal The most prominent feature of the graph, however, is the giganticspike that occurs during Week 9 This corresponds to the sending of the all.personnele-mail publicizing JPL 101 This spike is due almost entirely to the day that the e-mail wassent

Learning Results

The primary goal of the system was individual learning Success was assessed inattaining this goal in two ways The first, and most direct way, was to use the survey tosimply ask participants if they learned anything Almost 90% of the survey respondentsindicated that they had learned something from either the questions, the answers, or

18 Cooper, Nash, Phan, and Bailey

both Preliminary analysis found small but significant negative correlations (p <.01)between tenure and learning, and being a manager and learning No other relationshipswere found

The second approach to evaluating learning was to look at the quiz response data.Figure 7 shows the average scores for each of the 12 quizzes These data indicate that

on average, people missed one to two questions per quiz, indicating that a learningopportunity existed Detailed analysis of individual questions shows that the number ofrespondents getting a specific question right varied from a low of 33% to one questionwhere everyone who answered got it right

There was also interest in how well people performed across the different categories

of questions and in what questions were skipped Table 2 provides a summary of theperformance in each of the categories Inspection of Table 2 data indicates that JPLpersonnel performed well on questions relating to the three value-adding processes,slightly below average on Basics, History, and Missions, and significantly belowaverage on Stakeholder questions While JPL 101 is not intended as a diagnostic systemfor organizational knowledge, these results suggest a gap in knowledge about stakehold-ers that should be remedied Inspection of the data on questions that were skipped clearlyshowed that matching-type questions were skipped more often than multiple-choicequestion, with all five matching questions placing within the bottom-six response rates

Other

Feedback via e-mail and through the online form was overwhelmingly positive (Thesole negative comment received via any of the feedback mechanisms was a complaintabout the use of the all.personnel e-mail.) For example, one respondent wrote, “This isgreat and I love it! I learned more about JPL in the past few weeks just by taking thesequizzes then the three years I have been here Thank you.” Several constructivecomments were made about how to improve the system Respondents were pleased withthe quiz-type presentation, and one suggested that “JPL 101 is the paradigm that should

be used for all training and knowledge dissemination at JPL.”

One area of disappointment was the lack of suggestions for questions During betatesting for JPL 101, one of the most surprising results was the level of excitementindividuals had over the idea of the quiz, and their desire to contribute questions andmake suggestions for material Because of this response, the feedback form in the systemincluded a field specifically for submitting potential questions Only three suggestionswere received, resulting in two new questions

Summary

In summary, the variety of data collected during the 19 weeks of operation for JPL

101 provided valuable information used to assess overall performance and success ofthe system The following section discusses these results and the potential learning to

be gained from them

CURRENT CHALLENGES/PROBLEMS FACING THE ORGANIZATION

JPL 101 was a small effort created to share special information and promoteintraorganizational appreciation for the different areas that need to work together toaccomplish the JPL mission When JPL controls spacecraft en route to other planets,small forces applied in the right direction at the right time are the difference betweenreaching the destination and missing by hundreds of kilometers The JPL 101 effort wasviewed in a similar light

The motivating factors for the creation of JPL 101 represent common themes inorganizations, for example, getting different parts of the organization to work togethereffectively, communicating culture and values to new employees, addressing stake-holder concerns, aligning infrastructure and support functions with value-addingprocesses As with many KM systems, the effects of the knowledge conveyed throughJPL 101 cannot be measured directly (Cooper, 2003b) Conditions before and after remainvirtually indistinguishable The differences, if any, have been small and below thesurface, for example, less frustration when following a policy, a little more respect forothers doing their jobs, and a greater sense of community By having a positive individualimpact, we expect to have a positive organizational impact, as suggested by Jennex andOlfman (2002) While we cannot measure it, the net result of JPL 101 was that nearly halfthe employees learned something new that is relevant to the organization And that, inturn, should have a positive effect on the organization

As noted by Kuchinke (1995), “organizations have in fact little control over whetherlearning takes place, but they do have potentially substantial amounts of control overthe kind of learning that occurs within their bounds” (p 309) In this respect, JPL 101provides a learning opportunity where the content, by its mere presence, indicates adegree of organizational importance and the system serves as an intervention aimed atreducing thought-world differences between personnel

The deployment of JPL 101 also led to gaining new insights into the developmentand use of knowledge management-type systems at JPL First, fun worked The use ofhumor and clever construction of questions and answers did not diminish the fundamen-tal value of the content, but instead contributed to user satisfaction

Second, there were remarkable differences in the effectiveness of different tional communications channels, as evidenced by the usage data While one must be

institu-Table 2 Summary of performance across question categories

Questions

Average % Skipped

Average % Right

Product Development

20 Cooper, Nash, Phan, and Bailey

cautious about extrapolating from a small number of experiences, the data for JPL 101imply that specific channels are more effective in motivating participation than others

In this case, the all.personnel e-mail (which was short and clearly indicated thatparticipation would take a small time investment with high potential for payoff) resulted

in orders of magnitude increases in participation

Third, the differences in successful response rates for different question categories

do provide a level of diagnostic information regarding gaps in individual knowledgeabout the organization The particularly low scores in the stakeholder category rein-forced the concern about general awareness of stakeholder issues This informationcould be used to modify communication and training activities to place special emphasis

on areas with subpar performance

Fourth, the feedback responses were overwhelmingly positive, particularly withrespect to the quiz interface Given the JPL culture, it was felt that this was a goodapproach (Cooper, 2003a), but there was surprise at the level of enthusiasm and with thedegree of frustration expressed regarding other online training interfaces This resultindicates that modifications to existing training approaches may be warranted.Finally, the future value of a KMS is dependent upon continued support Manage-ment support (e.g., funding) for JPL 101 stopped immediately after the initial 12-weekdeployment No new content has been developed and updating of the current content

is on a volunteer basis This was anticipated and the questions were designed to minimizeobsolescence and the system incorporated mechanisms to make content maintenanceeasy (e.g., on the order of minutes to update questions or answer content) It is the sense

of ownership felt by the development team coupled with the intentionally nance design that keeps this system operational

low-mainte-JPL 101 has been in operation for over 18 months During that time, only fivequestions became obsolete due to reorganizations and personnel reassignments How-ever, the content included in the answers to those questions provided links that wouldtake the participants to the correct information Usage levels have dropped to less than

20 users per month, but there are both new and repeat users, with new employeesaccounting for about one third of the participants Online survey responses continue toshow that well above 90% of respondents feel they have learned something as a result

of participating

The factors motivating the development of JPL 101 still exist in the currentenvironment, and will probably continue to exist for the foreseeable future Organizationsmust continuously work to facilitate understanding and respect across the differentcomponents of the organization The potential impact of JPL 101 during its initial 12-weekdeployment was enhanced by having large numbers of employees from across theorganization learning and thinking about the same things at the same time The potentialnow has changed somewhat as small numbers of individuals access the system in an adhoc fashion, reducing the “shared-experience” aspect The system does, however,provide a means of reinforcing previous learning for repeat visitors, and can help newemployees begin the acculturation process Even the obsolete questions serve a purpose

by capturing a snapshot of the organization and key personnel as they had existed during

an important period in JPL’s history While the current organizational climate is notconducive to continuing development of JPL 101, we are confident that future opportu-nities will exist to extend the system

litera-Senge, P., Kleiner, A., Roberts, C Ross, R., & Smith, B (1994) The fifth discipline

fieldbook: Strategies and tools for building a learning organization New York:

Currency Doubleday

ACKNOWLEDGMENTS

The work described in this article was carried out at the Jet Propulsion Laboratory,California Institute of Technology, under contract with the National Aeronautics andSpace Administration We would like to acknowledge the contributions of Eric Ramirez

in the implementation and administration of JPL 101, and offer special thanks to BarbaraAmago, Winston Gin, Cara Cheung, Sanjoy Moorthy, and Angela McGahan for theircontributions An earlier version of this chapter was presented at the 37th HawaiiInternational Conference on Systems Sciences – 2004

REFERENCES

Brown, S.L., & Eisenhardt, K.M (1995) Product development: Past research, presentfindings, and future directions Academy of Management Review, 20(2), 343-378.Cooper, L.P (2003a) The power of a question: A case study of two organizationalknowledge capture systems Proceedings of the 36th Annual Hawaii International Conference on System Sciences.

Cooper, L.P (2003b) A research agenda to reduce risk in new product developmentthrough knowledge management: A practitioner perspective Journal of Engineer-

ing and Technology Management, 20, 117-140.

Davenport, T.H., Jarvenpaa, S.L., & Beers, M.C (1996) Improving knowledge workprocesses Sloan Management Review, Summer, 53-65

Dougherty, D (1992) Interpretative barriers to successful product innovation in largefirms Organization Science, 3(2), 179-202.

Jennex, M.E., & Olfman, L (2002) Organizational memory/knowledge effects on tivity, a longitudinal study Proceedings of the 35th Annual Hawaii International Conference on System Sciences.

produc-22 Cooper, Nash, Phan, and Bailey

Kuchinke, K.P (1995) Managing learning for performance Human Resource

Chapter II

A Knowledge Management Case Study in Developing,

It focuses on the Knowledge Management (KM) solution developed to support employees

to sustain their learning, to enable them to share their insights and experiences with others, and thus increase organizational capability The paper is written to illustrate

an example of a large organization’s efforts to engage employees to share their learning from a management programme across geographical and cultural boundaries.

INTRODUCTION

This case study reflects the work of a global organization in its knowledgemanagement efforts to sustain and transfer learning from a global leadership develop-ment curriculum It focuses on the Knowledge Management (KM) solution developed

to support employees to sustain their learning, to enable them to share their insights andexperiences with others, and thus increase organizational capability The paper is written

24 McGregor-MacDonald

to illustrate an example of a large organization’s efforts to engage employees to sharetheir learning from a management programme across geographical and cultural bound-aries

Georgensen (1982) estimates that learners retain approximately 10% of materialcovered in a tutor-led workshop when back at the workplace The KM strategy in thisproject was to support high-performing, high-potential employees to retain a greaterproportion of the tutor-led learning and experience This in turn increases organizationalcapability by transferring the learning to colleagues and delivers a greater return oninvestment to the business

A key challenge of the KM strategy was to effectively manipulate existing KMplatforms within the business and research and propose the use of additional ones.The issue was to make best use of the current multiple resources in the organization,acknowledging that not one of them was totally suited to meet the needs across the globe.The Learning and Development team worked to find a solution with either a range ofexisting platforms or, as a result of research and testing of new technologies, a new KMplatform to support the strategy

There are a number of cultural challenges associated with implementing effective

KM across a global organization with presence in over 100 countries, with different levels

of technology sophistication, language, and experience Revenue-generating businessdemands mean implementing an effective KM strategy with “learning” content asanother challenge entirely For example, time spent documenting personal reflectionsfrom learning and on-the-job experiences, and reading others’ reflections from learningand on-the-job experiences struggles to compete with business opportunities thatdeliver an immediate bottom-line return

The nature of the insurance industry is relationship based Interaction has cally been, and still is, predominantly face-to-face or over the telephone As Nixon (2000)confirms, many other industries have found implementing effective technology-based

histori-KM solutions with only face-to-PC interaction is a cultural and pragmatic challenge Intheir everyday role, brokers prefer to pick up the phone and talk to someone or go to seethem versus logging on to a computer, entering a password they need to have remem-bered and change regularly to maintain security protocols The Lloyds of Londonbroking environment, established in 1688, reinforces the face-to-face relationship-basedculture Experience of working with an internal client group to support employees to usethe system suggests that if the Internet connection is slow or a password is typedincorrectly thus denying access, users will pick up the phone before trying again, orworse, will avoid the system in future

BACKGROUND

The Organisation

Marsh Inc is the world’s leading risk and insurance services firm Its aim is “[t]ocreate and deliver risk solutions and services that make our clients more successful.”Founded in 1871, it has grown into a global enterprise with 400 owned-and-operatedoffices and 42,000 colleagues, who serve clients in more than 100 countries Marsh’s

annual revenues are $6.9 billion, and the company meets client needs in two principalcategories:

• Risk Management, insurance-broking, and programme-management services areprovided for businesses, public entities, professional services organisations,private clients, and associations under the Marsh name

• Reinsurance-broking, risk and financing modeling, and associated advisory vices are provided to insurance and reinsurance companies, principally under theGuy Carpenter name

ser-The organisation is made up of distinct divisions with specialist knowledge One

of the key business drivers for the future is to maintain and develop the specificknowledge within each of these divisions, while sharing more learning and experiencesacross the business, particularly to reduce “reinvention of the wheel” comments acrossdivisions and geographies

SETTING THE STAGE

Knowledge Management Platforms in Learning

Newman (1991) defines KM as “the collection of processes that govern the creation,dissemination, and utilization of knowledge.” The cascade and consistent communica-tion of corporate goals and performance management is pivotal to business success,learning interventions, and employees’ personal development In 2000, Marsh made afundamental shift in the mechanism used to cascade company strategy across the globe.Local performance management tools, processes, and procedures were replaced with onecommon approach to aligning goals and consistently measuring performance with theBalanced Scorecard.1

At the beginning of 2001, there was no common, pan-European technology platformspecifically targeting learning and the consistent documentation of learning in Marsh.E-mail provision was the one common tool and platform across the globe The companyhad a variety of software to support the creation and application of databases and hadthe capability to share databases across geographies, through shared network drives,Internet-based secure “filing” programmes, Microsoft Access and Lotus Notesprogrammes Few employees were aware of the range of these capabilities and even fewerwere aware of how to manipulate such tools

In 2001, the firm implemented a global learning management system with specific,pan-European capabilities including e-learning, registration for tutor-led learning, and

an online lending library with books, CDs, tapes, videos, and computer-based training(CBT) The system also provided the capability to record for each learner what learningthey had accessed and to allow an “approver” path for line manager involvement andalignment to learning Usage statistics have increased from 11% of the total Europeanpopulation in 2001 to more than 28% in 2004

In 2002, the organisation launched a company-wide portal, an interactive client andcolleague platform to source information about Marsh to both external and internalrequestors The portal is intended to ultimately replace local country-specific intranet