ORGANIZATIONAL LEARNING THROUGH POST-PROJECT REVIEWS IN R&D doc

In this paper we review the role of post-project meetings as a tool to improve organizational learning at the group level.. Post-project reviews should aim at capturing process knowledge

Organizational learning through

post-project reviews in R&D

Maximilian von Zedtwitz

IMD – International Institute for Management Development, P.O Box 915, CH-1001 Lausanne, Switzerland, zedtwitz@imd.ch

Post-project reviews are one opportunity to systematically improve performance in subsequent projects However, a survey reveals that only one out of five R&D projects receives a post-project review Post-project reviews – if they take place – are typically constrained by lack of time and attention as well as lack of personal interest and ability They focus mostly on technical output and bureaucratic measurements; process-related factors such as project management are rarely discussed

In this paper we review the role of post-project meetings as a tool to improve organizational learning at the group level Based on 27 in-depth interviews with R&D managers carried out between 1997 and 2001,

we categorize four classes of learning impediments These difficulties are not easily resolved, as is illustrated by examples from Hewlett-Packard, DaimlerChrysler, SAP, Unisys, the US Army, and others We propose a five-level post-project review capability maturity model, identifying some of the key capabilities that need to be in place in order to advance to the next process maturity level Most companies reside on the first or second maturity level Our conclusion is that many companies give away great potential for competence building by neglecting post-project reviews as a tool for systematic inter-project learning

1 Introduction

What do you take away from a finished project? A

product? A lesson? A bad feeling? The essence is

that most companies have not established a structured

approach to learning from projects after their

comple-tion Even worse, most projects that have been

prematurely terminated never undergo a retrospective

analysis on their causes of failure A recent survey

indicated that 80% of all R&D projects are not

reviewed at all after completion, and most of the

remaining 20% were reviewed without established

review guidelines

This paper raises the issue of organizational learning

through post-project reviews A post-project review is

here defined as a formal review of the project examining

the lessons that may be learned and used to the benefit

of future projects Post-project reviews – or

post-reviews – are more prevalent in industries where

knowledge is the main output of a project For instance, consultants regularly perform after-action reviews in order to capture lessons learned from their engagements Project reviews are commonly used as phase reviews, particularly in projects managed under the stage-gate paradigm The final review, however, is either focused exclusively on technical issues or dropped altogether due to time and management constraints Post-project reviews should aim at capturing process knowledge for enhancement of future project work, and hence differ from regular project reviews, project audits (Duffy, 1989; Neale and Homes, 1990), or project evaluations (Saladis, 1993)

In this paper we propose a five-level capability maturity model for post-project review processes in R&D Based on our review of the literature on organizational learning and R&D project manage-ment, we identify eight fundamental impediments to post-project reviews in R&D We reflect on a few

R&D Management 32, 3, 2002 # Blackwell Publishers Ltd, 2002 Published by Blackwell Publishers Ltd, 255

observations on how to cope with these impediments

to promote organizational learning based on team

performance

2 Research methodology

At the intersection of group learning and R&D project

management practice, the study of post-project reviews

has not attracted widespread interest from R&D

management researchers until recently Little research

has been published in the academic or management

literature In addition, practitioners have repeatedly

voiced their concern about the lack of established

criteria and guidelines to conduct post-project reviews,

but their focus on future project challenges usually

distracts them from a retrospective analysis of the past

The research leading to this paper is based on a

combinative effort of doctoral research, in-depth

interviews with R&D managers, and survey

explora-tion As the goal of the paper is to identify current

R&D post-project review practices and propose

managerial recommendations on how to improve

them, we chose to begin with a broad interviewing

sample for variety and reach of insight, preparing for a

focused and limited in-depth investigation that will

eventually produce a richer and more substantiated

output in our ongoing study As our level of analysis,

we chose the boundary between individual and team

learning

Between 1997 and 2001, we conducted a total of 27

interviews with R&D managers of leading companies

from a wide range of technology-intensive industries

and geographical regions The interviews were

con-ducted according to an interview guideline that focused

on project management in R&D, specifically learning

and information sharing in R&D teams Interview

minutes were returned to the interviewees for feedback,

comments and additional interpretation Where

possi-ble, we participated in project meetings and

post-project reviews This personal observation

comple-mented the interview data well Data triangulation, as

demanded by Yin (1991), was not possible in every

instance as very little has been published by either the

interviewed companies or third parties on the

investi-gated projects However, we had access to post-project

reports of some of the studied companies

In addition to the interviews, we conducted two

surveys of R&D post-project review practice in 2000

and 2001 Sixty-three R&D directors and managers of

different organizations participated in our

question-naire of the current state of R&D post-project reviews

in their companies, including information about

frequency and motivation of post-project reviews as

well as process-related factors during their execution

As this is ongoing research, we expect our

under-standing to be refined with the progress of this work

Our analysis based on the initial survey (see Koners

and Zedtwitz, 2001) was confirmed by findings from the second survey conducted a year later Considering all currently available data, however, we are confident that the present report presents a fairly reliable picture

of the fundamental issues of managing post-project R&D reviews

3 Post-project reviews in organizational learning and R&D management

3.1 Defining post-project reviews

According to the great satirist and critic George Bernard Shaw, ‘we learn from experience that men never learn anything from experience’ Luckily, re-search in organizational learning over the past decades shows an improving understanding of how people and

organizations learn (see Easterby-Smith et al (2000)

for a concise and recent review) Among a number of important mechanisms, the post-project review is one

of the most structured and most widely applicable approaches to passing on experience from one team to the next

We define a post-project review as the final formal review in the course of a project that examines any lessons that may be learned and used to the benefit of future projects (see also the projectnet.com definition and Wideman, 1992) This section, as well as the following section on team learning (including Figure 1 and Table 1), is strongly based on the contribution of Ursula Koners to our joint publication reporting findings from the initial post-project review survey (Koners and Zedtwitz, 2001) The main objective of post-project reviews is to initiate and facilitate continuous learning on all levels within an organiza-tion Learning effectively protects and enhances an organization’s competitive advantage (e.g

Wheel-wright and Clark, 1992; Johannessen et al., 1997;

Gupta, 1998; Drejer and Riis, 1999; Takeuchi and Nonaka, 1986)

Learning is critical in R&D organizations since it is the precondition for sustaining significant improve-ments over long periods of time The body of knowl-edge on learning in R&D and New Product Development has been growing in recent years, but the variety of learning concepts studied is immense and the underlying processes are still scarcely known (e.g Reger and von Wichert-Nick, 1997) One major drawback is the absence of investigations on learning

in R&D after a project is actually finished Most of the existing literature on post-project reviews in R&D repeats their importance and the fact that few organizations conduct post-project reviews regularly Generally, the final phase of an R&D project concentrates on issues like the financial closure, the project appraisal and the reallocation of equipment and workforce to future projects (EIRMA, 1998) This

paper wants to advance the discussion further in the

direction of inter-project learning capabilities – and

barriers – in post-project reviews in R&D

3.2 From team learning to organizational

learning

Argyris (1977) defined organizational learning as ‘a

process of detecting and correcting error’ The notion

of organizational learning has since evolved (see e.g

Crossan and Guatto, 1995; or Garvin’s (1993)

defini-tion of ‘an organizadefini-tion skilled at creating, acquiring

and transferring knowledge and at modifying its

behavior to reflect new knowledge and insights’)

We distinguish between three levels of learning:

individual learning, team and group learning, and

organizational learning (see Figure 1) At the basis of

all learning is the individual (e.g Barker and Neailey,

1999, p 60) Individual learning is the domain of

psychology and cognitive science; organizational

learn-ing is studied by social psychologists and organization

theorists In this context, team learning may have a

pivotal role in distributing, processing and interpreting

individual experience for organizational memory and

knowledge

Post-project reviews focus on the link between team

and organizational learning, and – as we will discuss

below – on the link between individual and

organiza-tional learning The process of organizaorganiza-tional learning

through team learning is best described in Argyris and

Scho¨n’s (1978) concept of single-loop and double-loop

learning Single-loop learning pertains to the detection

and correction of mismatches between experience and

a reference system without questioning or altering the

values of the system Double-loop learning takes place

when a mismatch is detected and used to adapt and

correct the reference system

Post-project reviews (should) focus on double-loop

learning After the completion of the project – when

the project team is ready to step back and reconsider

what it has done and what has happened – the team is

in the best position to review cause-and-effect relation-ships and should be required to propose improvements for the management and execution of future projects Unfortunately, for a number of reasons which we outline further below, few teams find the appropriate amount of time for this exercise Most learning is confined to individual learning, and substantial poten-tial for group learning is missed Although individual learning is an important ingredient of organizational learning, both the depth and reach of individual experience are on average more limited

3.3 Learning in the life of an R&D project

A review is certainly not the only way to learn in or from a project Although learning and knowledge accumulation in R&D are at the core of R&D, knowledge dissemination to other scientists and R&D managers is poorly facilitated and supported Learn-ing, state Ayas and Zeniuk (2001, p 64), ‘is not a natural outcome of projects knowledge created within a project is not always diffused, and lessons learned may not be shared across projects’

Learning through reviewing is one of the most frequent approaches to capitalizing on previous experience, and many R&D project management concepts (such as stage-gate (O’Connor, 1994; Cooper and Kleinschmidt, 1991) or milestone techniques) are based on formal points of reflection.1 Figure 2 illustrates a schematized phase model of R&D project reviews (Balanchandra and Brockhoff, 1995, p 32)

It would be dangerous, however, to focus exclusively

on formal methods of learning and neglect other informal and tacit means of knowledge and experience building Some of the greatest advances in R&D management in the 1990s have been soft and informal management techniques, which tend to be extremely effective at the individual learning level However, they are difficult to systemize and replicate

For example, SAP carries out a number of different reviews over the course of an R&D project Reviews

Individual Learning

Team/Group Learning

Organizational Learning

Levels of Learning How does learning occur?

Knowledge acquisition

Information Distribution Information Interpretation

Organizational Memory

How is knowledge embodied?

•Personal memory

•Personal experience

•Notes

•Individual capabilities

•Networks

•Reports

•Products

•Team-specific expertise

•Technologies

•Stories / Anecdotes

•Databases

•Procedures

•Processes

•Core competencies

Figure 1 From team learning to organizational learning (based on Huber, 1991).

can be held as specification reviews, design reviews,

coding reviews, and even reviews of online

documenta-tion Reviews are designed to ensure that everyone has

access to the same project-critical information;

infor-mation that is important for discussing problems and

finding solutions, discovering potential risk areas,

status tracking, introducing new design proposals,

coordination with other groups, and success checks

Reviews are not necessarily formalized and are

some-times held in small, informal group meetings

How-ever, the results of reviews should always be recorded

in project documentation

Learning from reviews is not limited to the life-time

of an R&D project While a final post-project review

closes out the current project, it may also create the

grounds for a new project For instance, after the

development of a client=server wholesale banking

software product by Unisys, a post-project meeting

(post-mortem in their terminology) was called

includ-ing the programme manager, the customer project

manager, the responsible Unisys International Engi-neering Center project manager, the Unisys Interna-tional Banking Service Center development director, its support manager, and its GUI engineer, the product’s marketing representatives, and the European marketing manager Based on reviews carried out by the Engineering Center the future benefits of lever-aging the development work, including process improvements for future projects and specific require-ments for product internationalization, the Engineer-ing Center and the BankEngineer-ing Service Center agreed to collaborate on a localization project of the banking software Here the post-project review was not only a triggering event to identify additional market potential for the just completed development project, but was also a means to help marketing people use engineering information for future project bids

Post-project reviews thus help improve learning from one R&D project to the next Post-project reviews introduce a systematic way to double-loop learning by making project-specific knowledge and

Idea

Stage I:

Feasibility

Stage II:

Development

Stage III:

Test Market

Stage IV:

Commercialization

Review

Review

Review

Review

Project

Graveyard

Figure 2 Stages in the life of an R&D project (source:

Balanchan-dra and Brockhoff, 1995, p 32).

Organizational and Technical Knowledge Pool

Idea

Stage I:

Feasibility

Stage II:

Development

Stage III:

Test Market

Stage IV:

Commercialization

Review

Review

Review

Review

Idea

Stage I:

Feasibility

Stage II:

Development

Stage III:

Test Market

Stage IV:

Commercialization

Review

Review

Review Review

Post-Project

Double-Loop Learning

Single-Loop Learning

Figure 3 Double loop learning through post-project reviews in R&D projects.

Table 1 Differences between single- and double-loop learning

Characteristics * Occurs through repetition and routine * Occurs through use of insights and non-routine

* Occurs at all levels in organizations * Occurs mostly in upper levels Consequence * Behavioural outcomes * Insights and collective consciousness

Examples * Institutionalizes formal rules * New missions and new definitions of direction

* Adjustments in management systems * Agenda setting

* Problem-solving skills * Problem-defining skills

* Development of myths, stories and culture Application in

post-project

reviews

å Discussion of variances in expenditures, missed deadlines etc å Suggestions for the application of lessonslearned to future projects

å Retrospective analysis of major obstacles experienced

å Deep analysis of cause-effect relations regarding major obstacles experienced

Source: adapted from Fiol and Lyles (1985, p 810).

experience available to a corporate-wide pool of

organizational and technical knowledge (see Figure 3)

But few companies have achieved double-loop

learning with the help of post-project reviews

Post-project reviews can limit double-loop learning if they

are too inward-oriented and focused on a reactionary

agenda Table 1 illustrates the differences between

what constitutes single-loop and double-loop learning

in current PPR practices Most can be generally

described as ‘single-loop’ and are therefore restricting

their inherent learning potential

4 A review of current post-project review

practices in R&D

4.1 Results from an exploratory survey on

R&D post-project reviews

Although most R&D organizations seem to

under-stand the potential benefit of post-project reviews, they

still do not make full use of this learning opportunity

We conducted two surveys among a group of R&D

directors and senior R&D managers on the extent and

quality of R&D post-project reviews in which they had

participated The venues were two executive training

events on innovation management held at a business

school in September 2000 and October 2001 We

collected a total of 63 valid responses This number is

too low to warrant in-depth statistical evaluation, but

it does allow some interesting observations about the

state of R&D post-project reviews

On average, 19.4% of R&D projects in which the

respondents had participated had been post-reviewed

Although post-project reviews were conducted in

almost every company and within most departments

represented, post-project reviews were conducted

mostly on an ad hoc basis or after particularly large

projects Only 12 (of 63) respondents indicated that

their companies tried to post-review as many projects

as possible, and 55.6% of the respondents stated that

their companies had not established formal guidelines

on how post-project reviews were to be conducted

17.5% of the respondents indicated that post-project

reviews followed similar practices to those employed in

earlier projects, and only 11.1% of the respondents’

companies had defined review practices

The most popular means of disseminating know-how

between projects was through individuals moving to

new projects (52.4%) or through written documentation

(39.7%) Just eight of 63 respondents claimed that

post-project review results were effectively used to improve

project management or were an integral part of

inter-project learning 39.7% stated that their review focused

mostly on technical criteria, and just 6.3% found that

their companies applied sound and consistent criteria to

every post-project review The complete questionnaire

and its results can be found in the Appendix

Our survey confirms a benchmarking study of 79 highly regarded R&D organizations conducted by Menke (1997) who stated that less than a quarter of the 79 organizations made full use of post-project reviews (see also Kumar, 1990) However, we are aware of the limitations of our investigation due to the restricted scope and sample of the survey But even with these considerations in mind we must conclude from our survey that great learning potential is lost by not taking formalized team learning seriously enough Among the 63 valid respondents, 59 indicated they would prefer to see more post-project reviews to follow

up on completed R&D projects This is no easy task, as

we will see from the following chapter

4.2 Post-project reviews in industrial R&D

What post-project review practices have been estab-lished in R&D-intensive companies, and have any of them best-practice character? Considering the low attention and time that R&D management devotes to post-project reviews, and the multitude of difficulties and obstacles faced in executing these reviews, we were expecting that only a few companies would have developed all-encompassing approaches that could serve as leading examples for their peers

We collected a number of examples that illustrate how some companies have addressed specific problems

of post-project reviews Hewlett-Packard (HP) is one of the companies that has been heralded for its fairly consistent and established implementation of post-project reviews Some of its business units do not only have official project closure meetings but also ‘retro-spective’ project meetings looking exclusively at the development process and potential improvements Their main aim is to gather root causes for problems witnessed during the project and then to make recommendations on how such events can be avoided

in future projects This effort is based on an HP-wide project management initiative established in 1989 and later reinforced in a 1993 HP project management council Since then, HP has been striving to develop project management as a company-wide core compe-tence This strategic emphasis on project management helped to accommodate post-project reviews despite the additional administrative requirements Special review facilitators focus on coaching post-project review meet-ings Still, the frequency of post-project review varies between business units The need for reviews has not yet trickled down into every R&D department, and the quality and result of each post-review meeting depends

on the skills and talents of the review facilitator Most companies rely heavily on personal networks

to disseminate experiences from project management, including all forms of personnel rotation, temporary project assignments and core team hand-overs Some companies also establish central service centres within their central R&D whose members are in effect human

dissemination tools for project experience

Daimler-Chrysler, for example, offers support and coaching

staff to its R&D project managers from its FTK=P

department These coaches are not involved in the

technical work of the project but support the project

team in all questions concerning the project execution

and facilitation process Depending on the size and

strategic importance of the projects, these coaches are

responsible for up to five projects and are able to

capture and re-apply their in-depth knowledge to

future assignments and fellow project members

In order to retain and develop a cadre of highly

skilled global R&D project managers, Hoffmann-La

Roche’s Pharma Division instituted an ‘International

Project Management Department’, coordinating a

resource pool of about 50project managers for all

R&D projects worldwide (see also Gassmann and

von Zedtwitz, 1998) The members of this ‘virtual’,

geographically decentralized department are also

assigned as managers to projects as part of a global

programme to ensure standards in quality and project

procedures Upon completion of these projects, project

managers return to the virtual resource pool Since

there are more projects in the pipeline than there are

potential project managers available, they are

imme-diately reassigned to new projects

Since the director of this department reports

directly to the corporate board, the internal political

position of R&D project managers has been improved

recently The director of the ‘International Project

Management Department’ is also on the International

Project Committee, which decides over the

approxi-mately 60global R&D projects at Roche Pharma The

director assumes a role as interpreter or liaison

between project managers and top management, thus

representing the interests of international project

management at Roche This virtual pool promotes

the project management idea Experienced project

managers are dispersed around the world and wander

from project to project – no matter where the project

will be conducted Roche thus manages to retain

much of the valuable procedural know-how to

conduct and lead international projects not only

inside the company, but also in a position where it

can be reapplied when needed

Although top-management support is important in

recognizing the strategic significance of post-project

reviews, it is not always beneficial that top

manage-ment personally participates in them Depending on

the company culture, the presence of research directors

might prevent open and frank discussions of

experi-enced problems DaimlerChrysler, on the other hand,

classifies R&D projects into ‘A’, ‘B’ and ‘C’ projects

(depending on their budget and strategic importance);

‘A’ projects are reviewed in front of the CTO, ‘B’

projects with the relevant R&D director, etc However,

the higher the hierarchical level of participants, the

more likely it is that the post-project review develops

into a marketing event rather than an analysis of the finished project

DaimlerChrysler introduced a new review proce-dure in R&D in 1998, giving directions on the format

of Powerpoint slides as well as details on mandatory information to be given Agilent stresses the impor-tance of cross-functional participants to enhance the array of issues raised during the post-review (see also the Unisys example above), underlining that ‘low-key’ events also help to achieve generally more honest results than formal meetings A similarly informal approach is adopted by many software development companies (e.g., SAP), although some of them regard post-reviews as wild brainstorming sessions without fixed agendas Professional networks and clubs at Dow Corning and DuPont also provide platforms for informal know-how exchange of project management Sinofsky and Thomke (1999) have collected a number

of hands-on guidelines on how to conduct a post-project review session Again, the approach chosen depends heavily on the existing company culture and underlying motive for conducting post-project re-views

Some companies insist on maintaining the post-review as a completely internal meeting with no outsiders present Others, for instance Novartis and Hewlett-Packard, opt for external facilitators in order

to profit from the perspectives of an objective outsider

to the project External facilitators may come from an internal coaching or quality management department, like at DaimlerChrysler FTP=K or Agilent, or hired externally to do the job These outside facilitators are usually consultants specialized in process develop-ment

Obviously, post-project reviews take place after a project is finished The time span between project closure and the post-review meeting, however, varies considerably Some companies choose to conduct post-reviews immediately after the delivery of the last project milestone when the memory is still fresh and most project members still present Other companies prefer to wait until after market introduction, when customer feedback can be taken into account as well Schindler, for example, waits approximately two years after the actual product launch before conducting the post-project review

While all these examples show that current post-project review practices in R&D are quite hetero-geneous, it does not seem to be appropriate to impose

a single best practice of post-project reviewing Different objectives and needs, different markets and industries, different cultural contexts, and differ-ing degrees of innovation all influence the way post-project reviews need to be conducted But before we propose a framework to develop better (rather than best) review practices, we must understand some of the underlying resistances to reflective learning for future projects

5 Resistance to learning from post-project

reviews

If post-project reviews were simple, few companies

would pass up the chance for additional systematic

learning However, a number of factors both at the

individual and the group level prevent the free flow of

information and the retention of critical project

knowledge for future use

Based on research done by Ursula Koners at

Cranfield University (see also Koners and Zedtwitz,

2001, p 128 – 131), we identified eight major problem

factors, which we grouped in four areas as barriers to

learning from post-project reviews (Figure 4):

1 Psychological barriers;

2 Team-based shortcomings;

3 Epistemological constraints;

4 Managerial problems

5.1 Psychological barriers

There is no organizational or team learning without

individuals creating and sharing information All

hu-man beings are limited in their capacity to learn based

on their experiences Our minds are programmed

through the process of evolution to a certain balance

of reflection and action Given the complexities and

challenges of our lives, particularly under high pressures

in modern society, it seems understandable that our

commitment to reflection on the past – particularly with

no immediate benefits to oneself – is rather limited This

mechanism is effective at the conscious as well as the

subconscious levels Here we focus on two impediments

pertaining to post-project review management:

1 The disinclination of team members to objectively

reflect upon past actions and their consequences,

particularly their own actions;

2 The bias to remember easy-to-categorize incidents

and the tendency to repress ambivalent experiences

5.1.1 Inability to reflect. The business life of today is generally not based on reflections of the past One popular ‘excuse’ is based on the claim that past models and experiences do not apply if circumstances change (Kransdorff, 1996) Most often it is found that managers have little awareness of past actions and rationales According to Busby (1999) the idea that

‘experience is a teacher in its own right’ is very dominant In other words, one should not be surprised that people find it difficult to communicate about the past since the past is not the objective of management Another reason for this inability to reflect is based

on the fact that traditional approaches to project management do not treat learning and reflection as central (Ayas, 1997); and consequently these activities are not considered as a vital part of the project management task In R&D, blue-sky researchers are the ones most likely to engage in learning and reflection for its own sake, but they are in the minority compared to product-oriented development engineers and scientists working on problems in applied research Once learning and reflection become a means and not

an end of a process, they are more easily forgotten

5.1.2 Memory bias and ambivalent experiences. Be-sides the selectivity of reflection, man is confronted with the selectivity of memory Psychology and psychoanalysis have made great advances in our understanding of why the human mind remembers certain things and forgets others Several layers of memory have been identified Repression is commonly considered to apply to unpleasant experiences, forcing

us to forget the actual incident that led to the experience but leaving us with a mental imprint influencing future behaviour

What is the role of repression of group experiences? Unpleasant experiences are not really forgotten and are often very present However, it is the ambivalent experiences that – both at the individual and the group level – become those memories that pose the greatest

Team-based

• Reluctance to blame

• Poor internal communication

Difficult to generalize Tacitness of process knowledge

Epistemological

•

•

• Time constraints

• Bureaucratic overhead

Managerial

Inability to reflect Memory bias

Psychological

Barriers to Learning from Post Project Reviews

•

•

Figure 4 Four major barriers to learning from post-project reviews.

threat to the comfort of people, and are hence

repressed unless they are (at least at the group level)

raised and discussed Since our world is not

black-and-white, most of our experiences are ambivalent If we

were not able to prioritize which experiences to discuss

and which to repress, we would spend all day debating

and interpreting our experiences and possibly stalling

any progress whatsoever Repression is a natural way

of limiting everyday complexity

Repression or selective memory in projects can

override potentially important and valuable information

for future use Future project teams, creating a new team

spirit and a new moral understanding, may understand

this experience in a completely different perspective It is

therefore important to analyse what repressions may

have taken place at the group level to carry over any

underlying experiences from a previous project

5.2 Team-based shortcomings

While team members are put together to work

hand-in-hand towards the same common goal, they are often

poorly suited to stand up face-to-face and give each

other a piece of their mind Frank feedback may hurt

and potentially affects the relationship between

in-dividuals for years: many people choose not to risk

their social networks for the sake of project quality

At the same time, much neutral information is not

passed on because its importance is undervalued or its

perceived validity limited:

1 The reluctance of individual team members to

blame other team members or their direct superiors,

and to take accountability;

2 Poor team-internal communication and the failure

to share different interpretations of common

experiences

5.2.1 Reluctance to blame. One explanation why

reviewing past projects does not always lead to a

successful learning experience is that it necessarily

involves looking back at problems and critical events

in the past Although Gulliver (1987) established in his

in-depth case study of the BP organization that people

genuinely want to help the company grow more

profitable by reviewing past behaviours, reviews often

suffer from the reluctance to allocate blame and

criticism since they might uncover cynical or

embar-rassing events

Unfortunately, this reluctance can be stronger than the

realization that the organization has a potential to learn

constructively from a project team’s experience

(Krans-dorff, 1996) In extreme cases, project members deflect

the blame away from themselves, citing unclear goals,

insensitive and unfair leaders, and ignorant clients

(Barry, 1991) By the same token, individuals are

unwilling to take accountability for failure, either because

failure may have a negative impact on their career record,

or because they are embarrassed to acknowledge failure

in front of others Research in this area is difficult to generalize beyond the particular case, and the degrees of reluctance and accountability probably depend on specific company culture and incentive systems

5.2.2 Poor team-internal communication. Excellent communication has always been held as critical for

effective R&D teams Tushman (1979), Allen et al.

(1980), and Katz and Allen (1982) described how team-internal communication affects the productivity of the

R&D function Imai et al (1985) or Dimanescu and

Dwenger (1996) are mentioned here, in lieu of many others who underlined the importance of cross-functional communication for new product develop-ment teams While this is a significant step forward, in practice internal communication across functions and groups is hindered by team-internal regrouping: Some people get along better with each other, being able and willing to share information more easily with some than with others In pathological situations, team-internal factions intentionally hide critical information

in order to gain unfair advantage in performance recognition or promotions Teams separated by physical distances have usually little opportunity to meet in person; they are hence restricted to electronic

or written communication Team members with different technical, functional, or cultural backgrounds

do not share the same vocabulary or referential context, which leads to misunderstanding and reduced knowledge exchange While there are great advantages associated with the inclusion of people from different backgrounds, locations, and cultures, team-internal communication is certainly not enhanced

5.3 Epistemological barriers

Even if human beings had the time and interest to fully devote their attention to the reflection and analysis of what happens around them, they would still find it difficult to grasp the most important issues and draw important conclusions for future behaviour all by themselves The human intellect is limited At the same time, certain kinds of experiences are inherently hard

to express and hence not easily shared with colleagues Group-based discussion on commonly shared experi-ence may thus be the only way to bring forth distributed key lessons for future project teams:

1 The difficulty of abstraction from single occur-rences and events to formulate more general recommendations;

2 The inherent difficulty of articulation of tacit knowledge and the precise formulation of ambig-uous assumptions and beliefs

5.3.1 Difficulties in generalizing from specific projects.

Abstract knowledge may be the basis for competitive

advantage if it is well embodied in an organization

(Hamel and Prahalad, 1994) Generalizing from

context specific project experiences is one of the main

hurdles and difficulties for post-project reviews in

general The human mind is not made to abstract

experiences to a general level so that they can be

applied to a wide range of future projects

Further-more, project results (no matter if they are positive or

negative) are often naively extrapolated in a simple

linear fashion The reality, however, is much more

complex, so that the outcome of a project depends on a

whole variety of interlinked variables which again are

very difficult to generalize

5.3.2 Tacitness of process knowledge. The nature of

process-related knowledge, experiences and insights

implies that they cannot be shared in the same way as

information stored in reports, databases or prototypes

Polanyi (1966) refers to this phenomenon as ‘tacit

knowledge’ and describes the underlying problem of

organizations: ‘We know more than we can tell’

Durrance (1998) draws from Polanyi, Takeuchi and

Senge’s work and claims that especially Western

cultures still prefer explicit knowledge, which is

quantifiable and definable Despite increasing

aware-ness of the difference between tacit and explicit

knowledge, companies are still struggling to convert

tacit into explicit knowledge so that it can be shared by

the entire organization Senge (1990) suggests

reflec-tion or careful observareflec-tion – a task mainly found in

post-project reviews

5.4 Managerial constraints

R&D is, like any other corporate function, under

constant performance pressure This pressure leaves

little time to step back and reassess a project that has

occurred in the past Furthermore, researchers are

known to shun guidelines and project management

rules imposed by a central department These

require-ments are often interpreted as unnecessary and

obstructive to the actual R&D work Hence we have:

1 Lack of time to deal with the past in a business that

typically looks three to five years ahead;

2 Reluctance to comply with bureaucratic and

admin-istrative requirements in association with following

up on post-project reviews

5.4.1 Time constraints. The most often stated reason

why post-project reviews are not conducted is lack of

time People are unlikely to devote time and effort to

yesterday’s problems since natural incentives favour

moving ahead to the next problem instead of spending

valuable time on reviewing a just completed project

This has been verified by Kotnour (1999) who – based

on results from his questionnaire survey of 43

experienced project managers – states that ‘most

project managers viewed producing lessons learned as

a valuable and important exercise However, they felt they did not have enough time to complete a formal lessons-learned process’

Particularly in an economy more and more deter-mined by Internet-speed competition, time will be increasingly short An R&D manager of a large IT company declared that ‘if you have the time to think, you’re doing something wrong’ When time is a critical resource, retrospection and contemplation are left to others

5.4.2 Bureaucratic overhead. Corporate control sys-tems tend to concentrate on budget, time and output Not surprisingly, if post-project reviews are conducted

at all, a substantial part of each review is devoted to complying with these control demands Particularly when post-project reviews are considered as additional bureaucratic chores imposed by a central project management department, project teams rarely have the necessary enthusiasm to engage in additional review work Moreover, the information requested

in a post-project review is often incompatible with information solicited during the execution of the project, and hence undermines the confidence of the team in the entire control system

As a consequence, the purpose of a post-project review is often reduced to ensure that the project complied with all bureaucratic procedures, and the important inter-project learning function is neglected

If learning is on the review agenda, it is often for the benefit of the team members only Kotnour (1999) discovered that the learning goals of project managers focus on costs, scheduling, performance and customer satisfaction – in line with what most corporate incentive systems favour In combination with time constraints, the disinclination of project teams to comply with bureaucratic procedures tends to keep the effort spent on post-project reviews to a mini-mum

6 A capability maturity-based model for organizational learning through post-project reviews in R&D

There is a growing body of evidence that projects may prove immensely beneficial to the long-term success of companies when these companies systematically in-corporate reflective practices into their project man-agement processes (DeFillippi, 2001, p 6) In the two previous chapters we have outlined why post-project reviews have been difficult to conduct, and how selected companies have found formalized ways and means to gain insight from R&D projects despite these obstacles

Overall, we have not been able to find a single best practice in post-project reviews in R&D Individual

approaches tend to be company-specific and optimized

to problems perceived to be most urgent by each

organization

A number of methodologies for structured learning

from projects have been formulated to provide R&D

managers with practical guidelines (e.g Collier,

De-Marco and Fearey, 1996; Barker and Neailey, 1999;

Kearth, 2000) If these methodologies were embraced

by R&D organizations, post-project reviews would be

conducted with greater consistency and greater

applic-ability of their outcomes to other R&D projects

However, the introduction of a new reflective

process, which is typically not perceived to add value

to current business activities, is resisted unless the

organization has reached a level of maturity that is

receptive to company-wide knowledge sharing

In the field of software engineering and

develop-ment, a capability maturity model (CMM) has been

proposed by the Software Engineering Institute to

describe the capabilities of software development

organizations and to provide guidance on how to

establish and improve software development processes

(see Carnegie-Mellon University, 1995) Analogous to

CMM, we propose a five-level capability maturity

model for post-project reviews Our post-project

review maturity model is designed as a framework

allowing the progression from an immature

unrepea-table review process to a mature, well-managed review

process (see Figure 5)

The maturity model is organized into five levels:

Initial; Repeatable; Defined; Managed; Optimizing

Each level is described in terms of key processes that

contribute to the degree of implementation and

institutionalization of the review processes in place

6.1 Level 1: Initial

Organizations at this level of process maturity

char-acterize post-project reviews as ad hoc or even chaotic

If a review process has been defined, then its success depends heavily on the skills and talent of individuals who conduct it Reviews are seldom planned but triggered rather in reaction to a major project-related event, such as a complete failure or a particularly successful project Because the review process is poorly defined and its execution not standardized, its outcome

is unpredictable and incomparable to other review results

Most organizations still appear to be at this level Our survey indicated that one in eight companies never conducts post-project reviews, and that half of the remaining companies use arbitrary or random selection criteria to review projects More than half of all respondents stated that their companies have no post-project review guidelines in place, and about a quarter responded that the review quality was based mostly on the capabilities of the team members

6.2 Level 2: Repeatable

The organization has established post-project review guidelines and sound review practices: the review process is repeatable and comparable to previous reviews Managing and planning of new reviews is based on experience with similar reviews, although specific review practices may differ With principal review policies in place, major disasters in review sessions or in review outcomes may be avoided

A simple example of a repeatable review practice is a post-completion report, which is usually based on a report template to ensure inclusion of relevant project improvement areas Such written reports, however, focus mostly on quantifiable and explicit data, such as costs, scheduling and technical=design deviations Nevertheless they provide a good means for transfer-able recommendations for corrective action, if these suggestions are made available at the group or management level

Maturity of Post-Project Review Processes

Optimizing

Managed

Defined

Repeatable

Initial

• Ad hoc PPR

• Reaction-driven reviews

• Based on capabilities of project individuals 1

2

3

5

• Establishment of PPR policies

• Introduction of sound review practices

• Based on experience with similar reviews

• PPR process standardized

• Establishment of sound and consistent review criteria

• PPR responsibility assigned to a unit

• PPR goals quantified and measurable

• Corrective action can be taken

• Quality of transferable knowledge predictable

Post-Project Review Capabilities

4

• Organization-wide PPR

• Consistent inter-project learning

• Proactive review of PPR pro-cesses

Figure 5 A capability maturity model of post-project review (PPR) processes.