IMPaKT: A framework for linking knowledge management to business performance

A number of organisations have recognised the importance of managing their organisation knowledge in a more structured manner. However, the question arises as to how to evaluate the benefits of a Knowledge Management (KM) strategy and its associated initiatives on the performance of the organisation. This paper presents a framework for the assessment of the likely impact of KM and discusses findings from an evaluation workshop held to critique the framework.

IMPaKT: A Framework for Linking Knowledge Management

to Business Performance

Patricia M Carrillo, Herbert S Robinson, Chimay J Anumba and Ahmed M Al-Ghassani,

Department of Civil and Building Engineering, Loughborough University, UK

P.M.Carrillo@lboro.ac.uk

Abstract: A number of organisations have recognised the importance of managing their organisation knowledge

in a more structured manner However, the question arises as to how to evaluate the benefits of a Knowledge Management (KM) strategy and its associated initiatives on the performance of the organisation This paper presents a framework for the assessment of the likely impact of KM and discusses findings from an evaluation workshop held to critique the framework

Keywords: Knowledge management, business performance, evaluation

1 Introduction

Business organisations are becoming

increasingly aware of the need for innovative

approaches to responding more effectively to

clients' demands and changes in the market

place Knowledge Management (KM) is central

to this and is increasingly recognised as an

integral part of an organisation's strategy to

improve business performance A key issue in

the implementation of KM strategies is the

evaluation of the likely impact The difficulty for

many organisations stems from the fact that

the implementation of KM initiatives has often

been ad hoc, without a coherent framework for

performance evaluation A Knowledge

Management (KM) initiative can be developed

to improve the performance of a simple task

and its impact easily evaluated However, as

we move away from simple tasks to

organisation-wide systemic problems, KM

initiatives become more complex and

intertwined This makes it difficult to evaluate

the impact of these initiatives on business

performance There is therefore a need for a

performance-based approach to KM that

explicitly shows the interactions between KM

initiatives and a set of measures for evaluating

their effectiveness and efficiency The

Knowledge Management for Improved

Business Performance (KnowBiz) project is a

three-year research project, sponsored by the

EPSRC and industrial collaborators, aimed at

investigating the relationship between KM and

business performance As part of the project,

an initial concept of an KM framework was

developed (Robinson et al, 2001) This

concept has now evolved into an operational

framework refined through a follow-up

technical workshop with the project's industrial

collaborators

This paper presents the development of a

framework for Improving Management Performance through Knowledge Transformation (IMPaKT) and discusses

findings from the application of the framework based on an evaluation workshop held with industrial partners Two distinct types of performance measures are identified to evaluate KM initiatives - an effectiveness measure, which relates to the degree of realisation of the strategic objectives and an efficiency measure reflecting the nature of the process used to implement KM initiatives

2 Linking KM strategy to business performance

Knowledge within the business context can fall within the spectrum of tacit (implicit) knowledge and explicit (codified) knowledge Tacit knowledge is stored in people's heads and is difficult to share Explicit knowledge is captured or stored in an organisation’s manuals, procedures, databases, and is therefore, more easily shared with other people or parts of an organisation Organisational knowledge is a mixture of explicit and tacit knowledge and the role of KM

is to unlock and leverage the different types of knowledge so that it becomes available as an organisational asset However, a key issue in

KM is the evaluation of the likely benefits KM strategies are more likely to be successfully implemented if a performance-based approach

is adopted that explicitly shows the interactions between KM initiatives and a set of performance measures for evaluating their effectiveness and efficiency

Carrillo et al (2000) suggested that KM could

be integrated into key performance indicators

(KPIs), and other performance measurement

approaches There is evidence that some

organisations are now implementing various

types of business performance measurement

models such as the Balanced Scorecard

(Kaplan and Norton, 1996) and the Excellence

Model (EFQM, 1999) A recent survey of

construction organisations shows that about

40% already have a KM strategy and another

41% plan to have a strategy within a year

(Carrillo et al, 2003) About 80% also

perceived KM as having the potential to

provide benefits to their organisations, and

some have already appointed a senior person

or group of people to implement their KM

strategy

However, a major problem in KM is evaluating

its likely impact on business performance

Performance is therefore a key issue and

performance measurement models provide a

basis for developing a structured approach to

KM Business performance measurement

models are being used increasingly to

encourage organisations to focus on

measuring a wider range of business

performance issues relating to processes,

people and product A recent survey

conducted by the KnowBiz research team

shows that over 35% of construction

organisations are using either the Balanced

Scorecard (BSC) or the Excellence Model

(EM) and about a quarter (26.4%) are using

other measurement systems, mainly the Egan

KPIs or bespoke models Over 90% of

organisations using the BSC or the EM also

have or plan to have a KM strategy in the short

term (within a year) However, a significant

factor identified in the case studies is the lack

of co-ordination between business

improvement and KM (Robinson et al, 2003)

Linking KM to business performance could

make a strong business case in convincing

senior management about the need to adopt a

KM strategy, particularly when the ability to

demonstrate benefits of KM is becoming more

important in the competition for funding

3 Research methodology

This framework is a deliverable for an ongoing

UK government EPSRC-sponsored research

project supported by a number of industrial

collaborators A variety of research methods

were used including literature review,

questionnaire survey, industry case studies

and semi-structured interviews for the

development of the framework A literature

review identified the key issues in knowledge management and performance measurement

A questionnaire survey and case studies with industrial collaborators were undertaken to identify practices, motivation, barriers and enablers in the application of KM and business performance measurement The initial concept

of the framework was developed based on the findings from the literature review, questionnaire survey and case studies The framework was further developed, reviewed and refined through a follow-up technical workshop and the applicability of the KM framework was validated through pilot studies and an evaluation workshop with industrial collaborators

4 The IMPaKT Framework

A knowledge management strategy should not only facilitate the transformation of the various types of knowledge within an organisation but should provide an evaluation mechanism to measure the effectiveness and efficiency of any strategy A three-stage framework for Improving Management Performance through Knowledge Transformation (IMPaKT) has been developed to link KM to performance measurement (see Figure 1)

The framework recognises that to be able to

assess the impact of knowledge management,

KM initiatives have to be aligned to an organisation's strategic objectives Key issues

at each stage are further explored through Templates (illustrated in the subsequent sections) supported by detailed guidelines For each stage, there are steps or thought processes required to structure business problems

4.1 Stage 1 - Developing a Business Improvement Strategy

The aim of Stage 1 is to provide a structure for formulating a strategic business plan by identifying the external (business) drivers, defining strategic objectives or goals, identifying critical success factors, and developing measures for monitoring performance improvement The outcome of Stage 1 is a business improvement plan with performance targets and measurable indicators to assess performance Table 1 shows a condensed version of the template for developing a business improvement strategy

Definition of Business Goals

Identify business aim/strategic objectivess and

develop measures

Assess Implications for the Organisational

Knowledge Base

Identify existing knowledge from a process, people and product perspective

Develop KM initiatives and align to improvement measures

Assess results on key performance measures

Assessment of the Impact on Management Performance

Determine knowledge gap from a process, people and product perspective

Assess results from process, people and product

measures

Business Strategy

Evaluation of Know

Figure 1: IMPaKT Framework

Table 1: Developing a business improvement

strategy

STAGE ONE STEPS

1.1 Choose a business problem with a knowledge

dimension This is achieved by asking whether

there is any knowledge the organisation ought

to have to improve the situation or solve the

problem

1.2 Place the business problem in a strategic

context by relating it to your external business

drivers, strategic objectives and critical

success factors

1.3 Select an appropriate set of measures to

monitor progress towards achieving your

strategic objectives, and identify the business

processes they relate to

1.4 Identify previous, current, target and

benchmark scores for various performance

measures and establish the performance gaps

Steps 1.1 to 1.4 are supported by detailed

guides such as a glossary to facilitate the

understanding of key terms of the Framework,

a sample of performance measures with their

metric definitions and examples of possible

benefits arising from improvement in key performance measures

The first step in Stage 1 is to choose a business problem and to analyse the knowledge dimension of the problem KM problems are business problems that are associated with, related to, or caused by a dysfunction in the processes of obtaining/capturing, locating/accessing, sharing or the application of knowledge The next step involves putting the business problem in its strategic context by identifying the organisation's external and internal forces For example, the external business drivers

(external forces) are the key issues influencing

an organisation to achieve or cope with radical changes in the business environment These issues could, for example, be technological (e.g the need for innovation), market or structural factors (e.g expansion/ downsizing), etc The selection of measures for performance monitoring is also a crucial aspect of Stage 1 The improvement measures are driven by the firm’s strategy and will therefore reflect the strategic objectives of the organisation

4.2 Stage 2 - Developing a KM Strategy

The aim of Stage 2 is to clarify the whether the

business problem has a knowledge dimension

and to develop specific KM initiatives to

address the business problem The outcome of

Stage 2 is a KM strategic plan with a set of

initiatives and implementation tools to support

business improvement Table 2 shows a

condensed version of the steps involved in

identifying the knowledge implications of a

business strategy and for developing

knowledge management initiatives for

business improvement

Table 2: Identifying KM problems and

initiatives

STAGE TWO STEPS

2.1 Clarify the knowledge dimension of your

business problem by identifying the KM

process(es) involved

2.2 Develop specific KM initiatives to address the

business problem

2.3 Select possible tools to support the KM

process(es) identified in the context of your

business problem

2.4 Identify possible relationships between KM

initiatives and performance measures and show

how they relate to the strategic objectives (the

Cause-and-Effect Map)

2.5 Prepare an Action Plan and identify change

management and resources required

Steps 2.1 to 2.5 are also supported by detailed

guides such as a questionnaire to identify the

KM sub-processes involved, a matrix for the

selection of the most appropriate KM tools and

a checklist to identify possible barriers and

facilitators prior to implementation

Identifying the KM sub-processes associated

with the business problem is the first step in

clarifying a KM problem Knowledge

management consists of distinct but

interrelated processes that are not linear but

can be cyclical and iterative Examples of KM

processes are generate, propagate, transfer,

locate and access, maintain and modify

(Anumba et al, 2001) Others have used

different classifications of the KM life cycle e.g

generate, codify and transfer (Ruggles, 1997);

creation, location, capture, share and use of

knowledge (Tiwana, 2000); discovery and

capturing; organisation and storage;

distribution and sharing; creation and leverage,

retirement and archiving (Robinson et al,

2001) The next step is to identify the KM initiatives required KM initiatives are systematic goal-directed efforts for addressing

a KM problem in order to achieve business improvement For example, a KM problem associated with client satisfaction could be improved by utilising more effectively information that already exists within the organisation about clients It may also include other initiatives such as setting-up a post-tender forum with clients or project closure meetings to share information A set of KM initiatives identified should align with the KM strategy However, KM tools are required for the implementation of initiatives A range of tools can be selected including both IT-based (hardware and software) and non-IT-based

systems (Robinson et al, 2001) The hardware

tools comprise the platform required to support

an organisation's knowledge management strategy The software tools vary from simple databases and groupware to intelligent decision support systems such as expert systems and business intelligence tools The non-IT-based systems will focus on such tools

as informal dialogue, mentoring, communities

of practice, formal network meetings and research collaboration forum to harvest new ideas It is also vital to assess an organisation's readiness before a KM strategy

is implemented An appropriate knowledge management context should be developed and

its readiness assessed against the reform needed, resources required and results

monitoring mechanism in place prior to the implementation of KM KM is useful but there

is a need to have the necessary reform in place, have adequate resources and to be able

to demonstrate the benefits through a result-oriented approach

4.3 Stage 3 - Developing a KM Evaluation Strategy and an Implementation Plan

The aim of Stage 3 is, therefore, to provide a structured approach for evaluating the impact

of KM initiatives on business performance The outcome of Stages 1 and 2 of the IMPaKT Framework is a business improvement strategy underpinned by KM The outcome of stage 3 is a KM strategy and an implementation plan with priorities and an appreciation of likely impact of various KM initiatives on business performance or key performance measures This stage is the most challenging, as the justification of KM initiatives depends on the expected benefits (e.g performance improvement) Two distinct types

of performance measures are identified; measures of effectiveness and measures of

efficiency Measures of Effectiveness are

outcome-based measures relating to the

degree to which target performance measures

are achieved but does not take account of the

cost of implementation Measures of Efficiency

are process -based measures relating to the

nature of the KM system used in

implementation and are a ratio of expected

benefit or utility per unit of KM investment It is,

however, recognised that organisations at the

embryonic stage of KM may not have a

full-scale measurement framework but may need

to start with basic qualitative performance

measures to demonstrate the benefits (APQC,

2001) More concrete measures may have to

be developed as an organisation progresses to

a transformation stage where KM

implementation is mature and well

co-ordinated Table 3 is a condensed version of

the steps for KM Evaluation

Steps 3.1 to 3.5 are supported by various

guides developed such as a KM cost and

benefit component checklists, and a KM guide

to evaluation/assessment techniques

4.3.1 Measures of Effectiveness

KM strategies need to be aligned to strategic

objectives These links will enable an

assessment of the effectiveness of KM in

terms of the degree to which strategic

objectives are realised The Cause-and-Effect Map (Figure 2), showing possible relationships between KM initiatives, performance measures and the strategic objectives they relate to, forms the basis for determining the contribution

of each KM initiative to the performance measures

Table 3: Developing KM evaluation strategy

and an implementation plan

STAGE THREE STEPS

3.1 Use the Cause-and-Effect Map developed in 2.4 to assess the likely contribution of the KM initiatives to the performance measures 3.2 Assess the probability of success of your KM initiative in improving your performance measures (the effectiveness measure)

3.3 Identify the cost components for implementing each KM initiative and the possible benefits (the efficiency measure)

3.4 Choose an appropriate method to assess (ex-ante) the impact of each KM initiative on your business performance

3.5 Prioritise your KM initiatives based on the two measures of performance

To handover our projects with zero defect

(A) Number of defects at handover (1) Zero Defects

Use existing knowledge to avoid defects

To increase profitability (B) Customer satisfaction

(2) Best Practice Construction Materials/Methods

To increase customer satisfaction (C) Profit

To reduce material waste from site (D) Defects inproducts

(3) Best Practice Legal Issues

To reduce corporate risk (E) Number of

cases/arbitration/disputes

Figure 2: Cause-and-Effect Map

The first step in the assessment process is to

identify the cause-effect relationships The

second step is to evaluate the impact

quantitatively or qualitatively A KM initiative

may have varying impact on performance

measures i.e the impact of an initiative may be

greater in some than others The impacts or contributions are determined using direct weighting techniques such as ranking and rating

4.3.2 Measures of Efficiency

Determining the contribution of a KM initiative

in improving performance score is not

sufficient on its own to make strategic

decisions, particularly where there are other

competing initiatives and resource constraints

The efficiency of the process used to

implement the KM initiatives should also be

evaluated This will also help uncover the real

costs of KM initiatives Information technology

(IT) tools and technologies form one third of

the time, effort, and money that is required to

develop and use a KM system (Tiwana, 2000)

'Information technology costs is the most

obvious, but the bigger, and often, hidden

costs are associated with people' (APQC,

1997) and the related time/cost associated

with setting up human interactive systems and

process reengineering or adjustments to core

and supporting business processes There are

various inputs or cost components of a KM

initiative as outlined below:

KM team component represents the cost

associated with both the core (e.g knowledge

managers) and support team (e.g IT

personnel) required for implementing

knowledge management initiatives

KM infrastructure component represents the

costs associated with providing the setting up

IT and non-IT systems to provide knowledge

creation and sharing capability

There are different types of cost associated with KM such as staff costs, organisational or (re)organisational costs, hardware and software costs As different KM hardware and software tools are used for the implementation

of KM initiatives, consideration should be given not only to their appropriateness in terms of functionality (i.e ease of use, integration, focus and maturity) but, also cost Costs could be direct or indirect, one-off/lump sum (e.g purchase and initial installation cost of hardware and software, consultant's fee etc.)

or recurrent/periodic (e.g hardware/ software maintenance costs, staff costs etc) or occasional costs (e.g hardware upgrades, support staff costs etc)

There are also different types of benefits to be expected such as:

People e.g direct labour saving, reduction in

staff turnover;

Processes e.g direct cost savings, increased

productivity;

Products e.g direct cost savings, increased

sales; and

Other e.g repeat customers, new customers

4.3.3 Evaluation Methods

The aim of evaluation is to identify the input i.e the nature of KM initiatives and their output i.e the consequences (both positive and negative)

in terms of changes in performance or contribution to business benefits or losses Table 4 shows the various evaluation techniques included in the framework

Table 4: Evaluation Techniques

Cost minimisation analysis: This involves a simple cost

comparison of KM initiatives as it is assumed that the consequences

(outputs) are identical or differences between the outputs are

insignificant It does not therefore take account of the monetary

value of the consequences (outputs)

When output of KM initiatives are identical in whatever unit of measurement is used

Cost effectiveness analysis: This involves the comparison of

KM initiatives where the consequences (output) are measured using

the same natural or physical units The assumption is that the output

is worth having and the only question is the cost of the input to

determine the most cost-effective solution

When output of KM initiatives are measured in the same natural or physical units e.g number of accidents prevented, reduction in absenteeism or waste, training man-hours, etc

Cost utility analysis: This involves a comparison of KM

initiatives (inputs) which are measured in monetary units with the

consequences (outputs) measured using utility or a preference

scale Utility refers to the value or worth of a specific level of

improvement measured by the preferences of individuals, teams or

organisation with respect to a particular outcome

When a significant component of the output cannot

be easily measured, quantified or expressed in monetary units

Useful in making internal comparison between divisions when there is, for example, a decision to introduce a pilot project within an organisation

Cost benefit analysis: This approach provides a comparison of

the value of input resources used up by the KM initiative compared

to the value of the output resources the KM initiative might save or

create Consequences of KM initiatives are measured in monetary

terms so as to make them commensurate with the costs

When a significant component of the output can be

easily measured, quantified or expressed in monetary units

Useful in determining return on investment (ROI), Internal Rate of Return (IRR), Net Present Value (NPV) or Payback Period of KM investments

A number of these techniques can therefore be

recommended in the framework depending on

(a) the existing techniques used by the

organisation and (b) the level of detail required

in evaluating the KM initiative

5 Framework evaluation

As part of the research programme, two

one-day workshops were planned for the

development of the IMPaKT framework The

first workshop was conducted at the end of the

first year of the research project to assess and

refine the initial concept and to provide ideas

for the detailed development of the framework

A second workshop was held at the end of the

second year This was an evaluation workshop

aimed at assessing the robustness of the

framework that has evolved Participants

familiar with KM and business improvement

issues were invited to the workshop following a

consultation with the project's industrial

collaborators There were thirteen participants

including a director of technical services,

senior business improvement and knowledge

managers, account manager, business

systems and IT managers, both from the

construction and manufacturing industries

Over three-quarters of the participants has a

high level of awareness on KM (76.9%) and

business improvement (84.6%) issues

5.1 Evaluation Methodology

The evaluation workshop started with a

presentation of the outline of the framework, a

workshop brief and workshop manual

consisting of tasks list with supporting

diagrams and guidelines Participants were

organised into four teams of three to four

people, with research team members acting as

workshop facilitators The workshop was divided into two sessions Session 1 was based on the use of Template 1 to develop a business improvement strategy with a KM response, and covered Stages 1 and 2 of the framework Session 2 was based on using Template 2 to develop a KM evaluation strategy and an implementation plan and covered Stage 3 of the framework

Each group was asked to choose a business problem with a knowledge dimension and to structure the problem using a template provided by the IMPaKT framework Each team went through the evaluation exercise using different examples of business problems

At the end of the workshop, a group discussion was held to identify issues regarding the use of the templates and an evaluation questionnaire was given to participants to complete The evaluation questionnaire consisted of statements reflecting key aspects of the framework's capabilities Participants were asked to rate each statement with respect to the degree to which they agreed or disagreed with it and to provide suggestions for improving the framework

5.1.1 Findings and Feedback

The results based on the analysis of the evaluation questionnaires completed by the workshop participants are shown in Figures 3

to 6 The questionnaire used a rating scale from 1 (strongly disagree) to 5(strongly agree)

Linking KM to business strategy Monitoring business strategy Monitoring KM strategy Evaluation component Glossary of terms

Average Ratings

5.0 4.0

3.0 2.0

1.0

Figure 3: Average ratings of key components of overall framework

The overall approach of the IMPaKT

framework was rated well in terms of the

framework's capabilities to link business

improvement and knowledge management,

and for developing and monitoring business improvement and knowledge management strategies (see Figure 3) The glossary of terms accompanying the framework was

considered helpful in the evaluation process

However, there were some concerns about the

terminology It was suggested that simplifying

or refining some definitions could help as some

of the terms used could mean different things

to different people or organisations The

evaluation component was also found to be

useful, although, it was rated slightly lower

than the other aspects of the framework It was

noted that the development of the framework represents a significant attempt to conduct a structured approach to assessing the benefits

of KM to be able to convince senior managers, Detailed findings of the components of the framework are presented in subsequent sections Figure 4 is a summary of the average ratings for Stage 1 of the framework

S t r a t e g i c c o n t e x t

L i n k s t r a t e g i c

o b j e c t i v e s t o

p e r f o r m a n c e

m e a s u r e s

L i n k p e r f o r m a n c e

m e a s u r e s t o

b u s i n e s s p r o c e s s e s

T y p e s o f

p e r f o r m a n c e

s c o r e s

A v e r a g e R a t i n g s

5 0

4 0

3 0

2 0

1 0

Figure 4: Average ratings of key components in Stage 1

All of the participants strongly agreed that the

framework allows an organisation to be able to

put its KM/business problems into a strategic

context The need to align the strategic

objectives of an organisation to performance

measures, and to be able to relate

performance measures to the business

processes they impact on, was also found to

be useful aspects of the framework, as the

ratings for both are high The performance

monitoring aspect encapsulating the different

types of performance scores (previous,

current, target and benchmark scores) were

also considered to be important, although the average rating of 4.00 is not as high as other aspects

Figure 5 is a summary of the average ratings for key aspects of Stage 2 of the framework The KM clarification process was found to be useful, so are the KM tools required in the implementation as part of a business improvement strategy

KM clarification process

KM tools

Level of organisational readiness Readiness audit checklist

Average Ratings

5.0 4.0

3.0 2.0

1.0

Figure 5: Average ratings of key components in Stage 2

Another key issue is the level of organisational

readiness to implement KM This is relevant

since, regardless of the enthusiasm and

resources directed towards improving KM,

these efforts may not be successful because

there may be fundamental technical and social

issues that need to be addressed For

example, installing a skills yellow page could

bring benefit but if employees are not willing to

provide updates of their experience then the

skills yellow pages will rapidly become

outdated Participants agreed that

organisational readiness is not only a very

significant factor to consider prior to

implementing a KM strategy, but the

accompanying checklist provided was useful in

identifying the barriers and facilitators to KM

Template 1 for stages 1 and 2 of the

framework was considered quite clear and

very useful in working through the issues or

problems selected One participant commented that it was 'useful thought process

to go through, well focussed and easy to use' Other participants noted that the template also provides a link with the external environment (external drivers) of an organisation and is a good template for a general business problem However, it was suggested that in dealing with some of the issues arising, it is important for senior management to be involved especially for key strategic issues Although the workshop was based on structuring hypothetical business problems, it was acknowledged that the framework could be more easily implemented in a company set up were real data is widely available Figure 6 is a summary of the ratings for key aspects of Stage 3 of the framework

L i n k s K M a n d

b u s i n e s s s t r a t e g y

K M i m p a c t o n b u s i n e s s

p e r f o r m a n c e

I d e n t i f i c a t i o n o f

c o s t s a n d b e n e f i t s

M e a s u r e o f

e f f e c t i v e n e s s

M e a s u r e o f

e f f i c i e n c y

E v a l u a t i o n G u i d e

A v e r a g e R a t i n g s

5 0

4 0

3 0

2 0

1 0

Figure 6: Average ratings of key components in Stage 3

Average ratings of slightly over 4 shows that

participants agreed that the framework does

facilitate an understanding of the links between

business improvement and KM, and also

provides a basis to be able to assess the

impact of KM It was also found to be helpful in

identifying the potential cost and benefits of

KM initiatives The measures of effectiveness

and efficiency are thought to be good

approaches for assessing the impact of KM on

business performance However, the ratings

for the evaluation guide to help identify the

most suitable techniques to assess the impact

of KM initiatives was only slightly above

average Some participants simply focused on

cost benefit analysis This is, in part, due to

their familiarity with, or popularity of cost

benefit analysis (CBA) compared to other

evaluation techniques

6 Discussion

Template 2 was relatively more difficult to use compared to Template 1, although it was acknowledged that the intention is clear Template 1 had undergone a number of iterations and pilot testing before being presented at the evaluation workshop However Template 2 was relatively new thus the workshop proved a valuable exercise in providing areas for improvement Some participants found it quite complex as evaluation is considered a difficult area However, the Cause-and-Effect Map was found to be very useful as the starting point for the evaluation It was also found to be useful to 'facilitate a structured way of thinking about a problem' and a 'good way to explain to management how everything is related - performance measures, initiatives and strategic objectives' There were suggestions

that the Cause-and-Effect Map could also be

used as a summary of the first session of the

workshop based on Template 1 The checklist

for identifying costs was found to be well laid

out and helpful, although it was noted that the

approach to costing might be different

depending on the cost models used in

individual organisations The benefit side was

more difficult to address, however, it was

agreed that the checklist does help in providing

some structure in the evaluation of benefits

Further refinement of the cost and benefit

evaluation checklists will continue But it was

suggested that putting more details into it

could probably make it more complicated and

possibly renders it less credible There was

also some concern about the repetition on

Template 2 However, due to the paper-based

version being used in the workshop it was felt

that certain aspects had to be repeated to

assist participants but this problem will be

overcome in an electronic/ automated version

of the framework, which would also enhance

delivery Other suggestions include clarifying

some of the headings to reflect the tasks list,

and simplifying Template 2 Issues were also

raised about how the framework could be

introduced to senior management and the level

of details of a KM implementation plan to be

provided to senior executives

All the recommendations made at the

evaluation workshop have been addressed in

the version of the template described in Tables

1, 2 and 3 The framework therefore provides a

solid basis for developing KM strategies that

are not only coherent but also consistent with

the overall strategic objectives of an

organisation The next stage of the research

involves refining the IMPaKT framework

further, and developing an automated version

and an IT architecture to facilitate the

implementation of KM strategies, and

integrating it into an existing KM tool called

CLEVER CLEVER helps organisations to

identify specific KM problems and guides users

through providing solutions to these problems

(Anumba et al., 2001)

7 Conclusions

The development of a three-stage Knowledge

Management framework (IMPaKT) to enable

the impact of KM on business performance

has been presented and discussed The

robustness of the framework was assessed

through a technical workshop with industrial

collaborators and a post-workshop evaluation

questionnaire The findings based on the

questionnaires analysed and the discussions

provide sufficient evidence of the potential of

IMPaKT as a structured framework for developing a KM evaluation strategy as part of business improvement The two measures of performance proposed to determine the effectiveness and efficiency of KM initiatives does not only ensure that appropriate initiatives are selected but enables the ranking

of KM initiatives in terms of level of impact on business performance and on specific performance measures The increasing number of organisations now implementing the Balanced Scorecard and the Excellence Model means that KM can be readily linked to performance measures The initial focus of the work reported and the evaluation is based on analysis from both construction and manufacturing organisations However, IMPaKT is a generic framework applicable to other sectors as well Further development and fine-tuning of the framework will continue as part of the on-going KnowBiz Research Project

References

APQC (American Productivity & Quality Center) (1997), Using Information Technology to Support Knowledge Management: Consortium Benchmarking Study (Final Report) Texas, USA

APQC (American Productivity & Quality Center) (2001) Measurement for Knowledge Management; http://www.apqc.org/free/articles/dispArticl e.cfm?ProductID=1307 on 6 March 2002 Anumba, CJ, Carrillo PM, Backhouse, CJ, Brookes, NJ and Sinclair MA (2001) Cross Sectoral Learning in the Virtual Environment – EPSRC Final Report, Loughborough University, UK

Carrillo, P.M, Anumba, C.J and Kamara, J.M (2000) Knowledge Management Strategy for Construction: Key IT and Contextual

Issues, Proceedings of CIT 2000,

Reykjavik, Iceland, 28-30 June, Gudnason, G (ed.), 155-165

Carrillo, P.M, Robinson, H.S, Al-Ghassani, A.M and Anumba, C.J (2002), Knowledge Management in Construction: Drivers,

Resources and Barriers (in press) EFQM (1999) Introducing Excellence,

European Foundation for Quality Management Brussels, Belgium

Kaplan, R.S and Norton, D P (1996) "The Balanced Scorecard - Measures that

Drive Performance", Harvard Business

Review, 70(1), 71-79

Robinson, H.S., Carrillo, P.M, Anumba, C.J and Al-Ghassani A.M (2001) Knowledge Management: Towards an Integrated Strategy for Construction Project