Show me what you do!
So far we have looked at some of the fundamentals that make up a management system and the basic understanding that an auditor needs to have in order to carry out a process management audit. We now turn our attention to the detail of how you should actually conduct an audit starting with the tools and techniques that you should adopt.
For years, auditor training has had a constant theme to it with one
message in particular being driven home time and again: ‘Show me the evidence!’
Above all else auditors have been trained to assess what an organization does against what it said it does, basing any decision as to how well they did it on the documented evidence they have been shown.
This technique of auditing is only relevant for assessing process management when compliance auditing to a specifc regulatory standard is required, such as those used in the medical or pharmaceutical industries, or against a standard such as ISO 9001:2008. This style of auditing may then be relevant to check that specifc detailed requirements are being met and effectively applied.
For the remainder of this book, the focus will be on auditing the effectiveness of process management, also required by ISO 9001:2008. This requires different tools and techniques to those required for both system and compliance auditing, and we need to recognize these differences.
22
Process Management Auditing for ISO 9001:2008
Auditor tools
There are basically two tools that should be used in both preparing for and carrying out a process management audit (see Figure 4.1, Figure 4.2 and
Table 4.1). Neither of them is complicated and in fact they are just plain common sense. Both, however, require the auditor to understand how a business works through its processes in order to use them effectively. This is one of the key competences of a successful process management auditor.
Once you understand them, they are so powerful that you can apply them to any process within any business, regardless of industry sector.
Improve
Monitor performance
Purpose of the process
Key performance process measures
Process objectives and targets
The process itself
Figure 4.1 Auditor tool 1
In process management auditing you are testing every one of the boxes in each process you audit at every level within each process, i.e. you go round this cycle with everyone you interview. The questions you use to test each one of the boxes will be phrased slightly differently and will be in a manner suitable to the person being interviewed, but nonetheless they will follow the same cycle. This aspect is critical for successful auditing. It is no good asking a member of staff a question that they do not understand, or using ‘management style’ or ‘standard’
language that they cannot relate to what they do. For example asking someone what ‘resources’ they use may not be understood, asking what ‘equipment’ they use might be. There is no right or wrong, but the language you use is important
Auditing tools and techniques
23
and needs to be based on the needs of the auditee not the auditor. It needs to be in the language used by the people within the organization itself.
Table 4.1 Defnitions of the elements of auditor tool 1
Purpose of the process Why the process exists – supplier inputs and customer outputs Process objectives
and targets
Specifcally the objectives and targets for this process that must relate to the overall business objectives and targets
The process itself The activities involved in the process Key performance
process measures
Measures directly related to the process itself and overall business objectives, in the way customers measure the process
Monitoring performance
Systematic, regular monitoring of the measures in order to assess process performance
Improvement Activities that are designed to close the gap between current performance and the target performance level required
Consequently the evidence provided by people being interviewed will also be appropriate for their level within the process and will almost certainly be mainly non-documented and subjective.
Auditor tool 2 follows a similar theme but extends to include those things that support the process in terms of:
• the competence of those working within the process to effectively carry out their tasks;
• the resources needed for process activities to be performed adequately;
• the knowledge and information needed to effectively carry out activities within the process;
• the budget for the process that takes account of the likely future demands on the process.
These infuences or constraints are only examples and in reality there may well be others. What you are looking for is anything that affects performance of the process, and it can come from any management discipline. Process management auditors therefore need a basic foundation in a range of business activities and disciplines. For example how can an auditor assess or make judgements on someone’s competence if they have no understanding of human resource management principles?
24
Process Management Auditing for ISO 9001:2008
Risk Inputs
Competence
Resources
Knowledge
Budget
Outputs
Measure Monitor Improve
Activity Activity Activity Activity
Procedures Procedures
Date: 04/08/2009BSI/PM: Siobhan FitzgeraldModifications: Approval of issue Operator: Nora Dawson 7769Department:Modifications:Date:Signature:
File name: 2009-01 730_4.2.eps BIP 201 5
Figure 4.2 Auditor tool 2
Auditing techniques
Questioning
Taking each box of ‘auditor tool 1’ let’s look at each one in turn and try to work out the most appropriate question to ask, and the place to look for answers.
As we go through each box we will, in addition, include all the elements from
‘auditor tool 2’. Remember, however, that evidence is not only gathered through questioning, but by a range of techniques.
The end result will be an audit checklist you will be able to use to prepare for and to audit most processes. You may well be able to come up with other areas and issues to raise; whatever they are they need to test the effectiveness of the process. As you go through the steps in the cycle you may well be able to identify areas where you need to dig a bit deeper, ask more questions and test any compliance issues that may become apparent. Inexperienced process management auditors tend to stay in the detail of compliance once they are in it. The ‘art’ is to keep the cycle in mind as you carry out the audit and ‘dip’ into the detail as required, coming out of it to move on to other parts of the cycle in order to build the links. It is not easy at frst to make this change, but once you’ve done it a few times it will become much more ‘second nature’.
Auditing tools and techniques
25
Table 4.2 Auditor questions
Part of auditor tool 1 Question
Purpose of the process • How does the process support the business strategy and objectives?
• What are the process’ supplier inputs and customer outputs?
• How do you determine what the customer requirements are; is this the ultimate customer?
• Where do you get your work from?
Process objectives and targets
• How do you determine your objectives and targets?
• What are your objectives and targets?
• How do they link to and support the overall business objectives?
• How do you plan for future customer demands and the likely resources required to support them?
The process itself • Can you describe the process?
• How do any procedures support the process?
• Who is your customer?
• How do you know what your customer requirements are?
• How does this process interact with other processes in the management system?
• Who do consider as your supplier?
• How does your supplier support you?
• How do you determine the competencies required for those responsible for process activities?
Key performance process measures
• How do you decide what key performance indicators to use?
• How are the process measures linked to business objectives and measures?
• How does your customer measure the performance of the process?
Performance monitoring
• How do you know what the current performance of the process is?
• How often is process performance measured?
• How is performance data communicated to the process team?
Improvement • How do you identify improvement issues?
• How do process team members contribute to improving process performance?
• How to you evaluate the success of improvement activities?
• How have improvement actions affected process performance?
• How are improvement actions communicated to the process team?
26
Process Management Auditing for ISO 9001:2008
Questioning techniques
The questions detailed above need to be thought about and tailored to suit the individual being interviewed and the level at which they support the process.
For instance, asking an operator carrying out a process activity if they know what the organization’s business objectives are would often be pointless in many organizations as the operator would more than likely think you were talking a foreign language! But beware that this is not always the case and, importantly, use your own knowledge of your own organization to get the language right.
As an auditor you have to consider what is the most appropriate question to ask and in this case it might be asking the operator who they consider is their customer and how they know they are meeting their customer’s requirements.
Show me how…
Tell me how…
How do you… ? Explain to me how… ? Who do you… ?
‘Auditors have to manage this dynamic’
Directors
Managers
Staff
Figure 4.3 Appropriate questioning techniques
Auditors that understand this dynamic and use it effectively in conjunction with both of the auditor tools will gather the greatest amount of information relevant to how effectively the business is managing its processes. The more information an auditor has on the company’s performance the more valuable the audit report they can generate from it becomes.
Auditing tools and techniques
27
Objective evidence
If we have established that the questions and questioning techniques you use as an auditor vary according to the person being interviewed and the level they are working at within the process, then it must also follow that the objective evidence you obtain will also vary accordingly.
In Chapter 2 we looked at examples of documented and non-documented objective evidence, so let us now consider what types of objective evidence we might fnd at different levels in the business, depending upon who we are auditing and what questions we are asking.
Taking some of the questions from the Table 4.2, Table 4.3 outlines the likely objective evidence you might expect to fnd.
Table 4.3 Objective evidence
Question Evidence from process owner Evidence from process staff
• How does the process support the business strategy and objectives?
• What are the process supplier inputs and customer outputs?
• How do you determine what the customer requirements are; is this the ultimate customer?
• Clear understanding of objectives
• Tells you what they are
• Able to link to customer outside their process and describe requirements
• Understands what the process is there to do
• Tells you what they are
• Able to link to next step in the process and describe requirements
• How do you determine your objectives and targets?
• What are your objectives and targets?
• How do they link to and support the overall business objectives?
• How do you plan for future customer demands and the likely resources required to support them?
• Link to overall company objectives and targets
• Tells/shows you
• Clear understanding of overall company objectives and targets and can demonstrate linkage
• Tells/shows you plan, gives example of having done it previously
• Understands their role in the process
• Understands process performance
• Understands company’s aims
• In touch with process customers and makes suggestions to process owner
28
Process Management Auditing for ISO 9001:2008
Question Evidence from process owner Evidence from process staff
• Can you describe the process?
• How do any procedures support the process?
• How does this process interact with other processes in the management system?
• How do you determine the competencies required for those responsible for process activities?
• Tells/shows you
• Tells/shows you links to process activity
• Tells you what the links are and how the communication between them works
• Understands roles and competencies in the context of process activities, linked to objectives
• Tells/shows you
• Tells/shows you when used and how
• Understands there are links to other processes and knows how they work
• Knows own competency and has been appraised/
reviewed in last year
• How do you decide what key performance indicators to use?
• How does your customer measure the performance of the process?
• Tells/shows you the indicators and which link to objectives
• Demonstrates customer communication by linkage of their needs to process measures
• Tells/shows you process measures being used
• Understands process performance in relation to the customer
• How do you know what the current performance of the process is?
• How often is process performance measured?
• How is performance data communicated to the process team?
• Shows you performance information
• Tells you
• Tells/shows you
• Tells/shows you
performance information
• Tells you
• Tells you/shows you
• How do you identify improvement issues?
• How do process team members contribute to improving process performance?
• How to you evaluate the success of improvement activities?
• Able to link performance data to improvement action
• Tells you and can give examples from team
• Links back to performance data to demonstrate effectiveness
• Talks through methods/
ideas and links to process owner
• Knows how and who to suggest improvements to
• Communication from process owner
Auditing tools and techniques
29
You will notice that the responses you get in terms of evidence are likely to be verbal rather than documented, which means you have to determine fact from fction just by listening to what people are saying.
But how can you do this? Let’s take just one of the questions and use it as an example.
Question: How do you know what the current performance of the process is?
The process owner’s response is to tell you that they have two process measures, products delivered on time as a percentage and number of product stock turns in a year. The targets are 99 per cent on-time delivery and 1 2 stock turns per year respectively.
They also tell you that since the measures were introduced six months ago they have achieved an average of 97.5 per cent deliveries on time and are on schedule for six stock turns for the frst half of the year.
You just listen to what they say and make a note of the information on your checklist.
The process staff member’s response is to tell you that the process owner meets with all the process staff once a month in the canteen where they talk through various items of interest including performance statistics. They tell you that a lot of what the process owner says is not of much interest to them apart from the delivery and stock turn measures as this has a direct bearing on the amount of bonus they receive each quarter.
They tell you that delivery performance of only 97.5 per cent has meant a reduced bonus for the last two quarters, but the achievement of six stock turns so far this year has at least given them a bonus payment albeit small.
You listen and compare their responses to those of the process owner, making any notes on your checklist. You then ask yourself: ‘Have I enough evidence to demonstrate that the question has been answered adequately and am I satisfed that the performance of the process is known at all levels in the process and by the people who need to know?’ What is your conclusion based on the two responses above?
30
Process Management Auditing for ISO 9001:2008
I hope you concluded that yes, the performance of the process was known at all levels in the process and by the people who needed to know. All this despite the fact you did not see a single piece of paper!
Congratulations! You have just audited Subclauses 5.1, 5.2, 5.4.1, 5.5.3, 7.1, 8.1, 8.2.3, 8.4 of ISO 9001:2008.
Methods of auditing
Quite rightly most methods of auditing involve face-to-face interviews/discussions with people in order to gain information and an understanding of how effectively something is being done. However, this is not always practical to do because of geographical locations, the high number of people needed to be seen or constraints on cost or time.
As mentioned at the start of this chapter, auditors should be fexible in their approach and be prepared to consider alternative methods and techniques of auditing that do not rely on just face-to-face interviews. These could include:
• groups – a number of process staff, suppliers, customers or stakeholders can be interviewed within a group environment to save the time and expense of travelling to them individually;
• questionnaire – could be used to assess a variety of issues, can be done confdentially to improve the honesty of the responses;
• email – again, any number of process staff, suppliers, customers or stakeholders can be interviewed remotely, as a group, to save the time and expense of travelling to them individually;
• telephone – this can be usually a very quick and simple way to confrm information;
• videoconference – planned well in advance, this method can be a really effective way of interviewing people working miles apart or even in different countries.
Organizations that have multiple sites spread over a large geographic area, including different countries, and those with large numbers of home or feld based employees are probably best suited to alternative methods of auditing other than face to face.
31