Auditing as a skill
Auditing is a skill and like any other skill needs practice to hone it. It involves an ability to evaluate or learn from the experience, subsequently changing the auditing style or approach to add more value to the activity. Clearly competence to audit is a key requirement but to enable this competence to be built (something that is less easy to train) are the personal attributes, inherent in any good auditor.
These attributes underpin the auditing activity and are the basis upon which competence is built.
ISO 19011 describes these attributes and although not an exhaustive list, it does provide a useful insight into what is expected. Above all the auditor should be ethical; auditors are placed in a position of trust by management to investigate how effectively the organization is being managed. As we have seen auditors need to assess effectiveness of actions taken as well as compliance.
To assess effectiveness requires the auditor to ‘expose’ areas of strength and weakness, identifying where the organization can make improvements or
changes that will enhance performance. In talking to different people at different levels within the organization, often being party to ‘sensitive’ information, the auditor should be careful to ensure that confdentiality is maintained at all times, whatever the pressure to disclose sources of information. This is not always easy and sometimes pressure is exerted, but those seeking the information should be made aware that its disclosure will break confdentiality, which may result in auditees being reluctant to take part fully in later audits to the detriment of future audits and therefore the organization.
54
Process Management Auditing for ISO 9001:2008
Equally the results should be a fair and honest refection of the fndings, reporting facts and not seeking to apportion blame or falling into the ‘solutionism’ trap. ‘Solutionism’ is where the auditor writes their report explaining how managers should actually carry out the improvements or resolve problems. No matter how well meaning, it is often dangerous to make recommendations to managers on how they should manage their organization – that’s their job, not the auditors. Many books or guides on auditing often suggest that the auditor should make recommendations but this needs to be done with care. It is one thing to make a statement that something is blatantly incorrect or is not working as well as it could and provide the evidence to support this.
It is quite another to go further than this and suggest how the improvement should be carried out. Very seldom does the auditor have as good a view of the organization as the manager. How the manager resolves problems or implements an improvement is up to them. Following the appropriate process, of course, is up to them. So, report the facts and leave any recommendations on what needs to be done or action that could be taken until after the audit. We have seen a number of internal and external auditors ruin a very good audit by making recommendations that are inappropriate and get a negative reaction from the manager – so be aware.
Auditing for effectiveness often involves understanding what is happening.
How an organization manages its business, how people carry out their tasks, what equipment they use and how they comply with legislation for example is up to them and the auditor can expect to see or observe activity that is different between one organization and another and even between one department or site and another in the same organization. In other words there is not necessarily a right or wrong way. Auditors need to be open-minded as to the activities undertaken and willing to consider different views or interpretation. What is more important is how effective these actions are on the fnal result achieved.
Adopting an open mind goes hand-in-hand with carrying out the audit in a tactful and diplomatic manner. Remember the easiest way to gather information is to ask people what is happening, what they do, how they could improve what they do etc. How the auditor handles this conversation, even if auditing using email and other non-traditional methods of auditing, is critical to success. If the auditor criticizes what someone is doing or how a manager is managing their part of the business then that person is likely to be more reluctant to provide the auditor with the information they need. Remember people are often not the problem, most of the time it is the system they are operating in, so identify where the system is failing rather than seeking to criticize, blame or expose the individual. The results will be far more welcome and of considerably more value to the organization.
What personal attributes do auditors need?
55
When auditing there is often a sense of something being right or not quite right, it’s a feeling. You can’t be certain because you might not have the evidence, but an instinct that there may be something that is taking place that is either incorrect or wrong or could be improved. This ‘second sight’ is all about perception, how the auditor sees, reads and understands situations. This perception may be drawn from looking at evidence from different sources – an adding together of information that doesn’t quite make sense and needs testing or examining further. Auditors need to develop and, more importantly, use this ability. Often the information an auditor needs won’t ‘stare them in the face’ or be straightforward and needs ‘digging out’ based upon reading a given situation.
Another area based upon perception is collecting ‘perception-based’ information.
This is often more valuable than fact-based or document-based evidence. The problem is that how people perceive situations, activities or events is often not evidenced by documents – it’s often verbal or an interpretation. The auditor therefore needs to be able to turn this information into fact or objective evidence.
This is achieved by using an appropriate sample size, testing the perception to get to the facts. This may mean that someone has perceived an event incorrectly or drawn the wrong conclusions. The auditor’s job is to work with these perceptions and draw conclusions separating the fact from the fction.
To do this requires persistence, the ability to keep going even though auditees may put obstacles in the way. You may not get exactly the information you need or you simply get frustrated knowing there is something to be identifed but you simply can’t fnd it. If you fnd yourself in this situation keep going, think about the objectives of the business and the scope of the audit. How important is it, will it put the business at risk? Perhaps a different approach is required to gather the information. Persistence is not about pursuing something for the sake of it, it is about making a judgement for the sake of the business, the audit and importance of the issue.
Following on from persistence is the need to make decisions in a timely manner based on the evidence that has been gathered. These conclusions should be clear, unambiguous and understandable. This allows the auditee to be able to review the conclusion or fnding using the evidence the auditor has provided.
Poor conclusions based on poor analysis leads to the auditee not being able to understand what the conclusion is about or why the issue has been raised. Often poor analysis of the evidence results in confusion and inevitably fndings that are lower level detail (mainly compliance related) rather than the identifcation of improvements or the need for change to enhance effectiveness.
Often auditors fnd themselves working on their own, gathering information whilst they work with the auditees. This ability to work independently is an attribute not to be underestimated. This requires the auditor to be a self-starter,
56
Process Management Auditing for ISO 9001:2008
self-reliant, and having the necessary equipment and motivation to see the audit through without the support from other auditors.
How about knowledge and skill?
For auditors, knowledge and skill can fall into a number of areas:
• knowledge and skills of auditing itself;
• the management system and its supporting processes that are being audited as well as the organization or business itself;
• professional knowledge around the subject of quality; and
• specialist knowledge of supporting business processes such as business planning, human resources, fnance, etc.
The auditor needs to have a mix of skills and knowledge to be effective. These are interdependent and should not be considered or developed in isolation of each other, i.e. no one area is more important than the other – they complement each other.
Knowledge of the auditing principles
Knowledge of the auditing principles is aimed at ensuring that audits are carried out in a consistent manner following a defned approach. These principles are identifed in ISO 19011 and should support any auditing procedures and approaches that the organization has in place.
It goes without saying that the auditor should be able to follow the organization’s auditing procedure and approaches.
The auditor should be able to create an audit plan based on the scope of the audit. This should show who is going to be audited, how and when and be agreed by the process owner. The effective use of time is very important. Auditors should not forget that for most organizations auditing is an overhead, a cost to be borne by the organization. Therefore the organization needs to not only get value from the audit but also collect, collate and report information and other data effciently and effectively. The audit plan should refect this need and auditors should adopt approaches and methods that are appropriate. As mentioned early in the book these approaches may well be non-traditional in nature but will be more cost-effective without distracting from the value of the audit.
With the plan in place, agreed with the process owner and communicated to those being audited, it is the responsibility of the auditor to ensure that the audit is carried out as planned, keeping to the timescales as shown. Sometimes in
What personal attributes do auditors need?
57
an audit the auditor will discover areas that need more investigation than the time allocated will allow or, perhaps, someone else needs to be interviewed who wasn’t on the original plan. In these circumstances the plan may need to be amended and this is the auditor’s responsibility. It is not good practice for the auditor to either start late or to end an interview after the time previously indicated on the plan. The auditee will be expecting the plan to be followed. If the plan needs to be amended then the auditor should discuss or communicate this to the process owner or the person showing the auditor round the organization, if one is being used, in order that a revised plan can be agreed and communicated. This may include going back to an auditee to check a particular issue or to gather more information. Planning an additional interview is preferable to ignoring the original plan, however tempting this may be.
The auditor needs to maintain confdentiality. This not only applies to sensitive business or organizational information but also to personal feelings and views that may be expressed by an individual or group. Clearly the auditor may well be provided with sensitive business information as part of the audit that should not be shared either within the organization itself or externally – it must remain confdential. There is a temptation to share information with work colleagues but the auditor doesn’t necessarily know what has been communicated and what hasn’t and the reasons for this. Therefore to avoid any ‘situations’ it is best to simply say nothing and use the information for the purpose for which it was given, i.e. for the audit. This approach will avoid and prevent any diffcult situations or misunderstandings.
The same applies to views expressed by auditees. To assess the effectiveness and to gather information required often requires the auditors to gather views and examples from people not directly carrying out the task involved. For example let’s say you are auditing the manufacturing process, then you may gather information from the sales team, i.e. the people who generate the orders and those who dispatch products and services as well to gain their views and the impact the production process has on them. Or perhaps you are auditing an improvement process as well as auditing the people involved in the actual process or improvement. You could also interview the people affected by the change to determine how effective the change has been in improving performance. In gathering these views from people ‘outside’ the process being audited but affected by its impact, the auditor may well be gathering views and opinions from a number of different people to create the ‘objective evidence’ and to form a conclusion regarding effectiveness. These views and opinions also need to be kept confdential and not shared either with other auditees, e.g. ‘I was speaking to X and he said …’, or outside the audit. If the auditor breaches this confdentiality then it is likely that the auditee will be less forthcoming with information the next time an audit takes place, thereby reducing the effectiveness of the audits taking place.
58
Process Management Auditing for ISO 9001:2008
Auditors should focus their attention on signifcant issues. This does not mean that areas of detail should be ignored but that the audit should focus on what is important to the success of the process and the organization rather than areas that have little impact or signifcance in the overall picture. Some auditors get a reputation for ‘nit-picking’, i.e. identifying or making an issue of small areas that in themselves have little or limited impact on performance. If the auditor is in any doubt as to whether or not an issue should be raised then think about the manager who will be receiving the report, will they be interested? Is it important to them?
Collecting information is the key requirement of the audit. The information often comes from a range of sources from across the organization. The various parts of information are then ‘added’ together to form a view or fnding. It is often not a case of taking one ‘piece’ of information in isolation but adding different data together to form the ‘picture’. Therefore a key principle is to test or verify the different pieces of information to confrm their appropriateness and accuracy.
Auditors need to develop a ‘sixth sense’ to help them with knowing how often and when additional information is needed to determine or verify a fnding. It is not possible to review or look at every document or piece of information used or generated by a process. In addition it is very rare that the amount of time allowed for the audit would be suffcient to interview every manager or staff member involved in the process. This is compounded by the need to gather information from those outside the process. To manage this the auditor can use sampling techniques to help determine what information is required. Although these can be scientifcally and statistically based the auditor can also apply common sense. For example if there are six projects to look at then perhaps two could be sampled; if there is suffcient difference in the two then perhaps a third could be reviewed to confrm the fnding. Or if there are 250 employees who need to have objectives and understand how they ft into the process then perhaps 10 could be interviewed for fve minutes (50 minutes in total) rather than two for 25 minutes (still 50 minutes in total) to allow the auditor to gain a wider view of what is happening.
Understanding management systems and processes
As we have outlined earlier in this book and in others that make up this series, understanding what a process-based management system actually is and the principles of managing an organization by process is really important. It is not the intention to revisit the principles of process management and its impact on organizational performance but auditors who do not understand the principles
What personal attributes do auditors need?
59
will not be able to audit effectively, often fnding it diffcult to move beyond compliance auditing.
This extends to understanding how the various processes that make up the system interact with each other and how support or reference documentation such as procedures and other information is positioned and used within the system. It would also include how resources, equipment, budgets, competence, team work, knowledge, other standards and frameworks, knowledge, environmental, health and safety and regulatory requirements, information technology, intellectual property, management ability and techniques, results, changes etc. can impact on process performance. This does not have to be an in-depth understanding but should, at the very least, be an awareness of the possible impacts so that the auditor is able to form judgements on possible areas for improvement.
In addition, as mentioned before, the auditor needs to have an appreciation of general business processes, what might make up such a process and how the organization has interpreted these business activities into the management system and therefore into its processes.
Another impact on process performance that the auditor needs to be aware of and understand is that the organizational culture will affect both the audit and, potentially, process performance. The auditor needs to appreciate the organizational culture they are working in and work within this, modifying their auditing techniques and methods accordingly.
What professional knowledge does an auditor need?
The fnal area of knowledge is that relating to quality. Accepting that we have covered the business knowledge needed in other sections, this area relates to the
‘quality’-specifc knowledge that needs to be understood. Quality terminology is, in effect, business terminology that we have already covered. This can be extended to include quality management principles, which are, in effect, business management principles.
Where specifc ‘quality’ knowledge is of use is in understanding specifc tools and techniques that have traditionally been used by quality professionals.
Of course as the management system is process based and as these processes cover a range of management disciplines, including ‘quality disciplines’, the auditor can expect these tools and techniques to be found or used in the appropriate processes. Examples of this could be:
• statistical control, which could be used to assist the measurement of process performance;
60
Process Management Auditing for ISO 9001:2008
• failure mode and effect analysis, which could be used in a design and development process; and
• cause and effect analysis, which could be used in an improvement process.
Understanding these tools gives the auditor a wider and deeper appreciation of how traditional ‘quality’ techniques can be used to improve and support process performance.
What skills does the audit team leader need?
The need to audit processes and their management for effectiveness and
compliance, particularly in larger organizations, may well mean that audit teams may be needed. In the past where compliance to procedures was the only real requirement, individuals working on their own were often suffcient to carry out an audit. This may well not be the case when auditing processes, for a number of reasons as follows.
• Not all auditors have the same level of auditing competence. Different auditors will have different auditing experiences and skills. As processes run across the organization, inevitably auditees will occupy different positions within the business. They will have different responsibilities at differing levels with the business, different attitudes and experiences; the same auditor may not have suffcient skill to audit them all. A good compliance auditor does not necessarily have the competence to audit the effectiveness of a business planning process.
• Lack of confdence or experience. Although this is often caused largely by inexperience, nonetheless it is a critical factor if the audit is to be a success.
A good example of this is an auditor with compliance auditing skills being asked to audit the managing director to determine how effective the management system is in meeting business objectives. Although in some organizations this may well be acceptable, even promoted in others, it may place the auditor in a position where they are not going to do justice to themselves or the audit. This may simply be because they are not of the right grade, position or may not have the confdence or experience to audit a senior manager.
• Lack of understanding of the business and the process. To audit processes effectively auditors require an understanding of a wide range of business principles. This does not have to be an in-depth understanding but an awareness. For example it is often commented that auditors need an understanding of quality, health and safety, and environmental issues (the integration myth), but what about business planning principles or how an asset is managed or how people develop skills, i.e. management principles