cm_011404_item_5_report-regulatory-framework-finalv1-2

Executive summary 1Background and context Analysis of the regulatory model Learning from research Comparisons with other regulators Using existing data Structure of the report 2 Introduc

Developing a strategic framework to guide

the Care Quality Commission’s

programme of evaluation

REPORT

Kieran Walshe and Denham Phipps

January 2013

Executive summary 1

Background and context Analysis of the regulatory model Learning from research

Comparisons with other regulators Using existing data

Structure of the report

2

Introduction Regulatory mission and purpose Registration

Standard setting Information gathering and risk assessment Inspection and reporting

Enforcement Information provision Reviewing the regulatory model

7

Introduction Learning from research Comparisons with other regulators Using existing data

Conclusions

23

Introduction Learning from research Comparisons with other regulators Using existing data

Conclusions

33

Introduction Learning from research Comparisons with other regulators Using existing data

Conclusions

43

6 The competencies of the regulatory workforce

Introduction Learning from research Comparisons with other regulators Using existing data

Conclusions

55

Executive summary

This report sets out to show how evidence and research can be used by CQC to evaluatehow well its current regulatory arrangements in health and social care work, and plan futurechanges in those arrangements to improve their effectiveness and efficiency We are verythankful for the help, advice and support we received from many CQC staff in itspreparation Of course, we remain responsible for its content and conclusions, and for anyerrors or omissions

We develop a “logic model” of the current regulatory arrangements which tries to map outhow they are intended to work, and what assumptions are made or consequences flow fromthose arrangements We conclude that as currently configured, CQC is a “safety-net”regulator, focused on dealing with poor performance but with limited capacity or capability

to drive or support wider performance improvement We note that even in those terms thecurrent regulatory model has some inherent problems, but that the direction of futurestrategy signals a shift towards a more ambitious interpretation of CQC’s remit and purpose,which has important implications for the regulatory design/arrangements

We then discuss four areas – differentiation in regulatory design, standard setting, risk-basedregulation, and the regulatory or inspection workforce – and look at what evidence isavailable from several sources about how current regulatory arrangements work Weconclude:

 CQC’s generic regulatory model is unusual (compared with other regulators) and hard tomake work There are good reasons to consider greater differentiation between sectors(such as NHS/healthcare and adult social care) and within large sectors like adult socialcare This might mean the development of more specific and tailored standards andguidance, and greater specialisation among the inspection workforce

 CQC’s current standards are largely construed as an essential or minimal level ofperformance CQC could within its existing legislative and regulatory framework create amore differentiated, demanding and service specific set of standards, and it couldconsider making more use of standards developed in or with the sectors it regulates

 CQC had used a risk-based model of regulation, in which it adjusted its use of regulatoryinterventions like inspection with providers based on an assessment of risk andperformance, but has recently returned to a universal schedule of annual inspection inmost sectors We find that even modestly proportionate or risk-based regulationrequires a strong and stable database of performance data which has clear predictivevalidity, and a graduated range of regulatory interventions short of full inspection

 CQC’s use of a generic inspection workforce is, as far as we can see, not emulated byother regulators and is problematic for several reasons Regulatory staff need contentexpertise in the area they regulate, methods expertise in the regulatory system, andinterpersonal or behavioural expertise in dealing with people and organisations insometimes difficult and contested circumstances Specialisation has many advantages,and investment in the development of the inspection workforce is worthwhile

We conclude that CQC could make more use of evaluations than it has in the past both whenintroducing innovations in its regulatory arrangements so that they are properly tested and

in the routine working of its regulatory arrangements so that it has ongoing evidence of theireffectiveness and impact

Chapter 1

Introduction

Background and context

This report is the result of a short research project commissioned by the Care QualityCommission (CQC) from the University of Manchester Its aim was firstly, to help CQC tobring evidence to bear on examining and exploring the effectiveness and efficiency of itscurrent regulatory arrangements; and secondly to help CQC to develop its own internalcapacity to undertaken and use research and evaluation so that, going forward, it can makebetter use of opportunities to test, trial and assess regulatory changes and innovations,improve its efficiency and effectiveness, and gain a more robust understanding of its impact

on the quality of health and social care in England

Analysis of the regulatory model

Our first step was to develop and test a “logic model” which mapped out the underlying

“programme theory” for each major component of the regulatory arrangements In thisapproach we drew both on established methods for programme theory explication (see forexample Bickman 1987; Rogers 2008) and recent developments in realist or theory drivenevaluation (see Pawson and Tilley 1997; Pawson and Manzano-Santaella 2012; Marchal et al2012) The purpose here was to make explicit the assumptions or presumed mechanisms bywhich these regulatory components bring about change in regulated organisations In ourexperience, there may be multiple and sometimes contradictory theories in use, and part ofthe value of mapping out the programme theory is to express these alternatives in formswhich then make them testable, and which bring to the surface areas of contestation orinconsistency

We developed the logic model around the main statutory functions of CQC – registration,compliance, enforcement, and information provision For each function, we useddocuments and interviews with CQC staff to explore the mechanisms at work, and to try tounderstand the regulatory model and the choices and consequences it represents Wewere acutely conscious that this was a time of change within CQC, and the organisation had

in September 2012 published a strategic review on which it was consulting stakeholders.That strategic review signalled important, even fundamental changes to the regulatorymodel, and so in our work while we focused on mapping the current regulatory model wetried to take explicit account of the likely future direction as well

As part of this work, we sought to establish what were the areas where research andevaluation might make an important difference to CQC’s decision making – where there wassignificant uncertainty about how (or how well) the regulatory model worked, and wherechange seemed likely On that basis we chose four topics for further research:

 Generic versus differentiated regulatory standards and processes – whether regulatorsuse the same methods, standards and processes for organisations in different sectors or

of different types or whether and how they differentiate and use different standards orprocesses for different organisations

 How regulatory standards are set and measured – issues like whether standards areminimal, median or maximal, how they are measured, whether compliance/achievement

is measured dichotomously or on a scale, how compliance is defined and how athreshold for acceptable performance is set, and whether different standards areused/applied for different organisations

 Risk based or proportionate regulation – to what extent regulators try to make theregulatory process responsive to organisational performance, and so focus moreattention on organisations which perform less well or represent greater risk, and howthis is done

 The regulatory or inspection workforce – what competencies are required, what kind ofpeople are used to undertake regulation and inspection, how are they recruited andtrained/developed, how is their performance appraised

In each of these four areas, we set about drawing together what was already known fromthe research literature; comparing practice at CQC with four other regulators; and exploringwhat was known or might be found from using existing data sources within CQC

Learning from research

We looked for existing research evidence in the four topic areas using a wide range ofbibliographic databases The general strategy for the literature review was to identifyrelevant literature in public administration, healthcare, management and safety science, andwhere appropriate, governmental policy documents The researchers searched theMEDLINE (1946-2012), EMBASE (1980-2012), ABI Inform (1971-2012), ASSIA (1987-2012),British Nursing Index (1993-2012), Social Services Abstracts (1979-2012), and HMIC (1979-2012) databases For the regulatory workforce topic, PsycInfo (1806-2012) was alsoconsulted The keywords used for each topic were as follows:

 Differentiation of models: regulat* model*

 Regulatory standard setting: regulat* AND standard*; ("nursing home" or "care

home") and regulation

 Risk-based regulation: risk based regulat*

 Regulatory workforce: regulat* AND inspect* AND (train* OR compet*)

In addition to the search of academic databases, public domain repositories (for example,government and NHS websites, and relevant professional organisations) were consulted.Also, the authors’ previous work on related topics, and the reference lists of retrievedarticles, were consulted for relevant material

Comparisons with other regulators

We identified four other regulatory agencies with whom to compare CQC in the four topicareas We chose two healthcare regulators – the Joint Commission for the Accreditation ofHealthcare Organisations in the United States and the Inspectie voor de Gezondheidszorg(the Healthcare Inspectorate) in the Netherlands We chose two UK-based non-healthcareregulators – the Office for Standards in Education, Children’s Services and Skills (OFSTED)which regulates children’s services, particularly schools, and the Homes and CommunitiesAgency which regulates social housing providers Our aim was to provide a range ofexamples of regulatory policy practice, in both the health and social care setting andelsewhere Some important characteristics of the comparator regulators – their overallremit/purpose, their size and scale, and who and what they are responsible for regulating –are set out in table 1.1 We gathered data about the comparator regulators through areview of published and unpublished documents, and interviews with a member of stafffrom each regulator The interpretation of the comparator regulators’ arrangementscontained in this report is, of course, ours rather than an official statement of the agenciesconcerned

The table illustrates that the four regulators vary in some important ways, though they havemuch in common Their statements of regulatory remit/purpose – taken from their owndocuments such as annual reports – generally all focus on performance improvementthough some are more ambitious than others They range in scale from about 120 staff to1,400 or more, and in annual turnover from £48 million (€55 million) to £167 million pa.Most though not all are public/state organisations (JCAHO is a private not-for-profitfoundation) All regulate thousands of organisations, and all but HCA regulate acrossmultiple sectors or service areas

Table 1.1 Comparing CQC with four other regulators

Name Care Quality

Commission

Joint Commission for the

Accreditation of Healthcare Organizations

Inspectie voor de Gezondheidszorg (Dutch Healthcare Inspectorate)

Office for Standards in Education, Children’s Services and Skills

Homes and Communities Agency

Sector Health and social

care

Healthcare Healthcare Childrens

services

Social housing Country England USA Netherlands England England

Organisational

form

Non Departmental Public Body

Private not for profit foundation

Part of Ministry for Health, Welfare and Sport, but partially independent

Non ministerial government department

Non Departmental Public Body

Formal

regulatory

remit/ purpose

to “protect and promote” the health, safety and welfare of service users and the

“general purpose of encouraging the improvement of health and social care services”

“To continuously improve health care for the public, in collaboration with other stakeholders, by evaluating health care

organizations and inspiring them to excel in providing safe and effective care of the highest quality and value.”

“Promotes public health through effective enforcement of the quality of health services, prevention measures and medical products.”

“inspects and regulates to achieve excellence in the care of children and young people, and in education and skills for learners

of all ages, thereby raising standards and improving lives”

“focus of our activity is on governance, financial viability and value for money as the basis for robust economic regulation; maintaining lender confidence and protecting taxpayers”

Annual

turnover

£149 million $165 million+ €55 million £167 million £55 million

(whole of HCA – regulation function not separated)

Curative healthcare, long term care, public health,

pharmaceuticals and medical devices, health professionals

Schools (maintained and independent), further education, adult learning, early years/ childcare, childrens homes, childrens social care, adoption/

fostering

Registered social landlords – incl LAs, housing associations and for-profit providers

No of

organisations

regulated

22k in health, social care 20k 3k organisations, 800k professionals 23k schools and about 100k other

providers

1,500 though

400 large RSLs are 90% of provision

Using existing data

We were keen to see whether existing data held by CQC, particularly that produced for or as

a byproduct of the regulatory process, could be used to understand current practice in CQC

in the four topic areas To that end, we met with CQC staff responsible for intelligence anddata analysis, to get a better understanding of data availability and they undertook someanalyses which we draw on in this report by way of examples, where possible Our aimhere has been to establish the principle that existing or routine data can be used to servethe purposes of evaluation, either retrospectively or prospectively

Structure of the report

The next chapter of this report – chapter 2 - sets out results of our work on the regulatorymodel It offers a detailed examination of the current regulatory model, organised aroundthe four main regulatory functions of CQC – registration, compliance, enforcement, andinformation provision We break down compliance, which is a large and complex function,into three components of standard setting, information gathering and risk assessment, andinspection and reporting For each of these functions, the chapter aims to describe whatCQC does, set out the alternate theories or logic models which underlie the function andwhich one seems plausibly to “fit” CQC, and comment on how CQC’s approach and thetheory or logic model fit together, and what might be the consequences of the model atwork The chapter concludes by reflecting on the overall current regulatory model, andnoting some of the key changes to that model signalled i CQC’s strategic review

Chapters 3, 4, 5 and 6 of the report then take the four topic areas in turn – differentiation inregulatory design, regulatory standard setting, risk-based regulation, and the regulatoryworkforce For each topic, the chapter first reviews the research evidence and seeks tocreate a framework for analysis, then explores how our four comparator regulators deal withthe topic in hand, and then considers whether CQC’s existing data could be used tounderstand how the current regulatory model works in this area Each chapter finishes withsome brief conclusions

Finally, chapter 7 of the report draws together our conclusions, and outlines some briefrecommendations for the future development of CQC’s framework for evaluation andresearch

“programme theories” at work and it can be valuable in two ways First, it require us to beexplicit about the way that regulation is meant to work, and so allows those underlyingassumptions or mechanisms to be questioned or contested and questions about how thetheoretical intent plays out in practice to be asked (Prosser 1999) Second, it provides aframework for evaluating the effectiveness of those regulatory arrangements, and decidingwhat data is needed to evaluate how well they work in practice Our methods forundertaking this analysis are described in chapter 1, but relied mainly on interviews withCQC staff and a review of relevant CQC documents.

Our starting point for this analysis was the four main regulatory functions of CQC –registration, compliance, enforcement, and information provision – and in the rest of thischapter we set out our understanding of the regulatory model in these areas We breakdown compliance, which is a large and complex function, into three components of standardsetting, information gathering and risk assessment, and inspection and reporting For each

of these functions, we try to do three things – describe what CQC does, set out the alternatetheories or logic models which underlie the function and which one seems plausibly to “fit”CQC, and comment on how CQC’s approach and the theory or logic model fit together, andwhat might be the consequences of the model at work

Before starting to discuss these regulatory functions, we raise three issues that essentiallyconcern CQC’s regulatory philosophy and its mission or purpose They arise from ourinterviews and document review, and are important determinants of regulatory design anddevelopment

Regulatory mission and purpose

A discussion of the regulatory model in use and how it works has to recognise that thechoice of model is shaped by the regulatory mission and purpose – what CQC is here to do.While that is defined in a number of places – in legislation, in CQC’s strategic plans andannual reports, and in communications with regulated organisations, we think there arethree aspects of mission and purpose which need to be highlighted because of their impact

on the regulatory model

1 Is CQC’s regulatory purpose to ensure that all regulated organisations meet certain minimum or essential requirements, or is it to raise performance and standards of care across all organisations, or does it seek to do both?

This is a crucial question because minimal or safety-net regulation requires quite differentregulatory processes, standards, measurement systems and so on from maximal orimprovement-oriented regulation We understand that while the legislation whichestablished CQC gave it a remit to “protect and promote” the health, safety and welfare ofservice users and the “general purpose of encouraging the improvement of health and socialcare services” (Health and Social Care Act 2008) its strategy has focused in recent yearsprimarily on acting to deal with poor quality care However, the recently published strategicreview seems to signal a substantial change in direction, with CQC once more seeking todrive improvement

2 Does CQC see its mission and purpose as determined by the legislative powers and duties

it has been given, or does it see those legislative powers and duties as a means through which to achieve its mission and purpose?

In other words, does CQC conceive of its mission in narrow terms as to fulfil the legal dutiesset out in the Health and Social Care Act 2008 and, where it wishes to, to exercise thediscretionary powers that legislation gives it? Or does CQC seek more broadly to embracethe wider mission and purpose defined in its legislative objectives, and regard the legislativeprovisions as providing a set of tools and a framework within which to enact that widermission and purpose? Again, this question matters because it shapes the approach CQCtakes to regulatory design and development, particularly how much discretion or room formanoeuvre it has (or believes it has) in regulatory design decisions A legislatively-ledregulator will tend to retreat to the core functions explicitly enshrined in legislation, and willnot seek to do things, even if they are necessary to its mission and purpose, which it is notexplicitly empowered to do by the legislation A mission-led regulator will be more willing

to use its informal and quasi-legal “soft” powers, and the leverage it gets from positionalauthority, professional standing, market pressures, media attention and the like to achieveits mission – and its strict legislative functions may be only part of what it does

3 What view does CQC take of the organisations it regulates, and what kind of relationships does CQC seek to have with regulated organisations?

Does CQC see the provider organisations it regulates as partners or as adversaries; asgenerally well-intentioned and honest or as amoral, utility-maximising and calculating? DoesCQC see the relationships it has with providers as essentially transactional ones which existjust to deliver the immediate purposes of any regulatory interaction, or does it see theserelationships – and some of their “softer” characteristics such as mutual trust and respect,

credibility, and longitudinality – as materially important to the regulatory process and itseffectiveness This question matters once again because whether you take an instrumental

or a socio-organisational view of these regulatory processes and relationships fundamentallyaffects the approach to the regulatory model

Of course, there may be other aspects of CQCs regulatory philosophy and mission whichdeserve debate too, but our point is that these issues – whether debated or unspoken – areimportant influences on the design and development of regulatory arrangements, as wehope the analysis of the CQC regulatory model in the rest of this chapter helps to illustrate

Registration

CQC has a statutory duty to register providers of health and social care, and the workload ofregistration associated with its growing regulatory scope (such as dental services and generalpractice) has been considerable The process of registration is well defined, and is delivered

by a separate operational workforce from that responsible for ongoing regulation However,our interviews suggested some divergence of views about purposes of registration, and themodels in use which these mechanisms might suggest We identified three different and notnecessarily compatible models which might be at work:

 Registration as a threshold for new providers to meet Here, registration sets a standard

which new providers have to reach before they are allowed to enter the market Itshould deter poor providers from coming in (when they see what will be expected ofthem and realise they cannot meet required standards) and it should drive aspirantproviders to improve in order to get their registration - a potentially powerful incentive

In this model, some providers might take quite a while to achieve the required standards,and some might well be rejected

 Registration as the start of a regulatory relationship with the provider Here, registration

is the first opportunity to get to know a new provider, to begin building up knowledgeabout their performance which will then be used in ongoing regulatory intervention and

to start to establish a constructive working relationship with them It is likely to involveextended and face to face contact with the provider, as well as initial data collection toestablish a baseline performance Again, it might take some time for providers tocomplete the process and get registered, and providers might be risk rated or triaged atregistration in this model in ways that then shape subsequent regulatory interactions

 Registration as administrative data capture Here, registration is primarily an

administrative process, in which the basic facts about the provider are established orchecked – issues like financial standing, the identity of key individuals, and conformancewith essential and statutory requirements It is a transaction which can be accomplished

quickly and efficiently, perhaps with little or no actual face to face contact, but at itscompletion there is limited assurance about performance, and no establishedrelationship In a sense, this model postpones dealing with any concerns aboutperformance from registration to ongoing regulation.

We think that as it is currently configured, CQC’s registration process and its metrics areprimarily constructed around the third model – to provide an administrative record ofregistered providers There is a focus in the registration metrics on speed and efficiency ofprocess Interviewees did not generally think that registration required providers to improve

to meet the essential standards, or that registration provided assurance that newlyregistered providers were meeting the essential standards Because registration isconducted by a separate team, it does not seem to initiate the regulatory relationship, andsome questions emerged about the transition of responsibility and the handover ofinformation to those responsible for ongoing regulation However, this probably means thatCQC has foregone opportunities for improvement in the registration process, thatregistration does not provide much assurance about the performance of new providers, andthat regulatory work which could be achieved through registration is essentially deferredand transferred to those responsible for ongoing regulation

Standard setting

CQC works to a set of regulations set out in secondary legislation by the Department ofHealth, which have then been mapped into a separate set of 28 outcomes in CQC’scompliance framework which is designed to be used across all providers For each outcome,there is some guidance about its interpretation in the compliance framework CQCinspections focus on what are often called its 16 essential standards (items drawn from theset of outcomes) and on each inspection compliance with at least five of the 16 essentialstandards is measured dichotomously – one in each of five areas or dimensions If aprovider is found to be non-compliant, enforcement action is taken using the statutoryregulations, not the outcomes

Almost every regulator sets out its expectations of providers in some form of standards,rules, or regulations, often accompanied by further guidance on interpretation andimplementation The processes for standard setting, the way they are written, and howthey are communicated are all connected to the wider regulatory purpose – what they arefor We can identify at least four potential models underlying the setting of regulatorystandards

 Standards as a mechanism to set or frame stakeholders’ values and performance

expectations Here, the regulator sets out its requirements, often at quite a high or

conceptual level, both to communicate them to providers and also to inform others –

commissioners, patients/users, and the public In so doing the regulator helps to setstakeholder expectations and to give prominence to important issues or concerns –shaping the climate of regulation rather than necessarily setting detailed requirements.These standards may be expressed in quite abstract and high level terms – more likeprinciples or expressions of purpose or intent - and may not be particularly designed formeasurement In a sense, such standards are an explicit expression of values more thatthey are a tool for performance measurement.

 Standards as a mechanism for improvement through self-enforced compliance It can be

argued that much regulatory compliance results not from regulatory interventions likeinspections but from providers seeing what the regulator wants, understanding thoserequirements and responding – possibly in anticipation of future inspection, but also forother reasons such as competitive pressures or professional motivations For this towork, the standards need to require improvement in most providers, who in turn need

to understand the standards well, largely agree with what they require, and know how toand be able to implement them So these standards are likely to be maximal, explicit,detailed, and accompanied by further guidance

 Standards as a mechanism for compliance through measurement and enforcement In

this model, the standards are seen as primarily providing a framework for measuringproviders’ performance, often via inspection, and then for tackling noncompliance orpoor performance This means that standards need to be expressed in terms whichfacilitate valid and reliable measurement There may be a particular focus on settingstandards in areas where enforcement is seen as needed, and on determining non-compliance reliably in ways that then allows enforcement Standards are likely to beminimal in nature since the regulator is unlikely to be able to take enforcement actionwith more than a small proportion of providers

 Standards as a mechanism for differentiating between providers In this model, the

standards are used to measure performance, not just to assess compliance and initiateenforcement Often, the intention is that comparative performance against thestandards will be used by providers themselves and by others (such as users orcommissioners or services) to make decisions about services they use, and that this willact as an incentive to improve performance These standards are likely to expressed interms which facilitate valid and reliable measurement and to be designed to measure awide variation of performance levels

These four models are not mutually exclusive, and a regulator might seek to draw on morethan one of them in its approach to standard setting In our interviews with CQC staff andreview of documents, it seemed that perhaps the first and more probably the third modelwere in use – standard setting for compliance through measurement and enforcement But

a number of quite contested issues emerged, suggesting that there are some tensions orinternal inconsistencies in the regulatory model which merit some discussion:

 Generic standards CQC has a single set of generic standards/outcomes which it uses to

assess the performance of quite a diverse range of providers in health and social care Itdoes not produce interpretative guidance or specific information for particular sectors orservice areas The case for doing this seems to be that the fundamental dimensions ofperformance are generic or universal and so can be understood and applied in anysector This may be so, but the more heterogeneous regulated organisations are, themore difficult it is likely to be to make generic standards valid and reliable, which theyneed to be if they are to be used in measurement and enforcement (as in the thirdmodel above) Moreover, if providers are to change in response to the regulator’sstandards, they need to have a good understanding of their content andimplementation, and for this purpose detailed sector-specific guidance is probably alsohelpful

 Minimal, median or maximal standards CQC describes its 16 standards as “essential”

and noncompliance with any of those standards may lead to compliance actions andenforcement Compliance with most of the standards is actually quite high (on average73% of organisations inspected comply with all essential standards) CQC has chosen tofocus its attention in recent years on finding and dealing with poor practice Theseobservations all suggest that CQC’s standards are minimal ones – a kind of safety netstandard below which no provider should fall Most but not all our interviewees thoughtthis was the case Some regulators adopt median standards – representing typical ratherthan minimal performance – and others set maximal or optimal standards, which arelikely to require many organisations to improve in order to meet them Minimalstandards would not be helpful in the second and fourth model outlined above

 Content or focus of standards CQC’s standards and particularly its outcomes, are

intended to focused on patient or user experience, and their measurement is thereforepredominantly undertaken through observing care or talking to patients/users andstaff/caregivers Most regulators have to decide how much they use their standards todefine and then measure directly the quality of services delivered, or how much they usethem to define and then measure systems and processes which are intended to ensurequality services are delivered - an important distinction The former is problematicbecause of the costs and complexity of direct observation at any scale, especially in alarger regulated organisation with many services, and because it provides only ameasure of performance at the point and place of inspection and observation with noindication of whether this is likely to be typical or sustained performance The latter isproblematic if those systems and processes are not good and reliable proxies for thequality of service Many regulators end up measuring both – with the balance reflectingthose measurement challenges and the scale and complexity of the organisation

 Understanding and implementation CQC provides quite limited guidance in its

compliance framework on how to interpret or implement the generic standards andoutcomes By introducing a separate set of “outcomes” which are, essentially, arelabeling and resequencing of the statutory regulations (and most of which are not, bymost usual definitions, outcomes) it has also added an additional layer of complexity Ifregulated organisations are expected to understand the standards and know how toimplement them then the lack of sector-specific guidance and the apparent complexity

of the compliance framework may be unhelpful

 Measurement We come later to examine the model underlying the inspection process,

but it is worth considering not just the measurability of standards, but the nature ofmeasurement used (a dichotomous judgement of compliant/not compliant versus amore discriminating scale of some kind) and the way that data from observation,interview and other sources is integrated into the data item of measurement It hasbeen quite hard for us to understand, through our interviews and our review ofdocuments like the compliance and judgement frameworks, how CQC measuresorganisations against the standards/outcomes and particularly how it reaches a reliableand valid compliance/noncompliance judgement based on quite limited and largelyobservational data

Perhaps the key conclusion from this discussion of the model underlying standard setting isthat different regulatory purposes require quite different kinds of standards A regulatorwho sets out mainly to prevent poor practice – what is often called a safety-net regulator –probably wants minimal standards, couched in simple and measurable terms without muchadded guidance or interpretation, which are then straightforward to enforce by justassessing compliance/non-compliance and then acting accordingly A regulator who seeks

to drive improvement across the whole sector probably wants maximal standards, with ahigher level of detail and description, and accompanied by quite specific guidance onimplementation Assessment probably involves a graduated rating of performance againstthe standard, along with feedback on the areas for improvement

Information gathering and risk assessment

CQC collates information about regulated organisations in a Quality and Risk Profile Someinformation is drawn from routine data, and this is supplemented by information from othersources such as CQC’s own inspections, feedback from other regulators, complaints,

“whistleblower” reports, etc Data is statistically aggregated to form “z-scores” which arenormalised quantitative estimates of risk The quantity and quality of information variesacross different types of regulated organisations The idea is that compliance inspectors usethe QRP to decide when to inspect organisations and what areas to focus on during theirinspections Our interviews suggested that in practice, QRPs were not well used, for anumber of reasons discussed below

The idea of collecting information about regulated organisations and using it to tailor theregulatory process to individual organisational circumstances or characteristics is a verycommon one, and is often called risk-based or proportionate regulation It is intuitivelyattractive, and seems on the face of it to be a way to make the best use of limited regulatoryresources, by focusing them on providers where there is a demonstrated need forperformance improvement There are three basic models at work here:

 Using information to determine when regulatory interventions are used By collecting

information about ongoing performance, the regulator is able to measure currentperformance and/or predict performance trajectory or future performance, and thisinformation is used to decide where to focus regulatory resources for interventions such

as inspections or visits Poor or declining performance is likely to trigger regulatoryintervention Importantly, this model assumes that performance can be measuredsufficiently accurately and in a timely fashion, and that valid predictive measures ofperformance are available It also assumes that regulatory staff are able to use thatinformation effectively in their decision making about when to intervene

 Using information to focus or direct attention during regulatory interventions By

collecting information about performance, the regulator is able to identify areas of poor

or questionable performance which it can then focus on during a regulatoryintervention such as a visit or inspection For example, it may identify particular services

or functions which appear to be poorly performing in comparison with others, in thisorganisation or elsewhere This model assumes again that valid performance measuresare available, and that regulatory staff are able to use the information effectively ininspections

 Making regulated organisations aware that information on performance may lead to

regulatory interventions Because regulated organisations know their performance is

being monitored by the regulator, and that poor or declining performance may lead to

an unwelcome regulatory intervention, they act themselves to monitor their ownperformance and to respond to emerging concerns before regulatory interventionoccurs This model assumes that valid performance measures are available, that thedata is provided not just to the regulator but also to the regulated organisation, and thatthey have the internal capacity to use it to change their performance

From our interviews, it seems that CQC has sought to apply both the first and second models

in its development of QRPs However, a number of factors have contributed to the use ofQRPs being less effective than they might have been First, the quality, completeness andtimeliness of information has been questionable For some organisations the QRP containsvery little information, while for others (notably NHS organisations) it contains a largeamount of information which is difficult to make sense of It is difficult to manage and use

“soft” intelligence about performance in the QRP, and it was not clear to us how the softintelligence that inspectors acquire is recorded, shared or integrated Second, it is far fromclear that the information in the QRP has sufficient predictive value – in other words, that it

is useful in identifying individual organisations that are performing poorly now or willperform poorly in future – and this does not seem to have been demonstrated Third, it isnot clear that compliance inspectors have the capacity and resources to use informationfrom the QRP in their inspections, as the second model assumes Fourth, the reintroduction

of annual inspections for most organisations has undercut the main purpose of the firstmodel outlined above, which is to direct regulatory resources and interventions

The principle of risk-based or proportionate regulation is straightforward, but its practice orimplementation requires the availability of timely ongoing monitoring data which is both avalid measure of current performance and a valid predictor of future performance This is achallenging requirement to meet, which may explain why many regulators use regulatoryarrangements which are only weakly or partially risk-based/proportionate

Inspection and reporting

CQC undertakes inspections of regulated organisations It has committed to inspecting mostorganisations at least once every 12 months Inspections are unannounced, with inspectorssimply arriving at the organisation with no advance notice The length of the inspection andthe number of inspectors involved varies with the sector and the size and complexity of theorganisation, from less than a day with single inspector to two or three days with a smallteam Inspectors carry a mixed portfolio of organisations and are not expected to have anyparticular content expertise in the sectors they inspect They can call on specialist inspectorswith content expertise if they wish to, though this is fairly rare Most time duringinspections is used in direct observation of care, talking with patients/users and talking withstaff On each inspection five of the 16 essential standards are checked (with the idea that

over a three year cycle all of the standards will have been checked) Compliance with eachstandard is recorded, and a narrative report of the inspection is produced.

Inspection is by far the commonest form of regulatory intervention At its simplest,regulators use inspections to find out whether regulated organisations are conforming withthe requirements set out in their regulations or standards But inspection can serve widerpurposes than measurement, and we identify four important models in use:

 Inspection as a driver for improvement in advance of inspection Foreknowledge of the

prospect of inspection or of the actual date of inspection leads the regulatedorganisation to assess itself against the regulator’s requirements and to seek todemonstrate compliance Changes are made to deal with areas of non-compliance inadvance of the inspection This links to the model of standard setting for self-enforcedcompliance which was outlined earlier

 Inspection as a measure of compliance to support enforcement This is the

straightforward use of inspection to measure whether a regulated organisation isconforming to the regulator’s requirements set out in their regulations or standards If it

is not, enforcement action – or the prospect of enforcement action – is used to make theorganisation implement necessary changes Often there is some form of follow-up orreinspection to check that the organisation has become compliant

 Inspection as a measure of performance to support improvement This sees inspection

as an external evaluation or assessment of the performance of the regulatedorganisation which provides a diagnostic opportunity for the organisation to then makeimprovements There may be – in some cases – formal enforcement action taken whenthere is serious non-compliance or poor performance, but in most cases formalenforcement action is not taken, and for most organisations the inspection wouldidentify at least some opportunities for improvement

 Inspection as a driver for other regulated organisations to improve Here, the potential

effects of inspection on other providers (not that being inspected) who have not beeninspected but who anticipate they may or will be in the future are important Inspectionresults are published and publicised in forms which are designed for other providers tolearn from them – either as examples of poor practice or examples of good practice

CQC seems to use only the second model in its inspection process This is consistent with aview of the regulatory process that sees it as primarily focused on preventing and dealingwith non-compliance with minimal standards, through enforcement, as discussed earlier insections dealing with standard setting and regulatory purpose and mission It is notconsistent with any wider regulatory purpose which, for example, sees it as concerned with

improving performance But even in those terms, there are at least four further issues to dowith the inspection process which seem worthy of more consideration

Firstly, the process of inspection is largely oriented towards direct observation andassessment of care processes, and it was not clear to us how well that works in largerorganisations, where only a very small proportion of care delivery can ever be observed orassessed It seems that there is an underlying but largely untested presumption thatobserved care processes in one part of an organisation are a good proxy for unobserved careprocesses elsewhere in that organisation Secondly, the decision to inspect a limited set ofessential standards (generally 5 of the 16 standards) at each inspection sets the scope ofinspection and enforcement to a subset of requirements rather than to all of them Weunderstand the rationale for this decision was essentially predicated on the assumption thatinspecting against fewer standards would take less time, though that does not seem to havebeen tested It means that longitudinal and interorganisational comparisons ofperformance are problematic (as the standards tested at each inspection vary) Thirdly, asalready noted, we found it difficult to understand how the dichotomous judgement aboutwhether or not an organisation is compliant with a given standard is actually reached, andwhat evidential burden or threshold to deem an organisation non-compliant was required.The reliability of inspection judgements of compliance does not seem to have been tested,and the absence of explicit guidance about the interpretation of standards for particularsectors or in particular circumstances means there is much reliance on inspector judgement.Fourthly, the decision to give inspectors mixed portfolios of organisations and to not requireinspectors to have any content knowledge or expertise in the sectors is predicated on theassumption that content knowledge or expertise is not needed to reach a valid and reliablejudgement, and that too has not been tested, was much contested by interviewees, andseems intuitively implausible

Overall, it seems that CQC’s inspection process is designed around the inspection forcompliance and enforcement model outlined above (and CQC does not attempt to use theother inspection models listed) but that it is difficult to reconcile even the fairly limitedterms and purposes of that model with its execution in practice, for the several reasonsoutlined above

Enforcement

CQC has a range of enforcement powers which it can use with non-compliant providers.These range from compliance actions and warning notices up through civil penalties andplacing conditions on registration to the suspension or cancellation of registration When aprovider is found to be non-compliant, a judgement framework is used to determine theenforcement response In practice, the great majority of enforcement actions have been

compliance actions though latterly CQC has made increasingly frequent use of warningnotices.

It is worth noting that formal enforcement is just one of the ways that regulators secureimprovement or compliance, and as the earlier sections of this chapter suggest, otherregulatory interventions (such as standard setting and inspection) have important rolesalongside enforcement However, in understanding how enforcement brings aboutcompliance we can identify four main models at work:

 Informal enforcement or the prospect of enforcement as an incentive to drive

compliance Here, it is the prospect of enforcement action, or informal action short of

enforcement which causes the provider to make changes and achieve compliance.Informal actions may include inspectors communicating the issues of concern verbally inadvance of a formal inspection report which would lead to enforcement, whilemechanisms like notice of deferred action may be used to give providers the prospect ofenforcement In both cases, it is assumed that the prospect of enforcement action issufficient incentive for providers to change, and that they have the capacity to change

 Enforcement as an incentive to drive compliance Here, the straightforward rationale is

that providers, faced with the costs associated with enforcement (for example, fines, loss

of business or damage to reputation, continuing or increased regulatory scrutiny andattention, etc) act to make changes and achieve compliance because doing so will costthem less in the short or long run This means that the actual impact of enforcementaction needs to be greater than the cost of compliance (or a provider might rationallydecide to accept the enforcement action as a business cost), and it again assumes thatproviders have the capacity to change

 Enforcement as a symbolic action to drive compliance In this model, the actual content

of enforcement action is seen as less significant, and enforcement is seen as having asymbolic purpose – in publicly identifying and labelling a provider as non-compliant Theactual penalty or cost associated with enforcement for the provider may be relativelytrivial This “naming and shaming” view of enforcement assumes that providers place ahigh value on their reputation and public standing, and perhaps see themselves as goodcorporate citizens with a social purpose which would be incompatible with being non-compliant with the regulatory requirements, so they will respond to the symbolic impact

of enforcement action It also, again, assumes that providers have the capacity tochange

 Enforcement as a driver for other regulated organisations to achieve compliance In this

model, enforcement action against one provider is seen as providing a lesson – and adeterrence to non-compliance – to other providers Here, enforcement action is likely

to be made public and publicised, with the intention that other providers will take

notice Again, it presumes that the prospect of facing similar enforcement action issufficient incentive for providers to change, and that they have the capacity to change.

It seems to us that CQC makes some use of a number of these models, but its enforcementactions are generally relatively trivial in terms of their actual impact on providers, so thedominant model might be thought to be that of enforcement as symbolic action It doespublicise its enforcement actions, and this might be thought to influence other providers toachieve compliance

However, some of the assumptions underlying these models deserve further consideration.Firstly, relying on enforcement action to secure compliance is an expensive approach todriving change, because enforcement action is usually resource intensive for the regulator.This means there is an effective and relatively low limit on how many providers the regulatorcan feasibly use enforcement action against Secondly, for enforcement action to have asymbolic value and deterrent effect as the models suggest, it probably has to be used quiterarely in any case Used routinely, enforcement risks becoming an accepted cost of doingbusiness, and an irritant rather than an incentive to providers to comply Thirdly, higherlevel enforcement powers are difficult to use in practice – they often have spillover effects

on other stakeholders than the provider, and their use can therefore appeardisproportionate - so regulators tend to make use mainly or only of lower level enforcementpowers, as indeed CQC does It is important to know whether such lower level enforcementpowers are indeed achieving their aim of getting the provider to comply Fourthly, andperhaps most importantly, all the enforcement models assume that the provider has thecapacity to change and comply If it does not, repeated enforcement actions are unlikely tosecure sustained compliance, may even result in a deterioration of performance, and otherways to bring about change and improvement in performance may be needed

Information provision

CQC publishes information from its regulatory processes on its website, in the form ofreports about providers and other outputs such as its annual reports and other publications.The website has been designed to allow users to find information about individual providersand to find reports of inspections and their results CQC also publishes a wide range ofother information not specific to particular providers, through its annual report, reports onthemed inspections, and guidance on its compliance and enforcement policies CQC doesnot publish all the information it holds about providers – for example, the content of theQuality and Risk Profile (QRP) discussed earlier is not published

Information provision might be thought to work in three ways:

 Information provision to be used by other stakeholders in their decision making Here,

the information published by the regulator is designed to be used by other stakeholders

such service users and their families, service commissioners/funders (such as localauthorities, PCTs/CCGs), the local and national media, etc The intended use of this datamight be quite instrumental – for example in choosing a nursing home or hospital – ormight be more symbolic – for example, in a local newspaper publicising findings from aninspection In both cases the intention would be that the use of information from theregulator exerts direct or indirect influence on the provider to comply with regulatoryrequirements For this model to work, it assumes that the information is provided in acontent, form and timescale that these target audiences will find comprehensible andusable, and that they have the capacity and motivation to access and use it.

 Information provision to be used by providers in compliance and improvement Here, the

target audience for information provision is the provider community, and the publication

of information about both poor and good practice may be intended to help themunderstand regulatory requirements and to encourage self-enforced compliance andimprovement For this model to work, it assumes that the regulatory process producesinformation that providers will find useful in understanding and conforming to regulatoryrequirements, and again that they have the capacity and motivation to access and use it

 Information provision as a mechanism for public accountability Here, the publication of

information is an end in itself – making both the regulator itself and regulatedorganisations accountable to the public and demonstrating that the regulatoryarrangements work There is not necessarily an expectation that publication will lead toany particular outcomes, or that there is a functional mechanism at work – publication isthe purpose in itself

It seems that CQC seeks to use mainly the first model in its information provision We arenot aware of any information that has been collected on whether and how much targetaudiences such as patients/service users, families, commissioners and others use CQC’spublished reports However, we would observe that the constraints which have alreadybeen discussed in the sections on compliance, to do with standard setting and the volumeand quality of information collected through the inspection process, may significantly limitits utility for others in their decision making The dichotomous judgements of compliance

or non-compliance cover only a partial set of the essential standards and in any case are notparticularly discriminating since most providers are fully compliant The narrative content

of reports is mostly focused on evidencing compliance or non-compliance, and does notprovide a coherent account of the quality of care or the overall performance of the provider

In short, the reports are designed for use in enforcement by CQC, not for use by otherstakeholders in their decision making The QRP, which may contain a wider data set onperformance, is not published

Reviewing the regulatory model

The purpose of this chapter was to set out the regulatory model in use, alongside commonalternate models, and to outline the consequences and implications of those regulatory

choices Those models are summarised in table 2.1 below, in which the models whichappear to be currently in use in CQC are shaded Overall, it appears that CQC’s currentregulatory model is largely consistent with that of a “safety net” regulator, focused mainly oronly on securing compliance with minimal standards by providers at the bottom end of theperformance distribution through enforcement actions.

Table 2.1 A summary of the regulatory model analysis

Component of

regulation

Regulatory models

Registration Threshold to meet for

performance Relationship buildingwith provider Administrative data capture

Compliance –

standard setting Framing values andexpectations Improvementthrough

self-enforced compliance

Compliance through measurement and enforcement

Differentiating performance of providers

Measure compliance to support enforcement

Measure performance to support improvement

Driver for other providers to improve

enforcement or prospect drives compliance

Enforcement as incentive to drive compliance

Enforcement as symbolic action to drive compliance

Enforcement as driver for other providers to achieve compliance

Information

provision

Information to be used by other stakeholders in decision making

Information to be used

by providers in compliance and improvement

Information as mechanism for public accountability

Even within its own terms, there are some important inconsistencies within the currentregulatory model First, the adoption of generic standards/outcomes with little or no sector-specific definition or guidance makes understanding and interpretation by providers and theregulator difficult, and is likely even with minimal, enforcement oriented regulation to makevalid and reliable measurement problematic Second, the model envisages proportionate orrisk-based regulation with most attention focused on the worst performing providers, but inpractice this has been difficult to operationalise and has now been partly abandoned Third,the use of a generic inspection process and inspection workforce, with no assumed contentknowledge of health and social care or specialist expertise is likely, even with minimal,enforcement oriented regulation to make valid and reliable measurement problematic.Fourthly, the likely utility of information provision is fundamentally constrained by thelimited nature of the data about performance gathered through the regulatory complianceprocess

One consequence of adopting the current regulatory model – focused on “safety net”regulation, compliance with minimum standards, and poor performing providers – is thatthe overall impact of the regulator is likely to be low A lot of regulatory resource isexpended on overseeing the majority of organisations, who are compliant anyway Whileregulatory resource is meant to be concentrated on poorly performing providers, it may bequite difficult for the regulator to secure sustained improvements in their performance, andeven if it does the overall effect on quality in the sector as a whole is probably quite limited.Enforcement is the main tool for securing improvement, but as noted earlier enforcement is

a resource intensive way to secure improvement Overall, this regulatory model may notseem like good value for money to important stakeholders in regulation

CQC’s current strategic review anticipates a number of important changes to the regulatorymodel which has been described in this chapter In particular it proposes that CQC will:

 “Develop a model of regulation based on what drives the greatest improvements in thequality of care”

 “Move towards a model of differentiated regulation This means we will regulatedifferent sectors in different ways To do this we will make greater use of information,including an evaluation of the impact of our regulatory activities.”

 “Need to regulate different services in different ways and at different times to make sure

we achieve the greatest improvement in quality In terms of the frequency and intensity

of inspections, this may mean revisiting our regulatory approach, including adapting ourcurrent annual and bi-annual inspection regime.”

The first of these three changes especially seems to us to imply some profound changes tothe regulatory model, and to each of its components outlined in this chapter andsummarised in table 2.1 With these changes to the regulatory model in mind, the followingchapters of this report examine the evidence base in four areas – differentiation inregulatory design, regulatory standard setting, risk based regulation, and the competencies

of the inspection workforce

in regulatory design

Learning from research

Given the diversity of health and social care services, activities and settings and the diversity

of organisational forms and types which deliver health and social care services, a questionconfronting any organisation that seeks to regulate in this area is whether, in regulatoryterms, “one size fits all” In answering this question, it is helpful to consider the service ororganisational characteristics which may vary, the degree of heterogeneity which may exist,and the implications of that variation or heterogeneity for the purposes of regulation

Of course, it is important to recognise that the mission or purpose of regulation may itselfvary across sectors or organisations Day and Klein (1987) compared British and Americanmodels to nursing home regulation, and found that each was informed by differentphilosophies as well as differences in the structure and nature of the nursing home sector inthe two countries At a basic level, a distinction could be made between a centralised,legalistic approach and a more informal and voluntary one An alternative distinction could

be made between deterrence-based models (which are focused on enforcing the regulatoryrequirements) and compliance-based models (which are focused on developing a workingrelationship with the regulated organisations) Day & Klein summarise these two

“dimensions” in terms of a more general distinction between a technological model (that emphasises administrative control and sophisticated assessment methods) and a social

interaction model (that emphasises social control and assessor judgement) They note that

whether one model, the other, or a combination of the two should be emphasised in a given

regulatory system depends on what the expectations of the system are given the social andpolitical environment.

One way to explore the way that regulatory mission and purpose might vary across sectors is

to use Hood’s (1991) framework for describing the core values of schemes for publicadministration These values are:

 “Keep it lean and purposeful” – this value emphasises the allocation of resources totightly defined tasks Success is defined as frugality, and failure as waste;

 “Keep it honest and fair” – this value emphasises fairness and mutuality Success isdefined as the proper discharge of one’s duties, and failure as abuses of office;

 “Keep it robust and resilient” – this value emphasises reliability and adaptability Success

is defined as robustness, and failure as a breakdown or the presence of risk

Hood argues that any or all of these might be represented in a given administrative scheme,although each has different implications for the design of a scheme and so may conflict witheach other He further alludes to the different value sets being associated with variations inorganisational design; for example, the third set is characterised by having spare capacity fordealing with crises and a culture that encourages learning and creativity The question toconsider is the extent to which each value set is represented within a regulatory scheme,and whether this dictates or is dictated by the nature of the organisations that are beingregulated For example, the regulators of a sector that is especially cost-sensitive mightemphasise efficiency, while regulation in a different sector with greater perceived or actualrisk might focus on robustness and resilience due to the consequences of salient technicalhazards A regulator with responsibility for a number of sectors might legitimately construeits mission and purpose – and so its regulatory design – differently for different sectors

Turning to the characteristics of regulated organisation, one characteristic which has beenseen as important is organisational type – often categorised as public or private; and not-forprofit or for-profit Bartlett and Phillips (1996) observed a shift in healthcare from apredominantly publicly-funded and publicly-provided system towards one based on a mixedeconomy – that is, with increasing levels of input from private and voluntary organisations

In addition, they noted that service provision itself has diversified, with an increasing range

of specialist services Bartlett and Phillips noted that the increase in private sector provisionled to an increase in administrative control of care homes, including revised and more tightlyapplied regulations and the introduction of lay assessors However, at the time this was notmatched by any change to the regulation of NHS long-stay wards, which was comparativelyless comprehensive Hence, a differentiation in regulation between public and private caredeveloped; that this was apparently driven by concerns about exploitation and standards ofcare in the private sector (Bartlett and Phillips, ibid.) suggests that the regulatory approachwas (possibly implicitly or unintentionally) a response to perceived organisationalcharacteristics of the providers

Another characteristic of organisations which may be important is organisational size orscale For example Lindøe, Engen & Olsen (2011) noted that the Norwegian petroleumindustry comprises a limited number of large and bureaucratic organisations; as such, itlends itself well to cooperation with legal authorities and it is easy to identify the partnerswith whom regulators should work Coastal fishing in Norway is somewhat different – thereare large numbers of small fishing boats that are autonomous, informally organised, andtraditionally risk-takers In addition, there is greater public awareness of adverse events inthe petroleum industry than there is of adverse events in coastal fishing Hence, coastalfishing is more resistant to the efforts of state regulators to impose safety standards WhileLindøe et al compared different industries, Nielsen (2006) compared different regulatoryareas in Denmark (county environmental regulation; municipal environmental regulation;fire precautions; occupational safety and health) This study found that inspectors in eacharea used data in different ways when making decisions about the level of sanction to apply

to a given breach: for example, inspectors in occupational safety and health, and those infire precautions, placed more weight on the gravity of a regulatory breach than didinspectors in other areas Meanwhile, inspectors in county environmental regulation andfire precautions placed more weight than did the others on the company’s “track record” ofbreaches and its “will to improve” Nielsen attributes these patterns to differences in thetype of responsiveness applied by inspectors across the different areas For example,inspectors were employing different degrees of “short-memory” responsiveness (to proximalissues concerning the instance of the breach under consideration) versus “long-memory”responsiveness (to distal issues concerning previous interactions with the otherorganisations) A question that Nielsen poses, which is pertinent for the current discussionbut not taken much further in that study, is how these differences might be furtherexplained by the characteristics of each regulatory area – for example, the types ofinstitutions that are involved

Another characteristic of organisations that might affect regulatory design is their systemiccomplexity and risk In other words: some organisations involve technically sophisticatedtasks, performed by specialized professionals who are organised into highly interdependentgroups and departments (in effect, a subset of the wider system described earlier byRasmussen) As Wiig and Lindoe (2009) explain, such characteristics could be ascribed tohospitals, meaning that risk in these settings needs to be viewed in the context of theseinterconnected elements Lynxwiler et al (1983) also noted that in the mining sector, large,complex organisations were believed to have more resources to deal with regulatorydemands, and so were more likely to benefit from leniency on the part of inspectors Hence,regulators need to be capable of recognising and working with the complex interactions thatmay occur within a regulated organisation For example, hospitals may have greatercomplexity than nursing homes In a study of risk factors in pharmacy practice (Phipps etal., 2010), found that community pharmacies were perceived to have particular risks due totheir being commercial enterprises and less “institutionalised” than hospital pharmacies,

and a review of the pharmacy regulator’s disciplinary hearings (Phipps et al., 2011) offeredqualified support for this view – referred pharmacists were more likely to be fromcommunity than hospital pharmacies However, it was not clear whether this was becausecommunity pharmacy was intrinsically “riskier” or because there were more effectivesystems for risk control in hospitals.

When regulators deal with large and complex organisations, there may also be a trade-offbetween the regulatory activity that could be carried out and the regulatory activity that theorganisations involved are able or willing to fund The more a regulator relies on materialand human resources to conduct its activities (for example, because of the administrativeburden associated with regular use of inspection and enforcement measures) the higher thecost will be (Emery et al., 2000) It may be the case that a more complex organisationrequires more resources to carry out an inspection to the same level of thoroughness as isconducted in a less complex organisation Netten et al (1999) recommends that regulatoryfees should be set in a way that reflects cost variations and is transparent to providers.While their argument was made with specific regard to care home regulation, it wouldpresumably be just as valid (and possibly even more so) for a regulator that was coveringdifferent types of healthcare provider

Given that regulatory interactions and organisational characteristics might differ betweenprofessions and sectors of work, the question arises of whether a single regulatory approachcan be adopted for a sector as diverse as healthcare Comparing hospital with nursing homeregulation in the United States, Walshe & Shortell (2004) found that both were similar withregard to the regulatory objectives (to promote high quality care), method of direction (amanual containing the required standards and assessment methods) and method ofdetection (a periodic inspection visit during which the provider is assessed against thestandards) However, they differed in terms of the regulatory models and approaches toenforcement; the hospital regulator is more focused on compliance, emphasisingeducational activities and drawing from the professional expertise of providers andinspectors to facilitate improvement Nursing home regulators, though, are more focused

on deterrence, emphasising the threat or actual application of sanctions in order to forceproviders to make improvements

As these studies suggest, different types of provider appear to lend themselves to differentapproaches However, whether these differences reflect differences in regulatoryrequirements per se between the two sectors, or are a product of differences in their socialand political context, is an interesting question to consider For example, care homes appear

in general to have been regulated in a more authoritarian manner than hospitals However,

is this difference a response to greater difficulties in controlling risks or maintaining quality

in care homes, or is it a product of a greater inequality of power between regulatoryagencies and healthcare providers in this setting? Or might it be due to neither of these, butinstead to a belief that private sector providers are more risky than public sector providers?

In summary, it seems that a range of organisational, service and system and regulatoryagency characteristics, as well as aspects of the wider social and political context, are likely

to result in legitimate and necessary regulatory differentiation Some of the key dimensionsare set out in table 3.1 below Overall, while a single sector regulator, dealing with arelatively homogeneous set of regulated organisations, might well adopt a single regulatorydesign, it seems likely that single sector regulators dealing with highly heterogeneousregulated organisations and multi-sector regulators dealing with diverse sectors willdifferentiate to at least some degree, and the extent to which they do this will reflect trade-offs between

Table 3.1 A typology of factors resulting in differentiation in regulatory design

Social and political context Public views/attitudes to the sector(s)

Past history of quality/service performance Professional and other

Relative power of stakeholders – providers, users, funders and others Regulatory agency Regulatory philosophy

Mission and purpose and how it is construed Level of resourcing for regulation

Organisational form and powers Governance and accountability Industry, service or system

characteristics Nature of service and service users like observability/specification of service, technicality, user autonomy/empowerment

Nature and form of any market or competition/ contestability in services Extent of other forms of control and accountability like choice, voice and exit for users, democratic oversight etc

Numbers of providers – degree of diversification or concentration in service provision

Heterogeneity among providers – degree of variation in organisational characteristics below

Organisational characteristics Size or scale

Ownership and governance Complexity and level of risk Capacity of internal management, organisational development and change capabilities

Comparisons with other regulators

Table 3.2 below sets out a structured summary of the way that four comparator regulators –the Joint Commission for the Accreditation of Healthcare Organisations, the DutchHealthcare Inspectorate, OFSTED and the Homes and Communities Agency – usedifferentiation in their regulatory design

It is immediately evident that all, with the exception of HCA (which is really a single sectorregulator), have quite highly differentiated regulatory arrangements They have differentregulatory standards for different sectors, though some note that it is important to haveconsistency of approach to standard setting, language/terminology, and other aspects

especially where some regulated organisations work across multiple sectors All produceguidance on the interpretation of those standards which, again, is sector specific Theretends to be somewhat greater commonality in their approaches to regulatory processes –with broad similarities for example in methods for undertaking surveys and inspections andfor reporting, though again there is differentiation in response to organisationalscale/complexity and risk – with differences in the intensity and periodicity of inspectionactivities between sectors

There is also rather greater commonality in their approaches to enforcement, perhapsbecause the enforcement powers are generally defined in statute, and so relate to theregulatory agency as a whole rather than to an individual sector However, the two Englishregulators, OFSTED and HCA, both have limited enforcement powers for public sectororganisations and rather greater formal regulatory power over private sector organisations.For example, while OFSTED regulates independent schools according to a set of regulations

in statute, it inspects (not regulates) maintained schools, for which that set of regulations donot apply Having said that, the inspection manuals and processes for maintained andindependent schools have much in common, and OFSTED’s lack of legislative regulatorypowers over maintained schools does not appear to constrain its ability to secure changewhere it is needed

It is notable that all the comparator regulators use regulatory staff who have contentexpertise in the sectors in which they work Some further specialise – for example havingmore experienced staff who deal with the more complex or difficult provider organisations,

or having some inspection staff who undertake particular kinds of activities such as handlinginvestigations or enforcement

Table 3.2 Comparison of approaches to regulatory differentiation

All in healthcare, but include hospitals, long term care, public health, pharmaceuticals/

devices, and health professions

Yes – schools, further education, adult learning, early years/childcare, childrens homes, childrens social care,

adoption/fostering

All in same sector – social housing - but large variations in organisational scale, and include public (LAs), not for profit (housing

associations) and for profit companies.

Yes – some commonality but standards set by sector and overseen

by IGZ History is that IGZ formed from merger of 4 inspectorates in 1995

Yes – quite separate inspection manuals and standards for different sectors

Limited commonality

No, though standards applied differently for larger and smaller RSLs (cut off is <1000 units).

Do they produce

guidance for

different sectors?

Yes – separate accreditation manuals and guidance notes for sectors

Yes – again sector take lead in setting standards, guided and overseen by IGZ.

Yes – emphasis on driving

improvement and spreading good practice through guidance

No – focus is on safety-net regulation, not on driving

Yes – broadly similar inspection processes but different indicator sets and used differently

Yes – similarities in overall inspection process but content, timing and length vary.

Yes for larger and smaller RSLs Smaller ones are not inspected routinely.

Matters as accreditation deemed to satisfy Medicare/Medicaid participation requirements.

Yes – powers vary across sectors (organisational, device/

pharmaceutical and professional regulation).

Yes – note that some sectors are regulated and others inspected (eg no regulations

in statute and no formal enforcement powers)

No – same enforcement powers though use focused on larger RSLs

Yes – inspectors have content background and some specialise in particular

inspection or investigation activities

Yes – inspectors have background in content area.

Regulatory staff mostly have background in social housing or in financial analysis (qualified accountants).

Using existing data

We can examine the extent to which CQC’s current generic regulatory model varies in theway it is used across the sectors it regulates through a range of existing data frominspections, enforcement actions and other sources The purpose of such an examination

is really to raise questions about how much variation there is, and where differences arefound, what their causes might be Such variations in the application of the current genericregulatory model might help to inform decisions about how and where to introduce greaterdifferentiation

 Does the use made of inspection vary across sectors? For example, does the frequency

of inspection or the use of different types of inspection (planned, responsive or up) vary across sectors?

follow- Do rates of compliance with the 16 essential standards vary across sectors? Are somesectors more or less likely to demonstrate overall compliance?

 Does the use of enforcement (compliance actions, warning notices, and otherenforcement actions) vary across sectors? How does the response to enforcement, interms of return to compliance, vary by sector?

These questions might also be asked within sectors – as noted above, some regulatorsdifferentiate their regulatory design based on organisational characteristics such assize/scale, risk, or ownership type For example, existing data could be used to examinewhether compliance rates vary within a sector such as adult social care on factors such asservice/client group type, ownership (public/private, for-profit/not-for-profit, singleprovider/chain of providers, etc)

Table 3.3 below provides an initial exploratory analysis of data on rates of inspection andrates of compliance across six sectors, using data for inspections carried out betweenOctober 2010 and September 2012 It can be seen that there is considerable variation inboth the use of inspections and in rates of compliance For example, the intensity ofinspection (in terms of number of inspections per location in the period), and the use ofresponsive and follow-up inspections varies substantially, and does not necessarily seemrelated to rates of compliance Overall compliance rates vary from 75% to 94% by sector,and there are also obvious variations in compliance rates for some standards Of course,the sectors themselves vary in size and one question to consider might be whether there issubstantial variation within the largest sector of adult social care, and if so how it might behelpful to subdivide or analyse this sector in more detail

Table 3.3 Comparison of inspection and compliance rates across sectors for inspections carried out between October 2010 and September 2012

Social care NHS Independent

healthcare Independentambulance Dental care All sectors

Conclusions

Most regulators – particularly those who are responsible for multi-sector regulation – makemuch more use of differentiation in regulatory design than CQC does at present However,the way that differentiation is used varies, and it seems that regulators often have differentregulatory standards, guidance and staff for different sectors, but try to keep basicregulatory processes, methods and powers common across sectors It is clearly important

to have a robust underlying logic to differentiation, which both explains and justifies thedifferences in use of regulatory resources, intensity of scrutiny and oversight, and response

to non-compliance Table 3.1 provides a useful starting point for that logic, and identifiesthat differentiation is perhaps necessary not just to deal with different sectors (such asnursing homes, hospitals, or homecare providers) differently, but also to differentiate within

a sector on the basis of organisational characteristics like ownership, size/scale, complexity,risk and known past performance or performance trajectory – a theme which we return to inchapter 5 which examines risk-based or proportionate regulation.

In this chapter, we first review the research literature on regulatory standard setting, andseek to use it both to provide a framework for analysis and to identify important commonthemes in standard setting We explore how our four comparator regulators set and usestandards in regulation, and then we turn to asking whether CQC’s existing data could beused to explore how the current standards are set and used We conclude by consideringthe implications for future standard setting and identifying some areas for future researchand evaluation.

Learning from research

The setting of standards – sometimes also called rules, regulations, directives or other terms– is a central feature of regulation (Black 1997) It is the main mechanism by which theregulatory agency communicates its requirements or expectations of the regulatedcommunity, both to regulated organisations individually and collectively and to otherstakeholders including the wider public whom regulation is usually there to protect (Hood et

al 1999) Cornock (2012) considers it important that a regulatory scheme has a clear andachievable purpose that is understood by both regulated organisations and those thatregulation seeks to protect, and standards are an essential tool in defining that purpose Forexample, advocates of regulation may assume that standard setting will led to improvedperformance and value for money, and that the application of standards will lead toimproved accountability (Braye and Preston-Shoot, 1999; Sutherland and Leatherman,2006) Wiener (2003) notes a different assumption underlying “traditional” regulatorymodels, that there is a known minimal acceptable level of care, and that this is represented

by standards, in order to ensure that providers do not fall below it

Before examining the nature and characteristics of regulatory standards, it is worthwhile firstreflecting on their purpose and place in regulation.

First, some regulators adopt a “control” approach, in which the focus is on monitoring anorganisation’s compliance with predefined standards of practice and applying sanctions ifcompliance is not found Others adopt a “support” approach, in which the focus is onadvising and assisting an organisation to set and improve its own standards of practice.Although they may seem to be alternative approaches which are likely to result in quitedifferent types and uses of standardds, and some regulatory policies favour one over theother, they can be used together in practice (Bruhn and Frick, 2011) According to Doddsand Kodate (2011), regulatory approaches in healthcare are the manifestation of two

“institutional logics” One is a logic of accountability in which healthcare professionals andorganisations are accountable to the state and public, and as such should be subject toadministrative controls in which explicit standards and requirements are likely to be central.The other is a logic of organisational learning in which organisations should learn fromprevious mistakes and near misses, and as such they should be encouraged to developmethods of organisational inquiry and learning, such as prospective and retrospectivehazard analysis, and it is rather harder to see a substantial role for explicit standards here Regulators adopting the “control approach” often place considerable emphasis oncompliance – assessing the extent to which a practitioner or organisation meets setstandards of practice This carries the advantage of being a standardised approach, so thatpeople and organisations involved in regulation know what it is expected of those that arebeing regulated and it is, in theory, possible to ensure that regulatory decisions areconsistent However, Wiener (2003) highlights some limitations to a “compliance regime” ofthis kind:

 The standards are not always evidence-based or seen to reflect what is “important” (forexample, they may look at administrative capacity rather than care processes oroutcomes);

 The “rules” might be applied inconsistently across inspectors;

 It can lead providers to focus their energies on meeting minimum requirements ratherthan striving for excellence;

 It can lead to an adversarial relationship between providers and the regulator, which inturn can create a negative public image of the sector;

 It requires a large amount of resources to execute;

 Regulatory sanctions can have a further negative effect on the care that the provider isable to provide to service users