Essentials of Clinical Research - part 7 ppt

Allison Abstract Implementation research is a new scientific discipline emerging from the recognition that the public does not derive sufficient or rapid benefit from advances in the he

removed only if they are correlated with covariates already measured and included

in the model to compute the score.68–70

Instrumental variable analysis is an econometric method used to remove the effects of hidden bias in observational studies.71,72 Instrumental variables are highly correlated with treatment and they do not independently affect the outcome Therefore, they are not associated with patient health status Instrumental variable analysis compared groups of patients that differ in likelihood of receiving a drug.73

Summary

In pharmacoepidemiology research as in for traditional research, the selection of an appropriate study design requires the consideration of various factors such as the frequency of the exposure and outcome, and the population under study Investigators frequently need to weigh the choice of a study design with the quality

of information collected along with its associated costs In fact, new demiologic designs are being developed to improve study efficiency

pharmacoepi-Pharmacoepidemiology is not a new discipline, but it is currently recognized as one of the most challenging areas in research, and many techniques and methods are being tested to confront those challenges Pharmacovigilance (see Chapter 5) as

a part of pharmacoepidemiology is of great interest for decision makers, ers, providers, manufacturers and the public, because of concerns about drug safety Therefore, we should expect in the future, the development of new methods to assess the risk/benefit ratios of medications

5 Glessner MR, Heller DA Changes in related drug class utilization after market withdrawal of

cisapride Am J Manag Care Mar 2002; 8(3):243–250.

6 Griffin JP Prepulsid withdrawn from UK & US markets Adverse Drug React Toxicol Rev Aug 2000; 19(3):177.

7 Graham DJ, Staffa JA, Shatin D, et al Incidence of hospitalized rhabdomyolysis in patients

treated with lipid-lowering drugs JAMA Dec 1, 2004; 292(21):2585–2590.

8 Piorkowski JD, Jr Bayer’s response to “potential for conflict of interest in the evaluation of

suspected adverse drug reactions: use of cerivastatin and risk of rhabdomyolysis” JAMA Dec

1, 2004; 292(21):2655–2657; discussion 2658–2659.

9 Strom BL Potential for conflict of interest in the evaluation of suspected adverse drug

reac-tions: a counterpoint JAMA Dec 1, 2004; 292(21):2643–2646.

10 Wooltorton E Bayer pulls cerivastatin (Baycol) from market CMAJ Sept 4, 2001; 165(5):632.

11 Juni P, Nartey L, Reichenbach S, Sterchi R, Dieppe PA, Egger M Risk of cardiovascular

events and rofecoxib: cumulative meta-analysis Lancet Dec 4–10, 2004;

364(9450):2021–2029.

12 Sibbald B Rofecoxib (Vioxx) voluntarily withdrawn from market CMAJ Oct 26, 2004;

171(9):1027–1028.

13 Wong M, Chowienczyk P, Kirkham B Cardiovascular issues of COX-2 inhibitors and

NSAIDs Aust Fam Physician Nov 2005; 34(11):945–948.

14 Antoniou K, Malamas M, Drosos AA Clinical pharmacology of celecoxib, a COX-2 selective

inhibitor Expert Opin Pharmacother Aug 2007; 8(11):1719–1732.

15 Sun SX, Lee KY, Bertram CT, Goldstein JL Withdrawal of COX-2 selective inhibitors rofecoxib and valdecoxib: impact on NSAID and gastroprotective drug prescribing and utili-

zation Curr Med Res Opin Aug 2007; 23(8):1859–1866.

16 Prentice RL, Langer R, Stefanick ML, et al Combined postmenopausal hormone therapy and cardiovascular disease: toward resolving the discrepancy between observational studies and

the Women’s Health Initiative clinical trial Am J Epidemiol Sept 1, 2005; 162(5):404–414.

17 Dubach UC, Rosner B, Sturmer T An epidemiologic study of abuse of analgesic drugs Effects of phenacetin and salicylate on mortality and cardiovascular morbidity (1968 to 1987)

N Engl J Med Jan 17, 1991; 324(3):155–160.

18 Elseviers MM, De Broe ME A long-term prospective controlled study of analgesic abuse in

Belgium Kidney Int Dec 1995; 48(6):1912–1919.

19 Morlans M, Laporte JR, Vidal X, Cabeza D, Stolley PD End-stage renal disease and

non-narcotic analgesics: a case-control study Br J Clin Pharmacol Nov 1990; 30(5):717–723.

20 Murray TG, Stolley PD, Anthony JC, Schinnar R, Hepler-Smith E, Jeffreys JL Epidemiologic

study of regular analgesic use and end-stage renal disease Arch Intern Med Sept 1983;

143(9):1687–1693.

21 Perneger TV, Whelton PK, Klag MJ Risk of kidney failure associated with the use of

aceta-minophen, aspirin, and nonsteroidal antiinflammatory drugs N Engl J Med Dec 22, 1994;

331(25):1675–1679.

22 Piotrow PT, Kincaid DL, Rani M, Lewis G Communication for Social Change Baltimore, MD:

The Rockefeller Foundation and Johns Hopkins Center for Communication Programs; 2002.

23 Major outcomes in high-risk hypertensive patients randomized to angiotensin-converting enzyme inhibitor or calcium channel blocker vs diuretic: the Antihypertensive and Lipid-

Lowering Treatment to Prevent Heart Attack Trial (ALLHAT) JAMA Dec 18, 2002;

288(23):2981–2997.

24 Pilote L, Abrahamowicz M, Rodrigues E, Eisenberg MJ, Rahme E Mortality rates in elderly patients who take different angiotensin-converting enzyme inhibitors after acute myocardial

infarction: a class effect? Ann Intern Med July 20, 2004; 141(2):102–112.

25 Schneider LS, Tariot PN, Dagerman KS, et al Effectiveness of atypical antipsychotic drugs

in patients with Alzheimer’s disease N Engl J Med Oct 12, 2006; 355(15):1525–1538.

26 Schneeweiss S Developments in post-marketing comparative effectiveness research Clin

Pharmacol Ther Aug 2007; 82(2):143–156.

27 Mellin GW, Katzenstein M The saga of thalidomide Neuropathy to embryopathy, with case

reports of congenital anomalies N Engl J Med Dec 13, 1962; 267:1238–1244 concl.

28 Food and Drug Administration Medwatch Website http://www.fda/gov/medwatch Accessed Aug 20, 2007.

29 Humphries TJ, Myerson RM, Gifford LM, et al A unique postmarket outpatient surveillance

program of cimetidine: report on phase II and final summary Am J Gastroenterol Aug 1984;

79(8):593–596.

30 Stricker BH, Blok AP, Claas FH, Van Parys GE, Desmet VJ Hepatic injury associated with

the use of nitrofurans: a clinicopathological study of 52 reported cases Hepatology May–

June 1988; 8(3):599–606.

31 Martin A, Leslie D Trends in psychotropic medication costs for children and adolescents,

1997–2000 Arch Pediatr Adolesc Med Oct 2003; 157(10):997–1004.

32 Williams P, Bellantuono C, Fiorio R, Tansella M Psychotropic drug use in Italy: national

trends and regional differences Psychol Med Nov 1986; 16(4):841–850.

33 Paulose-Ram R, Hirsch R, Dillon C, Losonczy K, Cooper M, Ostchega Y Prescription and non-prescription analgesic use among the US adult population: results from the third National

Health and Nutrition Examination Survey (NHANES III) Pharmacoepidemiol Drug Saf June

2003; 12(4):315–326.

34 Paulose-Ram R, Jonas BS, Orwig D, Safran MA Prescription psychotropic medication use among the U.S adult population: results from the third National Health and Nutrition

Examination Survey, 1988–1994 J Clin Epidemiol Mar 2004; 57(3):309–317.

35 Strom B Study Designs Available for Pharmacoepidemiology Studies Pharmacoepidemiology.

3rd ed: Wiley; 2000.

36 Risks of agranulocytosis and aplastic anemia A first report of their relation to drug use with special reference to analgesics The International Agranulocytosis and Aplastic Anemia

Study JAMA Oct 3, 1986; 256(13):1749–1757.

37 Wilcox AJ, Baird DD, Weinberg CR, Hornsby PP, Herbst AL Fertility in men exposed

pre-natally to diethylstilbestrol N Engl J Med May 25, 1995; 332(21):1411–1416.

38 Clark DA, Stinson EB, Griepp RB, Schroeder JS, Shumway NE, Harrison DC Cardiac

trans-plantation in man VI Prognosis of patients selected for cardiac transtrans-plantation Ann Intern

41 Donahue JG, Weiss ST, Livingston JM, Goetsch MA, Greineder DK, Platt R Inhaled steroids

and the risk of hospitalization for asthma JAMA Mar 19, 1997; 277(11):887–891.

42 Fan VS, Bryson CL, Curtis JR, et al Inhaled corticosteroids in chronic obstructive pulmonary

disease and risk of death and hospitalization: time-dependent analysis Am J Respir Crit Care

Med Dec 15, 2003; 168(12):1488–1494.

43 Kiri VA, Vestbo J, Pride NB, Soriano JB Inhaled steroids and mortality in COPD: bias from

unaccounted immortal time Eur Respir J July 2004; 24(1):190–191; author reply 191–192.

44 Mamdani M, Rochon P, Juurlink DN, et al Effect of selective cyclooxygenase 2 inhibitors and

naproxen on short-term risk of acute myocardial infarction in the elderly Arch Intern Med.

Feb 24, 2003; 163(4):481–486.

45 Suissa S Observational studies of inhaled corticosteroids in chronic obstructive pulmonary

disease: misconstrued immortal time bias Am J Respir Crit Care Med Feb 15, 2006;

173(4):464; author reply 464–465.

46 Suissa S Immortal time bias in observational studies of drug effects Pharmacoepidemiol

Drug Saf Mar 2007; 16(3):241–249.

47 Suissa S Effectiveness of inhaled corticosteroids in chronic obstructive pulmonary disease:

immor-tal time bias in observational studies Am J Respir Crit Care Med July 1, 2003; 168(1):49–53.

48 Clayton D, Hills M, eds Time-Varying Explanatory Variables Statistical models in

epidemi-ology Oxford: Oxford University Press; 1993:307–318.

49 Sato T Risk ratio estimation in case-cohort studies Environ Health Perspect 1994;

102(8):53–56.

50 van der Klauw MM, Stricker BH, Herings RM, Cost WS, Valkenburg HA, Wilson JH A

pop-ulation based case-cohort study of drug-induced anaphylaxis Br J Clin Pharmacol Apr 1993;

35(4):400–408.

51 Bernatsky S, Boivin JF, Joseph L, et al The relationship between cancer and medication

exposures in systemic lupus erythematosus: a case-cohort study Ann Rheum Dis June 1,

2007.

52 Maclure M The case-crossover design: a method for studying transient effects on the risk of

acute events Am J Epidemiol Jan 15, 1991; 133(2):144–153.

53 Maclure M, Mittleman MA Should we use a case-crossover design? Annu Rev Public Health.

2000; 21:193–221.

54 Marshall RJ, Jackson RT Analysis of case-crossover designs Stat Med Dec 30, 1993;

12(24):2333–2341.

55 Donnan PT, Wang J The case-crossover and case-time-control designs in

pharmacoepidemi-ology Pharmacoepidemiol Drug Saf May 2001; 10(3):259–262.

56 Barbone F, McMahon AD, Davey PG, et al Association of road-traffic accidents with

benzo-diazepine use Lancet Oct 24, 1998; 352(9137):1331–1336.

57 Handoko KB, Zwart-van Rijkom JE, Hermens WA, Souverein PC, Egberts TC Changes in medication associated with epilepsy-related hospitalisation: a case-crossover study

Pharmacoepidemiol Drug Saf Feb 2007; 16(2):189–196.

58 Greenland S A unified approach to the analysis of case-distribution (case-only) studies Stat

Med Jan 15 1999; 18(1):1–15.

59 Scneeweiss S, Sturner TMM Case-crossover and case = time-control designs as alternatives in

pharmacoepidemiologic research Pharmacoepidemiol Drug Saf 1997; 6(suppl 3):S51–59.

60 Suissa S The case-time-control design Epidemiology May 1995; 6(3):248–253.

61 Salas M, Hofman A, Stricker BH Confounding by indication: an example of variation in the

use of epidemiologic terminology Am J Epidemiol June 1, 1999; 149(11):981–983.

62 Stukel TA, Fisher ES, Wennberg DE, Alter DA, Gottlieb DJ, Vermeulen MJ Analysis of observational studies in the presence of treatment selection bias: effects of invasive cardiac management on AMI survival using propensity score and instrumental variable methods

JAMA Jan 17, 2007; 297(3):278–285.

63 D’Agostino RB, Jr Propensity score methods for bias reduction in the comparison of a

treat-ment to a non-randomized control group Stat Med Oct 15, 1998; 17(19):2265–2281.

64 Morant SV, Pettitt D, MacDonald TM, Burke TA, Goldstein JL Application of a propensity

score to adjust for channelling bias with NSAIDs Pharmacoepidemiol Drug Saf June 2004;

13(6):345–353.

65 Ahmed A, Husain A, Love TE, et al Heart failure, chronic diuretic use, and increase in

mor-tality and hospitalization: an observational study using propensity score methods Eur Heart

J June 2006; 27(12):1431–1439.

66 Rosenbaum PR, Rubin DB The central role of the propensity score in observational studies

for causal effects Biometrika 1983; 70(41–55).

67 Rosenbaum PR, Rubin DB Reducing bias in observational studies using subclassification on

the propensity score J AM Stat Assoc 1984; 79:516–524.

68 Austin PC, Mamdani MM, Stukel TA, Anderson GM, Tu JV The use of the propensity score

for estimating treatment effects: administrative versus clinical data Stat Med May 30, 2005;

24(10):1563–1578.

69 Braitman LE, Rosenbaum PR Rare outcomes, common treatments: analytic strategies using

propensity scores Ann Intern Med Oct 15, 2002; 137(8):693–695.

70 Harrell FE Regression Modeling Strategies with Applications to Linear Models, Logistic

Regression and Survival Analysis New York: Springer; 2001.

71 McClellan M, McNeil BJ, Newhouse JP Does more intensive treatment of acute myocardial

infarction in the elderly reduce mortality? Analysis using instrumental variables JAMA Sept

21, 1994; 272(11):859–866.

72 Newhouse JP, McClellan M Econometrics in outcomes research: the use of instrumental

vari-ables Annu Rev Public Health 1998; 19:17–34.

73 Harris KM, Remler DK Who is the marginal patient? Understanding instrumental variables

estimates of treatment effects Health Serv Res Dec 1998; 33(5 Pt 1):1337–1360.

Implementation Research: Beyond

the Traditional Randomized Controlled Trial

Amanda H Salanitro, Carlos A Estrada, and Jeroan J Allison

Abstract Implementation research is a new scientific discipline emerging from the

recognition that the public does not derive sufficient or rapid benefit from advances

in the health sciences.1,2 One often-quoted estimate claims that it takes an average

of 17 years for even well-established clinical knowledge to be fully adopted into routine practice.3 In this chapter, we will discuss particular barriers to evidence implementation, present tools for implementation research, and provide a frame-work for designing implementation research studies, emphasizing the randomized trial The reader is advised that this chapter only provides a basic introduction to several concepts for which new approaches are rapidly emerging Therefore, our goal is to stimulate interest and promote additional in-depth learning for those who wish to develop new implementation research projects or better understand this exciting field

Introduction

Overview and Definition of Implementation Research

Implementation research is a new scientific discipline emerging from the tion that the public does not derive sufficient or rapid benefit from advances in the health sciences.1,2 One often-quoted estimate claims that it takes an average of 17 years for even well-established clinical knowledge to be fully adopted into routine practice.3 For example, in 2000, only one-third of patients with coronary artery dis-ease received aspirin when no contraindications to its use were present.2 In 2003, a landmark study by McGlynn et al estimated that the American public was only receiving about 55% of recommended care.4

recogni-In this setting where adoption lags evidence Rubenstein and Pugh defined implementation research as:

S.P Glasser (ed.), Essentials of Clinical Research, 217

© Springer Science + Business Media B.V 2008

…scientific investigations that support movement of evidence-based, effective health care approaches (e.g., as embodied in guidelines) from the clinical knowledge base into routine use These investigations form the basis for health care implementation science Implementation science consists of a body of knowledge on methods to promote the sys- tematic uptake of new or underused scientific findings into the usual activities of regional and national health care and community organizations, including individual practice sites 5

More recently, Kiefe et al updated the definition of implementation research as:

the scientific study of methods to promote the rapid uptake of research findings, and hence improve the health of individuals and populations 6

Finally, the definition of implementation research may be expanded to encompass work that promotes patient safety and eliminates racial and ethnic disparities in health care

Forming an important core of implementation research, disparities research identifies and closes gaps in health care based on race/ethnicity and socioeconomic position through culturally-appropriate interventions for patients, clinicians, health care systems, and populations.7–10 Under-represented populations make up a signifi-cant portion of the U.S population, shoulder a disproportionate burden of disease, and receive inadequate care.11 In addition, these groups have often been marginal-ized from traditional clinical research studies for several reasons Researchers and participants often do not share common cultural perspectives, which may lead to lack of trust.12 Lack of resources, such as low levels of income, education, health insurance, social integration, and health literacy, may preclude participation in research studies.12

Gaps in health care, such as those described above for vulnerable populations, may be classified as “errors of omission”, or failure to provide necessary care.13 In addition to addressing errors of omission, implementation research seeks to under-stand and resolve errors of commission, such as the delivery of unnecessary or inappropriate care which causes harm In 1999, a landmark report from the Institute

of Medicine drew attention to patient safety and the concept of preventable injury.14

Studies of patient safety have focused on “medical error resulting in an ate increased risk of iatrogenic adverse event(s) from receiving too much or hazard-ous treatment (overuse or misuse)”.13

inappropri-For example, inappropriate antibiotic use may promote microbial resistance and cause unnecessary adverse events Therefore, an inter-governmental task force ini-tiated a campaign in 1999 to promote appropriate prescribing of antibiotics for acute respiratory infections (ARIs).15 In 1997, physicians prescribed antibiotics for 66% of patients diagnosed with acute bronchitis In 2001, based on data from rand-omized controlled trials (RCTs) demonstrating no benefit, guidelines recommended against antibiotic use for acute bronchitis.16,17 Although overall antibiotic use for ARIs declined between 1995–2002, use of broad-spectrum antibiotic prescriptions for ARIs increased.18 A more recent implementation research project success-fully used a multidimensional intervention in emergency departments to decrease antibiotic prescribing.19

In response to what may be perceived as overwhelming evidence that thousands

of lives are lost each year from errors of omission and commission, there have been strong national calls for health systems, hospitals, and physicians to adopt new approaches for moving evidence into practice.20,21 While many techniques have been promoted, such as computer-based order entry and performance-based reim-bursement, rigorous supporting evidence is often lacking

Even though our understanding of implementation science is incomplete, local clinicians and health systems must obviously strive to improve the quality of care for every patient This practical consideration means that certain local decisions must be based on combinations of incomplete empiric evidence, personal experi-ence, anecdotes, and supposition As with the clinician caring for the individual patient, every decision about local implementation cannot be guided by data from

a randomized trial.23,22 However, a stronger evidence base is needed to inform spread implementation efforts Widespread implementation beyond evidence raises concern about unintended consequences and opportunity costs from public resources wrongly expended on ineffective interventions.22

wide-To generate this evidence base, implementation researchers use a variety of techniques, ranging from qualitative exploration to the controlled, group- randomized trial Brennan et al described the need to better understand the ‘basic science’ of health care quality by applying methods from such fields as social, cognitive, and organizational psychology.24 Recently, Berwick emphasized the importance of understanding the mechanism and context through which implementation tech-niques exert their potential effects within complex human systems.25 Berwick cautioned that important lessons may be lost through aggregation and rigorous scientific experimentation, challenging the implementation research community to reconsider the basic concept of evidence, itself Interventions for translating evi-dence into practice must operate in complex, poorly understood environments with multiple interacting components which may not be easily reducible to a clean, sci-entific formula Therefore, we later present situational analysis as a framing device for implementation research Nonetheless, in keeping with the theme of this book,

we mainly focus on the randomized trial as one of the many critical tools for mentation research

imple-In summary, implementation research is an emerging body of scientific work seeking to close the gap between knowledge generated from the health sciences and routine practice, ultimately improving patient and population health outcomes Implementation research, which encompasses the patient, clinician, health system, and community, may promote the use of needed services or the avoidance of unneeded services Implementation research often focuses on patients who are vul-nerable because of race/ethnicity or socioeconomic position By its very nature implementation research is inter-disciplinary

In this chapter, we will discuss particular barriers to evidence implementation, present tools for implementation research, and provide a framework for designing implementation research studies, emphasizing the randomized trial The reader is advised that this chapter only provides a basic introduction to several concepts for

which new approaches are rapidly emerging Therefore, our goal is to stimulate interest and promote additional in-depth learning for those who wish to develop new implementation research projects or better understand this exciting field.

Overcoming Barriers to Evidence Implementation

Although the conceptual basis for moving evidence into practice has not been fully developed, a solid grounding in relevant theory may be useful to those designing new implementation research projects.26 Many conceptual models have been devel-oped in other settings and subsequently adapted for translating evidence into prac-tice.27 For example, implementation researchers frequently apply Roger’s theory describing innovation diffusion Rogers proposed three clusters of influence on the rapidity of innovation uptake: (1) perceived advantages of the innovation; (2) the classification of new technology users according to rapidity of uptake; and, (3) contextual factors.28 First, potential users are unlikely to adopt an innovation that

is perceived to be complex and inconsistent with their needs and cultural norms Second, rapidity of innovation uptake often follows a sigmoid-shaped curve, with an initial period of slow uptake led by the ‘innovators.’ Next follows a more rapid period of uptake led by the early adopters, or ‘opinion leaders.’ During the last adoption phase, the rate of diffusion again slows as the few remaining ‘laggards’ or traditionalists adopt the innovation Finally, contextual or environmental factors such

as organizational culture exert a profound impact on innovation adoption, a concept which is explored in more detail in the following sections of this chapter

Consistent with the model proposed by Rogers, multiple barriers often work synergistically to hinder the translation of evidence into practice.29 Interventions often require significant time, money, and staffing Implementation sites may expe-rience difficulties in implementation from limited resources, competing demands, and entrenched practices The intervention may have been developed and tested under circumstances that differ from those at the planned implementation site The implementation team may not adequately understand the environmental character-istics postulated by Roger’s diffusion theory as critical to the adoption of innova-tion Because of such concerns a thorough environmental analysis is needed prior

to widespread implementation efforts.29

Building upon models proposed by Sung et al.30 and Rubenstein et al.,5 Fig 13.1 depicts the translational barriers implementation research seeks to overcome The first translational roadblock lies between basic science knowledge and clinical tri-als The second roadblock involves translation of knowledge gained from clinical trials into meaningful clinical guidance, which often takes the form of evidence-based guidelines

The third roadblock occurs between current clinical knowledge and routine practice, carrying important implications for individual practitioners, health care systems, communities, and populations Given the expansive nature of this third roadblock, a multifaceted armamentarium of tools is required One tool, industrial-

style quality improvement, described below in more detail, operates at the level of the clinical microsystem, the smallest, front-line functional unit that actually deliv-ers care to a patient.31 Clinical microsystems consist of complex adaptive relation-ships among patients, providers, support staff, technology, and processes of care

To achieve sustainable success, researchers seeking to overcome this third tional barrier need to be effective advocates for changes in local and governmental health policy Finally, implementation research may inform clinical trials and basic science

transla-To promote the spectrum of research depicted in Fig 13.1, the 2003 NIH Roadmap acknowledges translational research as an important discipline.32 In fact, several branches of the NIH now have open funding opportunities for implementa-tion research The integration of research findings from the molecular to the popu-lation level is a priority The Roadmap seeks to join communities and interdisciplinary academic research centers to translate new discoveries into improved population health.33

Implementation Research Tools

The tools used to translate clinical evidence into routine practice are varied, and no single tool or combination of tools has proven sufficient or completely effective Furthermore,

it may not be the tool itself but how it is implemented in a system that drives change.34

Basic Science

Knowledge

Current Clinical Knowledge

Clinical Trials

Health Care Systems

Early Adoption

Widespread Adoption

1 st Translational

Block

2 nd Translational Block

3 rd Translational Block

*Industrial-style Quality Improvement

Fig 13.1 Translational blocks targeted by Implementation Research

In fact, this lack of complete effectiveness spurs implementation research to develop innovative adaptations or combinations of currently available tools.35

Below, we provide an overview of available tools, which are intended as basic building blocks for future implementation research projects Although different classification systems have been proposed,36 we arranged these tools by their focus:

on the patient, the community, the provider, and the healthcare organization We acknowledge that this classification is somewhat arbitrary because several imple-mentation tools overlap multiple categories

Patient-Based Implementation Tools

A growing body of evidence suggests that patients may be successfully ‘activated’

to improve their own care For example, a medical assistant may review the medical record with the patient and encourage the patient to ask questions at an upcoming visit with the physician Patients exposed to such programs had better health out-comes, such as improved glycemic control for those with diabetes.37,38 In another study, a health maintenance reminder card presented by the patient to the physician

at appointments significantly increased rates of influenza vaccination and cancer screening.39

Other interventions have taught disease-management and problem solving skills to improve chronic disease outcomes Teaching patient self-management skills is more effective than passive patient education, and these skills have been shown to improve outcomes and reduce costs for patients with arthritis and asthma.40 As part of the ‘col-laborative model,’ self-management is encouraged through better interactions between the patient, physician, and health care team The collaborative model includes: (1) identifying problems from the joint perspective of the patient and clini-cal care team; (2) targeting problems, setting appropriate goals, and developing action plans together; (3) continuing self-management training and support services for patients; (4) active follow up to reinforce the implementation of the care plan.40

Community-Based Implementation Tools

The Community Health Advisor (CHA) model has been implemented throughout the world to deliver health messages, promote positive health behavior change, and facilitate access to the health care system.41 Based on the CHA model, community members, usually without formal education in the health professions, undergo spe-cial training and certification CHA interventions have been used to promote pre-vention and treatment for a large array of conditions, including cancer, asthma, cardiovascular disease, depression, and diabetes CHA programs have also been developed to decrease youth violence and risky sexual behavior CHA interventions

may be especially relevant for underserved populations and those living in rural areas Although promising, CHA interventions often rely on volunteer workers who may be vulnerable to stress and burnout from work overload Also, intense training and oversight is often required to assure the accuracy of the health messages being transmitted A review by Swider found limited high-quality evidence that CHA interventions actually improve health outcomes Swider also called for additional rigorous research on the effectiveness and underlying mechanisms through which CHA interventions work.42 A more recent review commissioned by the Robert Wood Johnson Foundation found that specific CHA interventions may reduce health disparities, particularly for patients with hypertension and diabetes.9

Provider-Based Implementation Tools

Clinical Guidelines

Clinical guidelines have been defined as “systematically developed statements to assist practitioners’ and patients’ decisions about appropriate health care for spe-cific clinical circumstances.”43 In a systemic review of implementation strategies spanning the last 30 years, Grimshaw et al noted guideline dissemination efforts may lead to modest improvements in care.44 However, guideline dissemination alone is not sufficient for implementation.45

For many clinical situations encountered today, thousands of evidence-based guidelines and practice recommendations have been published Such sheer volume often precludes the individual practitioner from implementing all recommendations for every patient As an example, Boyd et al noted that if one were to treat a hypo-thetical 79 year old woman with diabetes, chronic obstructive pulmonary disease (COPD), hypertension, osteoporosis, and osteoarthritis, and follow all recom-mended guidelines for her multiple co-morbidities, the patient would require 12 medications at a cost of $406 per month.46

Continuing Medical Education

Continuing medical education (CME), a requirement for ongoing medical sure, has traditionally relied on text-based, didactic methods to promulgate clinical information However, passive, text-based educational materials and formal CME conferences do not lead to measurable improvements in practice patterns.47,48

licen-Rather, CME using interactive techniques which actively engage physicians are more effective in improving practice patterns and patient outcomes.49 Physicians who reflect on their own individual performance may identify areas for improve-ment and seek CME through multifaceted, self-directed learning opportunities Modalities that promote active learning – such as case-based problem solving – have shown to produce modest improvements in clinical practice.50

With the advantages of being convenient, flexible, and inexpensive, the Internet has become a useful platform to reach a wider audience for interactive CME Fordis

et al conducted a randomized controlled trial comparing live, small-group tive CME workshops with Internet CME.51 Both groups focused on cholesterol management All physicians received didactic instruction, interactive cases with feedback, practice tools and resources, and access to expert advice Knowledge scores for physicians in the Internet CME group increased more than scores for those in the live CME group Additionally, the online CME group demonstrated a statistically significant improvement in appropriate drug treatment for high-risk patients Success of the Internet CME may have been partially driven by the partici-pants’ ability to repeatedly return to the website for reinforcement and the ability

interac-to structure the learning experience interac-to meet individual needs

Academic Detailing

Academic detailing relies on site visits to physicians’ offices for intense ship building and one-on-one information delivery Important components for suc-cessful detailing include: (1) assessment of baseline knowledge and motivations for current behavior; (2) articulating clear objectives for education and behavior; (3) gaining credibility with ties to respected organizations through ongoing rela-tionship building; (4) encouraging physicians to actively participate in educational interventions; (5) using graphic representations for educational materials; (6) focusing on a limited number of ‘take-home’ points; and, (7) supplying positive reinforcement for improved behaviors during follow up.52 Representatives from pharmaceutical companies have effectively used academic detailing to boost prod-uct sales In a systematic review, academic detailing yielded modest effects; how-ever, significant resources were needed to sustain these projects.44

relation-Opinion Leaders

Several implementation programs have relied on influential colleagues Opinion leader strategies may include using celebrities, employing people in leadership positions, and asking those doing front-line work to refer ‘up the ladder.’ Studies examining the effectiveness of opinion-leader strategies have produced both posi-tive and negative findings, and the precise mechanism for how change is accom-plished remains elusive.53

Physician Audit and Feedback

The utility of audit and feedback hinges on developing credible data-driven maries of how patient populations are being managed In theory, such reports may prompt clinicians to reflect on their personal clinical practices and motivate

subsequent improvement Performance feedback may focus on outcomes (such as percentage of patients with diabetes who have achieved glycemic control) or proc-ess (such as the percentage of patients with diabetes for whom the physician meas-ured glycemic control) The credibility of performance feedback relies on the ability to capture the many clinical nuances that the physician must consider when delivering care to the individual patient Because the difficulties in capturing these clinical nuances have not yet been completely surmounted, comparisons of per-formance to a data-driven, peer-based benchmark may be more appropriate than comparison to an arbitrary standard of perfect performance Kiefe et al found that feedback with peer-based benchmarks led to better quality of care, but other studies have reported mixed or modest results.44,54

Organization-Based Implementation Tools

Industrial-Style Quality Improvement

This type of improvement activity originated outside of health care and has acquired such labels as Total Quality Management (TQM) and Continuous Quality Improvement (CQI) These approaches make two fundamental assumptions: (a) that poor outcomes are attributable to system failures, rather than lack of indi-vidual effort or individual mistakes, and (b) achieving improvement and excellence, even in the absence of system failures, is possible through iterative cycles of plan-ning, acting, and observing the results In general, complex systems must have built-in redundancy to function well If an individual makes a mistake at one point

in the system, checks and balances built into other parts of the system may prevent

an adverse event However, as described in the example below, patient safety may

be endangered by simultaneous failure of multiple system components, thus ing built-in redundancy

defeat-As a simple example, multiple mechanisms should be in place to ensure that incompatible blood products are not given to hospitalized patients Delivery of the wrong blood type to a patient requires failure at multiple points, including prepara-tion of the blood in the blood bank and administration of the blood by the nurse Taking such a systems approach stands in stark contrast to blaming individuals, thereby avoiding low morale and reluctance to disclose mistakes

Improvement activity usually proceeds through a series of ‘plan-do-study-act’ cycles These cycles emphasize measuring the process of clinical care delivery at the level of the clinical microsystem, which has been previously described Here, small amounts of data guide the initial improvement process The process empha-sizes small, continuous gains through repeated cycles and does not rely on the statistical significance of the measurements Although many health care institutions have adopted such methodology based on compelling case studies, additional studies with high-quality experimental methods are still needed.55

Systems Reengineering

Instead of incremental changes to clinical microsystems, major redesign of the entire system may be undertaken For example, in the 1990s the Veterans’ Health Administration (VHA) undertook a major reengineering of its health care system, focusing on the improved use of information technology (IT), the measurement and reporting of performance, and the integration of services.56 By 2000, the VHA had made statistically significant improvements in nine areas, including preventive care, outpatient care (diabetes, hypertension, and depression), and inpatient care (acute myocardial infarction and congestive heart failure) Additionally, the VHA performed better than the fee-for-service Medicare system on 12 of 13 quality measures.56 Because systems engineering requires changes on such a large scale, little evidence exists about its efficiency and effectiveness in yielding more improvements than smaller changes.3

Computer-Based Systems

Computer-based systems target links in the process of care delivery that are most prone to human error Such systems may provide clinical decision support by assisting the clinician with making a diagnosis, choosing among alternative treat-ments, or deciding upon a particular drug dosage Other functions may include delivery of clinical reminders and computerized provider order entry (CPOE).57

A systematic review documented improvements in time to therapeutic goals, decreases in toxic drug levels and adverse reactions, and shorter hospital stays.58

However, adverse effects of computer-based systems have also been reported, including increased mortality rates, increased rates of adverse drug reactions, delays in medication administration, increased work load, and new types of errors.59–62 These data illustrate that adverse drug reactions may be either increased

or decreased after the introduction of computer-based systems Therefore, computer-based systems should not be implemented without safeguards to prevent unintended consequences We need more work to better understand how computer-based systems interact with human users and the complex health care environment and how these interactions affect quality, safety, and outcomes

Public Report Cards

Public reports on the quality of health care delivered by institutions are ing For example, public reports may focus on risk-adjusted mortality after cardiac surgery or quality at long-term care facilities In addition, such reports will proba-bly be expanded to include physician groups and individual physicians Public reports are often promoted under the assumption that the public will use them to

proliferat-choose high-quality providers, thus better enabling a competitive ‘medical marketplace.’ However, this promise has yet to be realized Although scant evi-dence links report cards to improved health care, report cards may have profound adverse effects: (1) physicians may avoid sicker patients to improve their ratings; (2) physicians may strive to meet the targeted rates for interventions even in situa-tions where intervention is inappropriate; and, (3) physicians may ignore patient preferences and neglect clinical judgment.63 Even worse, report cards may actually widen gaps in health disparities.64

Pay-for-Performance (P4P)

Currently, there is mounting pressure to tie reimbursement for health care services

to quality measurement Although allowing market forces to freely operate through P4P reimbursement may seem logical, systematic reviews have not yielded conclu-sive results Because not everything that is important is currently measured, linking reimbursement to measured quality may divert attention from important, but unmeasured aspects of care (i.e., ‘spotlight’ effect) As with public reporting, P4P may actually widen health disparities, although empiric data are lacking

To date, evidence informing the effectiveness of P4P in improving the delivery

of health care is limited One study found that when implemented in physician practice groups, P4P produced improvements for those with higher baseline per-formance but had minimal effect on the lowest performers.65 Glickman et al found hospitals voluntarily participating in the P4P initiative for myocardial infarction did not show appreciable improvement.66 A recent study found that hospitals participat-ing in P4P and public reporting programs sponsored by the Centers for Medicare and Medicaid Services had slightly greater improvements in quality than those only participating in the public reporting program.67 Several ongoing studies may soon deliver new insights about P4P

Advancing Implementation Science

Because the implementation science base is still emerging, researchers have at their disposal an array of tools which are variously effective depending upon the patient population and delivery setting Moving beyond the tools described above, we need

to develop innovative adaptations and approaches to bridge the gap between clinical knowledge and health care practice We need to test the effectiveness of these new approaches with rigorous scientific methods to avoid adverse consequences from the wide-spread dissemination and adoption of unproven interventions.22 Therefore, in the remainder of this chapter, we discuss the critical design elements for implemen-tation randomized controlled trials, followed by an example of an implementation research study

Designing Implementation Research Studies

Overview of Implementation Research Study Design

Multiple designs are available for implementation research projects Somewhat analogous to the traditional clinical trial, randomized designs for implementation research allow causal inference and offer protection from measured and unmeas-ured confounding.35 As described below in more detail, such designs include an active intervention, random allocation to a comparison or intervention group, and blinded assessment of objective endpoints

Falling lower in the hierarchy of evidence, implementation studies may use designs that are neither randomized nor controlled For example, a research team may observe a single group for changes in health care delivery or patient outcomes before and after intervention implementation In this case, the observed changes may result from multiple factors not associated with the intervention Secular trends, such as increasing use of specific medications, may produce broad, population-based changes, irrespective of the intervention under study Without a comparison group, secular trends may be confused with intervention effects.35 Interrupted time-series designs use advanced statistical methodology with data collected from multiple points in time before and after the intervention to better account for secular trends

In addition to confounding from secular trends, uncontrolled study designs are susceptible to other ‘non-interventional’ aspects of the intervention For example,

an intervention may bestow more attention on patients or clinicians through data collection, leading to self-reported improvement through placebo-like effects Comparison groups, even without randomization, offer important protection against secular trends and placebo-like effects Non-randomized allocation to intervention and comparison groups does not assure that both groups are similar in all important characteristics Matched study designs may balance study groups for a limited number of measured characteristics In contrast, successfully implemented rand-omization equalizes recognized and unrecognized confounders across study groups and is, therefore, essential for cause-and-effect inference

In summary, limitations of study designs without randomization or a comparison group include difficulty establishing causality, confounding, bias, and spurious associations from multiple comparisons.23 Although such studies are generally considered to be lower within the evidence hierarchy, they may provide useful information when randomized controlled trials (RCTs) are not feasible or generate important hypotheses for subsequent testing with more rigorous study designs In keeping with the theme of this book, we focus the remainder of this chapter on RCTs for implementation research In contrast to the traditional clinical RCT, implementation studies frequently randomize groups (clusters) rather than individ-uals Therefore, we place particular emphasis on the cluster RCT.68 Because imple-mentation studies typically involve a complex set of design issues, we strongly recommend that investigators obtain expert consultation with methodologists and statisticians during the planning stages, rather than postponing this activity until after the intervention has been completed and the data are ready to analyze

Implementation Randomized Controlled Trials

Many principles for the design of high-quality, traditional RCTs discussed where in this book also apply to implementation research In contrast, the following discussion emphasizes particular facets of the implementation RCT that may diverge from the more traditional clinical trial As a discussion guide, our approach

else-is approximately parallel to the Consolidated Standards of Reporting Trials (CONSORT), which were designed to encourage high-quality clinical randomized trials and promote a uniform reporting style The CONSORT criteria emphasize the ability to understand the flow of all actual and potential research participants through the experimental design Although originally designed for the traditional or

‘parallel’ clinical trial,69,70 the CONSORT criteria were subsequently modified for the cluster RCT.71,72 Finally, an exhibit at the end of the discussion provides a spe-cific example of an implementation randomized trial

Participants and Recruitment

In contrast to the randomized clinical trial where patients are the unit of tion and analysis, implementation randomized trials have a broader reach For example, key participants in implementation RCTs may be doctors, patients, clinics,

interven-or hospitals, interven-or hospital wards Because implementation research is conducted in the ‘real world’ and often seeks to engage busy clinicians who are otherwise over-whelmed with their usual activities, recruitment may be particularly difficult Therefore, recruitment protocols for implementation research demand careful con-sideration and may require a dedicated recruitment and retention team Often mul-tiple options (e.g., word of mouth, e-mail, phone, fax, personal contacts, or lists from professional organizations) must be pursued, and still the desired number of participants may not be reached

Human Subjects

The need for approval of implementation studies by an institutional review board (IRB) has sometimes been questioned under the assumption that the work is being performed for local quality improvement and not for research However, randomi-zation is not generally used for local quality improvement projects In addition, the intention to publish study findings in the peer-reviewed literature or present at national scientific conferences clearly places the work in the research domain Although IRB review is always required for implementation research, the research protocol may pose minimal danger to participants, and the review may be con-ducted under an expedited protocol We refer the reader to more detailed reviews

on this topic.73–75

Investigators designing cluster RCTs must carefully consider the ethical issues that arise when consent occurs at the cluster level with subsequent enrollment of participants within the cluster If the target of the research is clearly the clinician, informed consent may often be waived for the patient For studies that focus on the clinician but collect outcomes from medical record review or administrative patient records, the researchers may consider applying for a waiver of informed patient consent Such waivers are especially reasonable when a large volume of patient records would make patient informed consent impractical Implementation research usually generates personally identifiable health information, which may be subject

to the Health Insurance Portability and Accountability Act (HIPAA) Waiver of HIPAA consent by the patient may often be obtained based on requirements similar

to waiver of informed patient consent Finally, it may be necessary to obtain sent from both patients and providers if the intervention targets both populations.Investigators should develop detailed plans to protect the security and confidential-ity of study data Data should be housed in physically secured locations with strong logical protection, such as password protected and encrypted files Access to study data should be only on a ‘need-to-know’ basis Participant identifiers should be main-tained only as necessary for data quality control and linkage Patients and clinicians should be assured that personal information will not be revealed in publications or presentations Data integrity should also be protected with detailed protocols for veri-fication and cleaning, which are beyond the scope of this chapter.76

con-We agree with the International Committee of Medical Journal Editors (ICMJE) that descriptions of all randomized clinical trials should be deposited in publically available registries before recruitment begins.77 The ICJME includes interventions focusing on process-of-care within the rubric of clinical trials Trial registries guard against the well-recognized bias that negative studies are less likely to be published than positive studies Negative publication bias may significantly limit meta-ana-lytic studies, leading to the false conclusion that ineffective interventions are actu-ally effective Registries also increase the likelihood that participation in clinical trials will promote the public good, even if the study is negative Although the tem-plate is not customized for implementation research, one such registry may be found at http://clinicaltrials.gov

Intervention Design

The previously described tools may serve as useful starting points for an innovative intervention design, which is often achieved using a formative-evaluation proc-ess.78,79 Formative evaluation incorporates input from end users to refine an inter-vention during the early stages of development Following this approach, Glasgow

et al recommend key features to include in the content design: (1) barrier analysis;(2) integration of multiple types of evidence; (3) adoption of practical trials that address clinician concerns; (4) investigation of multiple outcomes, generalizability, and contextual factors; (5) design of multilevel programs using systems and social