Vladimir Grosboisa,∗, Barbara Häslerb, Marisa Peyrea, Dao Thi Hiepc, Timothée Vergneb a UPR AGIRs, Animal and Integrate Risk Management, International Research Center in Agriculture for
Trang 1Vladimir Grosboisa,∗, Barbara Häslerb, Marisa Peyrea, Dao Thi Hiepc,
Timothée Vergneb
a UPR AGIRs, Animal and Integrate Risk Management, International Research Center in Agriculture for Development (CIRAD), TA C 22/E
Campus International Baillarguet, 34398 Montpellier Cedex 5, France
b Veterinary Epidemiology, Economics and Public Health, The Royal Veterinary College, Hawkshead Lane, North Mymms, Hatfield, Herts
AL9 7TA, United Kingdom
c Center for Interdisciplinary Research on Rural Development, Vietnam National University of Agriculture, Trau Quy, Gia Lam, Hanoi,
Viet Nam
Article history:
Received 3 July 2014
Received in revised form 5 December 2014
Accepted 15 December 2014
Keywords:
Intervention
Disease surveillance
Decision making
Type I error
Type II error
Surveillancesystemsproducedatawhich,onceanalysedandinterpreted,supportdecisions regardingdiseasemanagement.Whileseveralperformancemeasuresforsurveillanceare
inuse,notheoreticalframeworkhasbeenproposedyetwitharationalefordefiningand estimatingeffectivenessmeasuresofsurveillancesystemsinagenericway.Aneffective surveillancesystemisasystemwhosedatacollection,analysisandinterpretation pro-cessesleadtodecisionsthatareappropriategiventhetruediseasestatusofthetarget population.Accordingly,wedevelopedaframeworkaccountingforsampling,testingand datainterpretationprocesses,todepictinaprobabilisticwaythedirectionandmagnitude
ofthediscrepancybetween“decisionsthatwouldbemadeifthetruestateofapopulation wasknown”andthe“decisionsthatareactuallymadeupontheanalysisandinterpretation
ofsurveillancedata”.Theproposedframeworkprovidesatheoreticalbasisfor standard-isedquantitativeevaluationoftheeffectivenessofsurveillancesystems.Weillustratesuch approachesusinghypotheticalsurveillancesystemsaimedatmonitoringtheprevalenceof
anendemicdiseaseandatdetectinganemergingdiseaseasearlyaspossibleandwithan empiricalcasestudyonapassivesurveillancesystemaimingatdetectingcasesofHighly PathogenicAvianInfluenzacasesinVietnamesepoultry
©2015ElsevierB.V.Allrightsreserved
1 Introduction
Knight-JonesandRushton,2013;Otteetal.,2004).These
∗ Corresponding author Tel.: +33 467593833; fax: +33 467593799.
E-mail address: Vladimir.grosbois@cirad.fr (V Grosbois).
Knight-JonesandRushton,2013).Importantly,tocombatanimal
http://dx.doi.org/10.1016/j.prevetmed.2014.12.014
0167-5877/© 2015 Elsevier B.V All rights reserved.
Trang 2management
et al., 2011; Drewe et al., 2012,2015; Hoinville et al.,
2007a).Such evaluationforsystemsaimingatdetecting
Yamamotoetal.,2008;Knight-Jonesetal.,2010).Finally,
2 General overview of the rationale
variables
Trang 3SURVEILLANCE DATA Non-exhausve, non-representave, parally distorted
ASSESSMENT EPIDE MIO LOGICA L SITUATION
PREVEN TIO N/CONTROL MEASURES
That are actually implemented (modalies/intensity)
TRUE EPIDE MIO LOGICA L SITUATION
PREVEN TION/ CONTROL MEASUR ES
That would be implemented given a
perfect knowledge of the epidemiological situaon
Data ana lysis an d interpr etaon
Decision making proc ess
Intervenon strategy
Definedbasedon epidemiological modellingand cost-effecveness and/or cost-benefitanalyses
Data genera on proc ess
Sampling, reporng, diagnosing, tesng
EFFECTIVEN ESS AN D ECONOMIC EFFICIENCY of the risk prevenon/control measuresthatare actually implemented
EFF ECT IVEN ESS AND ECONOMIC
EFFICIENCY
of the prevenon/controlmeasures
that would be implemented given a perfect knowledge of the
epidemiological situaon
Surveilla nce Effecve nes s
Fig 1. Proposed approach for the evaluation of the effectiveness of a surveillance system.
Table 1
Examples of simple intervention strategies for various surveillance objectives.
Surveillance
objective
Monitoring
prevalence
Country/region Yearly
prevalence of a disease (Prev)
Prev ≤ Threshold Do nothing Prev > Threshold Implement
systematic testing in slaughter-houses before products are put on the market Disease case
detection
Herd Disease status No infected
animal in the herd
Do nothing ≥1 infected
animal in the herd
Cull the herd
Demonstrate
freedom
from disease
Country/region Yearly
prevalence of a disease (Prev)
Prev ≤ Threshold Allow
exportations
Prev > Threshold Ban
exportations Early detection
of an
emerging
disease
Country/region Instantaneous
incidence rate (IIR)
IIR = 0 Do nothing IIR > 0 Launch
intensive surveillance and in depth case investigation Limit movements
S−, S + : epidemiological states for which the “no intervention” and “intervention” options, respectively, are required; I−, I + : description of actions associated with to the “no intervention” and “intervention” options.
formthebasisforaninterventiondecision.Therelevant
epidemiologicalscaleisthescaleatwhichdecisionsare
being made about implementing an intervention Such
decisionscanbeforexampletostartvaccinatinganimals
inthetargetpopulationifthediseaseprevalencecrosses
adefinedthresholdornottodoanythingifsurveillance
todocumentfreedomfromdiseasedeliverstheexpected result(i.e.freedom).Thescalecanbeanimal,herd,country,
Trang 4regional orglobal level.The relevant statevariable isa
variable, such as prevalence or incidence, that reflects
the current epidemiological situation, and which value
determines the intervention measures considered as
appropriatebystakeholdersanddecisionmakers.Table1
(Tomassenetal.,2002)
processes
response
3 Illustrations of effectiveness assessment three contrived surveillance system examples and an empirical case study
Trang 5Table 2
Examples of decision making rules relying on the analysis and interpretation of surveillance data The decision rules correspond to the mitigation strategies presented in Table 1
Surveillance
objective
Scale Statistics used
to assess epi-demiological status
Monitoring
prevalence
Country/region Proportion of
positive tests in the samples collected over a year P(+)
P(+) ≤ Threshold Do nothing P(+) > Threshold Implement
systematic testing in slaughter-houses before products are put on the market Case detection
of disease
Herd Result of a
pooled test
Negative test result
Do nothing Positive test
result
Cull the herd Demonstrate
freedom
from disease
Country/region Proportion of
positive tests in the samples collected over a year P(+)
P(+) ≤ Threshold Allow
exportations
P(+) > Threshold Ban
exportations
Early detection
of an
emerging
disease
Country/region Case reporting No case
reported
Do nothing ≥1 case
reported
Launch intensive surveillance and in depth case investigation Limit movements
A − , A + : assessments of epidemiological state for which the “no intervention” and “intervention” options, respectively, are implemented; I − , I + : description
of actions associated with the “no intervention” and “intervention” options.
Table 3
The two types of error used as effectiveness criteria.
True epidemiological status
S + intervention required S−intervention not required Assessment of the epidemiological status resulting from the generation, analysis and interpretation of surveillance data
A − intervention not implemented Type II error
S − , S + : epidemiological states for which the “no intervention” and “intervention” options, respectively, are required; A − , A + : assessments of epidemiological state for which the “no intervention” and “intervention” options, respectively, are implemented.
variablewhichconditions decisionsinterms of
preven-tion/interventionmeasures is usually the prevalenceof
thediseaseinthefocalpopulation.Theprevalence
cate-goriesconsideredasrequiringdistinctinterventionoptions
aredeterminedaccordingtothesocalled“design
preva-lence”.Whenevertheprevalenceinthepopulationisbelow
thedesignprevalence,theterritoryisconsidered“asfree
fromthedisease”(S−)andnomeasuretolimititsspread
is implemented (for instance no limitations to animal
trading:I−)whereaswhenevertheprevalenceinthe
pop-ulationisabovethedesignprevalence,measurestolimit
itsspreadareimplemented(forinstanceanimaltradingis
restricted:I+).Thecrucialaspectofthemitigation
strat-egyisthedeterminationofthedesignprevalence.Itcan
bechosenbasedontherelativelikelihoodofprevalence
levelsgiven thepresenceofthediseaseontheterritory
orbyconsideringhowthemagnitudeofsanitaryand
eco-nomicconsequences ofthepresenceofthediseasevary
as a function of the prevalenceof that disease So the
designprevalencecanbetheminimumexpected
preva-lenceofthediseaseprovideditispresentontheterritory
orthemaximumprevalenceatwhichthesanitaryand eco-nomic consequences of thepresence of thedisease are consideredasnegligible.Thestatisticsusedtoassessthe epidemiological situationfrom surveillancedata is usu-allythebinaryvariablereflectingwhetheratleastonecase hasbeendetected(A+)ornocasehasbeendetected(A−)
Inthenumerouspapersinwhichthisapproachhasbeen used toassess theeffectiveness ofsurveillance systems aimingatdemonstratingthefreedomofaterritoryfrom
adisease(e.g.Martinetal.,2007a;Martin,2008;Frössling
etal.,2009;Hoodetal.,2009;Christensenetal.,2011),the
(Martinetal.,2007b)althoughothermethodshavebeen
Trang 6Table 4
Information for assessing the effectiveness of a contrived surveillance system aiming at monitoring the prevalence of an endemic disease.
Surveillance
objective
Knowing how prevalent is an endemic disease to inform decisions about vaccination strategy
Relevant scale Country (population of 100,000
animals) Relevant
epi-demiological
variable
Individual level prevalence (p)
Intervention strategy S−
p ≤ 0.1
S +
0.1 < p ≤ 0.2
S ++
p > 0.2
I −
no vaccination
I +
vaccination is implemented only in high risk areas
I ++
vaccination is implemented
in all areas
Surveillance
data
generation
process
n = 100 randomly chosen individuals are sampled over a
1 month period (coverage = 0.1%) Each sample
is tested using a test with sensitivity Se = 0.90 and specificity Sp = 0.95 Statistics
computed
from
surveillance
data
Number of sampled units testing positive (n p )
Decision rule 1 (test performances
not accounted for)
A −
n p ≤ 0.1 * n
A +
0.1 * n < n p ≤ 0.2 * n
A ++
n p > 0.2 * n
I −
no vaccination
I +
targeted vaccination is implemented
I ++
vaccination is implemented
in all areas Decision rule 2 (test
performances accounted for)
A−
n p ≤ (0.1 * Se + (1 − 0.1) * (1 − Sp)) * n
A +
(0.1 * Se + (1 − 0.1) * (1 − Sp)) * n
<n p ≤ (0.2 * Se + (1 − 0.2) * (1 − Sp)) * n
A ++
n p > (0.2*Se + (1 − 0.2)*(1 − Sp)) * n
I −
no vaccination
I +
targeted vaccination is implemented
I ++
vaccination is implemented
in all areas
S − , S + , S ++ : epidemiological states for which the “no intervention”, “low intensity intervention” and “high intensity intervention” options, respectively, are required; I−, I + , I ++ : description of actions associated with the “no intervention”, “low intensity intervention” and “high intensity intervention” options, respectively; A−, A + , A ++ : assessments of epidemiological state for which the “no intervention”, “low intensity intervention” and “high intensity intervention” options, respectively, are implemented.
3.2 Monitoringtheprevalenceofanendemicdisease
This section presents a contrived example an active
surveillancesystemaimingatmonitoringprevalenceofa
cattlediseasetoinformdecision-makersonwhich
vacci-nationstrategytoimplementatthenationallevel
3.2.1 Informationrequiredforassessingeffectiveness
Table4summarisestheinformationrequiredtoassess
(Table4).Thedatainterpretationprocessconsistsin
Trang 7Fig 2.Probabilities that data generation, analysis and interpretation processes result in the implementation of different intervention options as a function
of the true epidemiological state n: sample size; Se: sensitivity of the test; Sp: specificity of the test; S−: vaccination is not required; S + : targeted vaccination
is required; S ++ : mass vaccination is required.
0.68
rule1inTable4).InFig.4 testsensitivityandspecificity
system
Trang 8Fig 3.Sensitivity of surveillance effectiveness to changes in sampling and sample testing procedures n: sample size; Se: sensitivity of the test; Sp: specificity
of the test.
Fig 4. Sensitivity of surveillance effectiveness to changes in data analysis and interpretation procedures n: sample size; Se: sensitivity of the test; Sp: specificity of the test.
Trang 9Table 5
Information for assessing the effectiveness of a contrived surveillance system aiming at detecting an emerging or exotic disease early.
Surveillance objective Detecting an emerging disease
following its introduction in a territory as soon as possible Relevant scale Country (population of 10,000
animals) Relevant epidemiological variables Cumulative incidence
(correlated with time elapsed since introduction and spatial spread)
Intervention strategy S−Thediseasehasnotyetbeen
introduced
S + The disease has been introduced but cumulative incidence is <0.5%
S ++ Cumulative incidence is
≥0.5%
I − Keep low intensity surveillance with 50 individuals sampled daily
I + Cull detected infectious cases and reinforce surveillance with
100 individuals sampled daily
I ++ Cull detected infectious cases, reinforce surveillance with 200 individuals sampled daily, and limit animal movements Surveillance data generation process Randomly chosen individuals
are sampled daily Samples are screened for antibody using a test which sensitivity is Se = 0.8 and specificity is Sp = 1.
Seropositive samples are tested for pathogen detection using a test which sensitivity and specificity are 1 Statistics computed from surveillance data Cumulative number of
detected cases (n p )
− No case detected so far A + One case detected A ++ At least two cases detected
I − Keep low intensity surveillance with 50 individuals sampled daily
I + Cull the case if it is infectious and reinforce surveillance with
100 individuals sampled daily
I ++ Cull detected infectious cases, reinforce surveillance with 200 individuals sampled daily, and limit animal movements
S − , S + , S ++ : epidemiological states for which the “no intervention”, “low intensity intervention” and “high intensity intervention” options, respectively, are required; I − , I + , I ++ : description of actions associated with the “no intervention”, “low intensity intervention” and “high intensity intervention” options, respectively; A−, A + , A ++ : assessments of epidemiological state for which the “no intervention”, “low intensity intervention” and “high intensity intervention” options, respectively, are implemented.
importantepidemiologicalstatevariablessuchas
cumu-lativeincidence, spatial spread, and averagenumber of
transmission eventsthat link a case to the index case
Astime elapsedsince theoccurrence of theindex case
increasessodocumulativeincidenceandspatialspread
Consequently,thelatertheimplementationofintervention
measures(relativetothetimeofoccurrenceoftheindex
case)thelargerarethelossesalreadygeneratedbythe
dis-ease.Thisalsomeansthatcostsofinterventionmeasures
requiredwillincreasewithmoreanimalsand/orholdings
beingaffected
Theperformance of a surveillance system aimingat
detectinganemergingpathogenasearlyaspossiblecould
thusbeevaluatedaccordingtotwocomponents:its
abil-itytodetectatanypointintimethepresenceofthefocal
pathogeninthefocalhostpopulationanditsabilityto
eval-uatethespatialspreadandprevalenceofthefocalpathogen
onceitspresencehasbeendetected.Thefirstcomponent
is probably the most important because a surveillance
systemwhich performs well in terms of instantaneous
detectionprobabilitywillallowimplementationof
preven-tion/controlmeasuressoon aftertheintroductionofthe
pathogen,whenthelossesalreadygeneratedbythedisease
aswellastheresourcesrequiredformitigationmeasuresto
controlitsfurtherspreadarelimited.Thesecondcriterion
reflectstheabilityofthesurveillancesystem,once detec-tionhasbeenachieved,toprovideinformationthatallows theimplementationofmitigationmeasureswhichnature and intensitywouldbeconsideredasadaptedby stake-holdersanddecisionmakersgivenperfectknowledgeof therealepidemiologicalsituationintermsofprevalence andspatialspread.Thissecondcomponentisrelatestothe evaluation ofsurveillancesystemsaimingatmonitoring theprevalenceofadisease.Suchattributesarepresented
inSection3.2
Trang 10Fig 5. Effectiveness of a contrived surveillance system aiming at detecting the introduction of an emerging disease as early as possible.
processes