Brandttopost.CCSPSC-Evaluation-Toolkit_NCCRT-Conference-Dec17_long-Read-Only

Colorectal Cancer Screening Program in South Carolina CCSPSC: Connections to the NCCRT Evaluation Toolkit Heather M.. Brandt, PhD, CHES Associate Dean for Professional Development, Grad

Colorectal Cancer Screening Program in

South Carolina (CCSPSC):

Connections to the NCCRT Evaluation Toolkit

Heather M Brandt, PhD, CHES

Associate Dean for Professional Development, Graduate School

Associate Professor, Arnold School of Public Health

University of South Carolina

The Colorectal Cancer Screening Program in South Carolina is funded by the Centers for Disease Control and Prevention (Grant #:

NU58DP006137) The grant is awarded to Drs Heather Brandt and Frank Berger of the Center for Colon Cancer Research at the University of South Carolina Contact: Hiluv Johnson, program coordinator, hsjohnso@mailbox.sc.edu

National Colorectal Cancer Round Table Meeting | December 7, 2017

Colorectal Cancer Screening Program

participation in CRC screening by working with partner

health systems to implement priority evidence-based strategies.

http://cccr.sc.edu/outreach/ccspsc/ccspsc-program

MOA Complete Sites Selection Building Partnership with

Monitor Implementation

Conduct Technical Assistance (TA)

Support Implementation of Evidence-based Strategies

Collect Annual Data Evaluation Activities Annual Review Process

Phase 5

Sustainability and

Maintenance

Connections to the NCCRT

Evaluation Toolkit, Version 4

Access the toolkit at:

http://nccrt.org/resource/evaluation-toolkit/

A few disclaimers…

the evaluation toolkit due to my background in

intervention implementation and evaluation and work with community partners to do this.

the Core for Applied Research and Evaluation (led by

Dr Lauren Workman)

approach, but it is important to note that we are able

to connect our approach to the 7 steps.

– We used the CDC Framework for Program Evaluation in

Public Health

NCCRT Evaluation Toolkit: 7 Steps

• Step 1: Describe and map your program

• Step 2: Prioritize your evaluation questions

• Step 3: Design the evaluation

• Step 4: Identify or develop data collection

instruments

• Step 5: Collect the data

• Step 6: Organize and analyze information

• Step 7: Using and sharing evaluation results

Step 1: Describe and map your

program

activities, and additional activities to increase CRC screening

approach to implementation of the interventions – noted inputs, activities, and outputs (e.g., logic model, protocol, other tools)

MOA Complete Sites Selection Building Partnership with

Monitor Implementation

Conduct Technical Assistance (TA)

Support Implementation of Evidence-based Strategies

Collect Annual Data Evaluation Activities Annual Review Process

Phase 5

Sustainability and

Maintenance

Evidence-Based Interventions

Select at least two priority, evidence-based interventions:

• Provider assessment and feedback

• Provider reminders and recall

• Client (patient) reminders

Optional supportive activities:

Step 2: Prioritize your evaluation

– CRC screening, fidelity to CRC screening guidelines,

satisfaction, implementation of evidence-based

interventions, health system data use and quality

– environmental scan, training and technical assistance,

fidelity to action plans, engagement of stakeholders, policy implementation, use of resources (cost effectiveness),

sustainability

Step 3: Design the evaluation

• Guidance provided by the CDC Colorectal

Cancer Control Program

by the CDC; approved in January 2016

• Used the CDC Framework for Program

Evaluation in Public Health as a guide

• Data collection at multiple points, multiple

sources

• Mixed methods approach

Step 4: Identify or develop data

collection instruments

• CRC screening data – guidance provided by the CDC for

standardized collection

– Baseline, quarterly, and annually

• CRC incidence and mortality data

– Working with the South Carolina Central Cancer Registry

• Process data tools (selected examples):

– Implementation Status Tracking

– Organizational Assessment and Environmental Scan

– Readiness Assessment

– Training Evaluation (pre-test, post-test, follow-up)

– Observation Form (evidence-based intervention specific forms)

– Continuous Quality Improvement (CQI) Tracking tool

• Qualitative data tools: Focus groups/interviews with stakeholders, providers, patients

CCSPSC Readiness Assessment Tool: FQHC Readiness Criteria

Based on R=MC2

(Dymnicki et al., 2014)

Step 5: Collect the data

tools) prescribes when data are collected, how, and by whom

– Iterative and ongoing process of considering what

data must be collected (vs what we would like to

know), how we are collecting the data

– CRC screening data

• Baseline and annual data reported to CDC

• Quarterly data used for our program and with partner FQHCs to monitor short-term outcomes

3/31/18

CDC PY3:

6/30/18

7/1/17-10/1/18

3/31/19

CDC PY 4:

6/30/19

7/1/18-10/1/19

3/31/20

CDC PY5:

6/30/20

7/1/19-10/1/20

3/31/21

Annual CRCS Annual CRCS

Annual CRCS

FQHC System and Sites: PY1

Annual CRCS

FQHC System and Sites: PY2

FQHC System and Sites: PY3

FQHC System and Sites: PY4

CRCS Data

CRCS Data

CRCS Data

CRCS Data

Admin Data

Admin Data

Admin Data

Admin Data

Admin Data

FQHC System and Sites: PY1 FQHC System and Sites: PY1 FQHC System and Sites: PY1 FQHC System and Sites: PY1

FQHC System and Sites: PY4

FQHC System and Sites: PY2 FQHC System and Sites: PY2 FQHC System and Sites: PY2

FQHC System and Sites: PY3 FQHC System and Sites: PY3

FQHC System and Sites: PY1

CareSouth – Lake View

Carolina Health – McCormick

Eau Claire – Eastover

HopeHealth – Kingstree

Little River – Main

New Horizon – Greer

FQHC System and Sites: PY2

CareSouth – Lake View Regenesis –750 Church St.

Eau Claire – Waverly HopeHealth – Timmonsville Little River – Carolina Forest New Horizon – West Faris

Step 6: Organize and analyze

information

• Evaluation team leads data management and analysis

• Protocol specifies processes for managing data

• Evaluation team provides regular updates and offers opportunities for interpretation:

Step 7: Using and sharing evaluation

results

loop)

– Routine updates to partners

• Summary reports of evaluation data provided

– CCSPSC newsletters (quarterly)

– CRC screening data shared (collected baseline,

quarterly, and annually)

– Presentations at professional and scientific meetings

– Baseline data snapshots (next slide)

Example of content from quarterly newsletters

“Take Home” Points

planning – and being intentional in ensuring the

plan meets the needs of the program.

– Evaluation can evolve over time.

accounted for.

• Capacity within partner FQHCs to enter, provide,

retrieve data is important (see also importance of

stakeholder engagement).

* I asked Dr Lauren Workman for her “take home” points

about the evaluation of our program

Working together to increase colorectal cancer screening in

South Carolina!

• Hiluv Johnson, Cindy Calef, Jay Whitmore, Minjee Lee, Amanda Collins

• Core for Applied Research and Evaluation (CARE), Arnold School of Public Health, University of South Carolina (led by Dr Lauren Workman)

• South Carolina Primary Health Care Association

• American Cancer Society

• Colorectal Cancer Prevention Network of the Center for Colon Cancer Research

• Eight federally-qualified health center (FQHC) systems in South Carolina (15 FQHC sites across the eight systems)

University of South Carolina Contact: Hiluv Johnson, program coordinator,

hsjohnso@mailbox.sc.edu