OKSOC Using the Ohio Scales for Assessment and Measurement 2013 Tampa Presentation

O KLAHOMA S TATEWIDE S YSTEMS OF C ARE E VALUATIONOklahoma Systems of Care OSOC Expansion Implications for Evaluation from Phase IV and Phase VI SAMHSA grants, Oklahoma’s wraparound pr

Trang 1

Using the Ohio Scales for Assessment and Outcome Measurement in a

Statewide System of Care

Trang 2

O KLAHOMA S TATEWIDE S YSTEMS OF C ARE E VALUATION

Oklahoma Systems of Care

(OSOC) Expansion

Implications for Evaluation

from Phase IV and Phase VI SAMHSA grants, Oklahoma’s

wraparound program expanded from an initial 6 counties to 56.

from the national evaluation and longitudinal study.

longitudinal study.

of youths in OSOC wraparound.

collection from enrolled families.

 A scalable solution would incorporate line staff as data collectors,

so burden – in time and training – had to be limited.

Trang 3

Pushmataha

Hughes

Coal Pittsburg Latimer Leflore

Haskell McIntosh

2006 2007 2008 2009 2010 2000

Trang 4

Trang 5

Development of State Evaluation

 Psychometric tool originally in use would not scale

 Costs and logistics related to credentialing requirements

were prohibitive.

 Evaluation committee of evaluators, staff, families and

youth developed the state evaluation tool set – based on

extracts from longitudinal study public domain

instruments – in spring and summer of 2004.

 2003/2004 national conferences exposed evaluation team members to the Ohio Scales, in use by other Systems of

Care communities

 State evaluation committee chose to build OSOC

Assessments around the Ohio Scales in July 2004.

Trang 6

Why the Ohio Scales?

 Originally designed as a statewide instrument

 Qualities appropriate to statewide implementations

with follow up at 6 month intervals

Youth—Administered to youth aged

12 and older at baseline with follow up at 6 month intervals Worker—Administered at baseline with follow up at 3 months and then 6 month intervals from baseline

 Good and growing research base

Articles on flash drive.

Trang 7

Instructions: Please rate the degree to which your child has experienced the

following problems in the past 30 days

3 Yelling, swearing, or screaming at others 0 1 2 3 4 5

5 Refusing to do things teachers or parents ask 0 1 2 3 4 5

8 Breaking rules or breaking the law (out past curfew, stealing) 0 1 2 3 4 5

11 Can’t seem to sit still, having too much energy 0 1 2 3 4 5

12 Hurting self (cutting or scratching self, taking pills) 0 1 2 3 4 5

13 Talking or thinking about death 0 1 2 3 4 5

15 Feeling lonely and having no friends 0 1 2 3 4 5

17 Worrying that something bad is going to happen 0 1 2 3 4 5

Ohio Scales: Youth Problem Scale

Produces a score of 0 - 100

Externalizing Subscale

Internalizing Subscale

Delinquency Subscale

Trang 8

Ohio Scales: Youth Functioning Scale

Instructions: Please rate the degree to which your child’s problems affect his or her

current ability in everyday activities Consider your child’s current level of

3 Dating or developing relationships with boyfriends or girlfriends 0 1 2 3 4

4 Getting along with adults outside the family (teachers, principal) 0 1 2 3 4

5 Keeping neat and clean, looking good 0 1 2 3 4

6 Caring for health needs and keeping good health habits (taking medicines or brushing teeth) 0 1 2 3 4

7 Controlling emotions and staying out of trouble 0 1 2 3 4

8 Being motivated and finishing projects 0 1 2 3 4

9 Participating in hobbies (baseball cards, coins, stamps, art) 0 1 2 3 4

10 Participating in recreational activities (sports, swimming, bike riding) 0 1 2 3 4

11 Completing household chores (cleaning room, other chores) 0 1 2 3 4

12 Attending school and getting passing grades in school 0 1 2 3 4

13 Learning skills that will be useful for future jobs 0 1 2 3 4

15 Thinking clearly and making good decisions 0 1 2 3 4

16 Concentrating, paying attention, and completing tasks 0 1 2 3 4

17 Earning money and learning how to use money wisely 0 1 2 3 4

18 Doing things without supervision or restrictions 0 1 2 3 4

19 Accepting responsibility for actions 0 1 2 3 4

Produces a score of 0 - 80

Resiliency Subscale

Trang 9

Ohio Scales

Guidelines from the Texas Study

2003 study by the Texas Department of Mental Health and

Mental Retardation used comparisons with CBCL and SDQ to validate the scales and to establish classification criteria

for the 20-item Ohio Problems and Functioning scales:

 Score of 25 and above = critical impairment

 Score of 17 – 24 = borderline impairment

 Decrease of 11 or more points = clinically significant improvement

 Score of 44 and below = critical impairment

 Score of 45 – 53 = borderline impairment

 Increase of 8 or more points = clinically significant improvement

Trang 10

Significant Improvement in Ohio Scales

Problems and Functioning Scores

Benchmark Value = 70-80% of 'Impaired' Youths Should Show Significant Improvement at 6 Months (-11 points for Problems, +8 for Functioning)

Site

# Youths with Baseline and 6-Month Assessments

# Youths with Impaired Baseline Scale Scores

% Youth with Baseline Impairment

# Showing Significant Improvement

% of Impaired Showing Significant Improvement

Trang 11

Trang 12

OSOC Youth Data 2007 – 2011

Referrals to Wraparound from Primary Sources

Ohio Scales Problems – Baseline

Source of Referral (# referrals)

Impaired Borderline Normal

Children and youth referred by

Child Welfare and Juvenile

Justice are more likely to score

in the normal range than those

referred by other sources.

Trang 13

Source of Referral (# referrals)

Impaired Borderline Normal

Referrals to Wraparound from Primary Sources

Ohio Scales Functioning - Baseline

Children and youth referred by

Child Welfare and Juvenile

Justice are more likely to score

not impaired than those

referred by other sources.

Trang 14

Total Enrollments by Impairment at Baseline

Normal, 408, 11%

Borderline,

516, 15%

Impaired Function Only,

-284, 8%

Impaired Problems Only,

-618, 17%

Impaired - Both Function and Problems,

1761, 49%

wraparound youths score in

the normal range on both

Problems and Functioning

scales.

youths are critically impaired

on both scales.

that our referral networks are

functioning well and that

OSOC is providing services to

appropriate families.

Trang 15

6-Month Improvement in Problems / Functioning

by Gender, Caregiver Form

Gender (P = Problems count; F = Functioning count)

Females who are

rated impaired at

baseline show

clinically significant

improvement at a

higher rate at six

months than males

on both scales.

Trang 16

Significant Improvement, Baseline to 6 Months

by Age – Caregiver Form

Trang 17

4 = Most of the Time

5 = All of the Time

Trang 18

Functioning Scale Item

Caregiver Youth Scale values: 0 = Extreme Troubles

1 = Quite a few Troubles

2 = Some Troubles

3 = OK

4 = Doing Very Well

Trang 19

High Externalizing/High Internalizing and Improved

Trang 20

Impaired Youths with More than

6 School Days Missed at Baseline

Age Group (# Youths)

No Clinically Significant Improvement

Baseline to 6 Months

Over 6 School Days Missed Under 7 School Days Missed

Trang 21

OSOC WFI 2008 Pilot Study

Ohio Scales Change by Caregiver WFI Score

Caregiver WFI Avg Problem Change Avg Functioning Change

o In 2008-09, OSOC piloted the WFI 4.0 with 4 sites and 8 wraparound facilitators The results were compared to the improvements in the Ohio Scales.

o Generally, as WFI ratings fell, so too did Ohio Scales improvement.

Trang 22

Future Plans for the Ohio Scales

 Add borderline cases to the outcomes and

appropriateness analyses.

 Explore the use of the Ohio Scales as a pre-referral team

screener, to provide additional information for the team

that staffs each referral.

 In response to research, recommend that the age range

for the Youth Form of the Ohio Scales be extended down

to 9-year-olds.

 Add usability features to the on-line data system to

improve the ability of families and staff to use and

interpret Ohio Scales results.

Trang 23

Resources

 Ash, S., & Weis, R (2009) Recovery Among Youths Referred to Outpatient Psychotherapy: Reliable Change, Clinical

Significance, and Predictors of Outcome Child and Adolescent Social Work Journal 26, 399-413.

 Davis, T., Gavazzi, S., Scheer, S., Kwon, I., Lammers, A., Fristad, M., & Uppal, R (2011): Measuring Family Caregiver

Perceptions of Support in Caring for Children and Youth With Mental Health Concerns, Journal of Social Service Research,

37:5, 500-515.

 Ogles, B M., Carlson, B Hatfield, D R., & Karpenkov, V V (2008) Models of case mix adjustment for Ohio mental health

consumer outcomes among children and adolescents Administration and Policy in Mental Health and Mental Health Services

Research, 35, 295-304.

 Dowell, K A & Ogles, B M (2008) The Ohio Scales Youth Form: Expansion and Validation of a Self-Report Outcome

Measure for Young Children Journal of Child and Family Studies, 17, 291-305.

 Turchik, J., Karpenkov, V.V., & Ogles, B M (2007) Further Evidence of the Utility and Validity of a Measure of Outcome for

Children and Adolescents Journal of Emotional and Behavioral Disorders, 15, 119-128.

 Carlston, D L & Ogles, B M (2006) The impact of items and anchors on parent-child reports of problem behavior The Child

& Adolescent Social Work Journal, 27, 24-37.

 Ogles, B M., Carlston, D L., Hatfield, D.R., Melendez, G., Dowell, K., and Fields, S A (2006) The role of fidelity and

feedback in the wraparound approach Journal of Child and Family Studies, 15, 114-128.

 Ogles, B M., Dowell, K., Hatfield, D, Melendez, G., & Carlston, D (2004) The Ohio Scales In M E Maruish (Ed.), The use

of psychological testing for treatment planning and outcome assessment (3rd ed., Vol 2) (pp 275-304) Hillsdale, New Jersey:

Lawrence Earlbaum

 Ogles, B M., Melendez, G., Davis, D C., & Lunnen, K M (2001) The Ohio Scales: Practical Outcome Assessment Journal

of Child and Family Studies, 10, 199-212.

 Texas Department of Mental Health Mental Retardation (2003) Validation and norms for the Ohio Scales among children

served by the Texas Department of Mental Health and Mental Retardation

http://www.dshs.state.tx.us/mhprograms/RDMCAtrag.shtm

Trang 24

E-TEAM

The Educational Training, Evaluation, Assessment, and Measurement (E-TEAM) Department of the University of Oklahoma is

a full service department with senior researchers, data analysts, technical writers, and data base developers and managers

representing decades of experience in all phases of research data processing E-TEAM also has several years’ experience as

evaluation manager for the State of Oklahoma Systems of Care (OSOC) grant community, implementing its national and local

evaluations as well as special OSOC collaborative studies with child welfare and juvenile justice Belinda Biscoe, Ph.D.,

Assistant Vice President, Public and Community Services, at the University of Oklahoma, College of Continuing Education, is

the Director of E-TEAM E-TEAM has staff with extensive experience in tribal projects in conjunction with the American

Indian Institute (also located at the University of Oklahoma) E-TEAM also has bi-lingual staff to facilitate communication with

Spanish-speaking customers and subjects

The Oklahoma Systems of Care E-TEAM evaluation staff:

 Melinda Anthony, Family Interviewer

 José Diaz, Data Analyst

 Kelly Phillips, Research Assistant

 Sharon Strait, Research Assistant

 Geneva Strech, Senior Research Associate

 Stephen Strech, Youth Interviewer

 John Vetter, Senior Research Associate

 Lisa White, Research Associate

Định dạng
Số trang	24
Dung lượng	2,17 MB