The Construct and Predictive Validity of Instruments Measuring th

Kinesiology Faculty Publications and Presentations Department of Kinesiology4-1-2016 The Construct and Predictive Validity of Instruments Measuring the Psychosocial Correlates of Televis

Kinesiology Faculty Publications and Presentations Department of Kinesiology

4-1-2016

The Construct and Predictive Validity of

Instruments Measuring the Psychosocial Correlates

of Television Viewing

Raheem J Paxton

University of North Texas Health Science Center

Pascal Jean-Pierre

University of Notre Dame

Sae-Hwan Park

University of North Texas Health Science Center

Yong Gao

Boise State University

Stephen D Herrmann

Sanford Research

See next page for additional authors

This document was originally published in Journal of Health Disparities Research & Practice by Digital Scholarship@UNLV Copyright restrictions may

apply.

Journal of Health Disparities Research and Practice Volume 9, Issue 1, Spring 2016, pp 46-59

© 2011 Center for Health Disparities Research School of Community Health Sciences

University of Nevada, Las Vegas

The Construct and Predictive Validity of Instruments Measuring

the Psychosocial Correlates of Television Viewing

Raheem J Paxton, University of North Texas Health Science Center

Pascal Jean-Pierre, University of Notre Dame

Sae-Hwan Park, University of North Texas Health Science Center

Yong Gao, Boise State University

Stephen Herrmann, Sanford Health

Gregory J Norman, University of California – San Diego

ABSTRACT

Background: Many studies have examined the consequences of prolonged television viewing,

but few studies have examined the psychological states that contribute to this behavior In this

study, we evaluated the construct and predictive validity of psychosocial correlates of television

viewing in a population of African American (AA) breast cancer survivors (BCS)

Methods: AA BCS (N = 342, Mean age = 54 years) completed measures of decisional balance,

self-efficacy, family support, and time spent watching television online Exploratory structural

equation modeling (ESEM) was used to examine the construct and predictive validity as well as

the differential item functioning of the instruments among population subgroups

Results: The construct validity of the measures was supported among subgroups The scales were

measuring the construct similarly among the education and body size groups, but not among age

groups Subsequent analysis indicated that pros (β = -0.19, P < 0.05), cons (β = 0.18, P < 0.05),

and self-efficacy (β = -0.16, P < 0.05) were significantly associated with time spent watching

television

Conclusions: Minor modifications may be needed to support the validity and reliability of the

decisional balance and self-efficacy subscales among older survivors More studies are needed to

modify these measures to establish sufficient levels of construct and predictive validity in this

population

Keywords: African American, breast cancer, cancer survivorship, reliability, sedentary

behavior, television viewing, validity

INTRODUCTION

Sedentary behavior (i.e., watching TV, sitting, reclining, or lying down) has emerged as a

major risk factor for chronic disease.1 In particular, prolonged periods of sedentary behavior

have been associated with an increased risk for developing colorectal, endometrial, and ovarian

cancer.1 Out of all forms of sedentary behavior, television viewing has been associated with the

worst outcomes because it is often linked with increased caloric intake of calorie dense foods2

and is associated with a lower metabolic rate than other forms of sedentary behavior (e.g., riding

or driving in an automobile).3, 4 Aside from sleeping, television viewing occupies the most time

in domestic settings.5 Sedentary behaviors (e.g., television viewing) may have an even greater or

compounding impact in people who are struggling with incapacitating diseases and/or treatment

outcomes such as cancer

Excessive television is a maladaptive lifestyle behavior Among cancer survivors,

prolonged sitting was shown to be associated with diminished quality of life,6 weight gain,7

larger waist circumference,8 ischemic heart disease,9 and premature mortality.10 The negative

health impact of prolonged sitting along with excessive TV viewing time in survivors

underscores the urgent need for the development and testing of effective interventions to mitigate

this problem This may be true especially for African American breast cancer survivors who

report excessive sedentary behavior and multiple comorbid conditions According to a recent

study of African American (AA) breast cancer survivors, 43% reported excessive television

viewing (i.e., watched television for ≥ 2 hours/day) and approximately 70% reported at least one

comorbid condition in addition to cancer.11 Thus, there is a need for studies to examine the

factors that predispose AA BCS to prolonged periods of sedentary behavior overall, but more

specifically television viewing given its adverse consequences

There is a need for systematic studies that assess the underlying psychological and

situational reasons why people engage in excessive television viewing However, limited data

exist on the psychosocial correlates of television viewing Previous studies assessing these

correlates have focused almost elusively on adolescents, with one study published in a healthy

adult population.12 Norman et al.13-15 examined the psychometric properties of several

psychosocial correlates (i.e., decisional balance, self-efficacy, social support, and behavior

change strategies) for sedentary behavior and found that these items were significantly associated

with time spent sitting Van Dyck et al.12 applied these instruments to an adult population and

observed that similar results An important caveat that has been overlooked in assessing the

predictive validity of instruments previously designed for another population is the assessment of

the psychometric properties Establishing the construct and predictive validity of these

instruments is a necessity, especially in vulnerable populations with high rates of sedentary

behavior and television viewing Assessing these measures in a population of African American

breast cancer survivors will not only address current gaps in the literature, but also provides new

data on a high risk, underrepresented, and vulnerable population

The aims of the current paper was to assess the construct and predictive validity of

instruments that measure the constructs of decisional balance, self-efficacy, and social support

using a robust psychometric procedure called Exploratory Structural Equation Modeling.16-18

Specifically, we will:

a) Determine the constructs validity of measures of decisional balance (i.e., pros & cons)

and self-efficacy for reducing television viewing and social support for sedentary

behavior reduction;

b) Determine the measurement equivalence/invariance (or differential item functioning) of

the instruments among age groups, body size groups, and educational levels to ensure

that the items are being measured similarly among subgroups; and

c) Determine whether the instruments are associated with time spent watching television

METHODS

A total of 342 AA BCS from the Sisters Network, Inc were surveyed to assess

psychosocial correlates of television viewing The Sisters Network is the largest AA breast

cancer survivorship organization in the United States The Sisters Network is a national

organization that contains 40 affiliate chapters in 19 geographically distinct states BCS were

recruited for the present study between May of 2012 and July of 2012 via multiple email blasts

and posting of anonymous survey links on social media blog sites affiliated with the Sisters

Network Detailed information related to our recruitment methods and response rates were

described elsewhere.19 Eligibility criteria included (a) being 18-80 years old at diagnosis, (b)

diagnosed with operable invasive breast cancer, (c) not currently undergoing treatment (with the

exception of hormone therapy), and (d) have no evidence of recurrent disease Institutional

Review Board approval was obtained at the University of Texas MD Anderson Cancer Center

prior to data collection and it was assumed that by reading the consent form on the initial survey

web page and answering survey questions, women gave their consent to participate in the current

study The protocol was later approved by the Institutional Review Board at the University of

North Texas Health Science Center following the transfer of the primary author The consenting

procedure was approved by the Institutional Review Board at each institution

Measures

Television-viewing Time Time spent watching television or videos/movies were reported

by participants separately for weekdays and weekend days during the previous week Total

television time was calculated as the sum of the time participants watched television on

weekdays and weekend days This measure has been shown to have reasonable reliability and

validity for estimating television-viewing time in adults.20

Psychosocial variables The items used in the current study were adapted from validated

questionnaires previous developed for adolescents.12, 14 The original items were adapted and

applied to a population of adults in a previous study.12 Van Dyck et al.12 adapted 4 items each

that represented pros (e.g., I think watching TV is boring), cons (e.g., I enjoy watching TV for

many hours at a time), and self-efficacy (e.g., confidence to turn off the TV even when there is a

program on that you enjoy) for reducing television time Three family support items were

adapted from similar items that were initially developed for physical activity.15 Example of the

“my family encourages…,” my family discussed…,” my family helped me to think of ways…”

All items were rated on a 5-point Likert scale Pro and Con items were rated from strongly

disagree (1) to strongly agree (5), self-efficacy items were rated from I’m sure I can’t (1) to I’m

sure I can (5), and family support items were rates from never (1) to very often (5)

Body size groups The study participants’ self-reported height and weight were used to

compute their BMI (weight in kilograms divided by height in meters squared: kg/m2) Study

participants were categorized as obese if their BMI was ≥30 kg/m2 and non-obese ≤30 kg/m2

This cut-off was chosen because ~80% of the population self-reported a BMI > 25 kg/m2

Socio-demographic and Medical Data All socio-demographic and medical data were

self-reported by participants We collected data on the following variables: current age,

education, time since diagnosis, disease stage at diagnosis, and comorbid conditions Comorbid

conditions (e.g., cardiovascular disease, blood sugar/diabetes, digestive disorders, arthritis, and

osteoporosis) were summed to represent an ordinal number

Statistical Analysis

Descriptive statistics were computed for the sociodemographic and medical

characteristics Construct validity of the relevant instruments were examined using Exploratory

Structural Equation Modeling (ESEM) The measurement equivalence/invariance (ME/I) of

these instruments were evaluated among age groups (i.e., 18-49, 50-59, 60+), weight status

groups (i.e., non-obese and obese), and educational levels (i.e., < college graduate and college

graduate) These specific sub populations were explored given the sample size for each group

and the large percentage of women with college educations and self-reporting a BMI ≥ 30 kg/m2

Exploratory Structural Equation Modeling

Exploratory Structural Equation Modeling (ESEM) is the integration of exploratory

factor analysis (EFA) and structural equation modeling (SEM) in an effort to provide a flexible

measurement structure for item indicators.19 The ESEM has all of the benefits of traditional EFA

such as factor rotations, while enabling the inclusion of path coefficients (among covariates and

other factors), multi-group analysis, and test measurement equivalence/invariance (ME/I).19

ESEM also provides fit statistics and modification indices similar to those generated in

traditional SEM We chose ESEM in lieu of traditional Confirmatory Factor Analysis (CFA) to

facilitate exploration of the true structure validity of these instruments CFA prevents cross

loading of items, leading to over-estimated factor correlations and distorted relationships.19 In

contrast, ESEM provides flexibility when knowledge of the measurement structure is limited

All models were examined with the Maximum Likelihood estimator that is robust to

non-normal distributions (i.e., ESTIMATOR = MLR) and a Geomin rotation algorithm ESEM

models were calculated with full-information maximum (FIML) estimation in MPlus version 6.0

(Muthen & Muthen, 1998-2008) FIML uses an iterative process and simultaneous estimating

equations to account for the presence of missing data.21 FIML yields accurate fit indices and

parameter estimates with up to 25% simulated missing data.21 The extent of missing data in this

study ranged from 0% for sociodemographic characteristics to 24.6% for social support items,

which is under the recommended threshold

Model Fit

Criteria for establishing fit of ESEM models are similar to that of traditional CFA and

SEM All models are evaluated based on how well structural model resembles close, exact, and

absolute fit to the data According to Hu and Bentler,22 the Comparative Fit Index (CFI) and the

Standardized Root Mean Square Residual (SRMR) are optimal for examining structural models

with smaller sample sizes The CFI and SRMR reveal the models closely fitted the data when

values are ≥0.95 and ≤0.08, respectively Hu and Bentler22 proposed that using cut off values ≥

0.96 for the CFI in combination with values of ≤ 0.10 for the SRMR resulted in lower type I and

II error rates These fit statistics were chose over other criteria (i.e., χ2 and the Root Mean Square

Error Approximation) which are sensitive to sample time and inflate error rates.22

Multi-group Factorial Invariance

Assessment of measurement equivalence/invariance is a multistage approach.23 In the

first series of ESEMs we examined the fit of the measurement model for the overall population

and individually for each sub-group We then tested models that sequentially imposed constraints

to model parameters to insure equality of the overall measurement structure, factor loadings, and

item intercepts among subgroups Three sequential levels of invariance tests were assessed here

In the first model, we tested the extent to which the same pattern of fixed and free model

parameters was equivalent among groups (i.e., configural invariance).23 In the second model, we

tested the extent to which the factor loadings for the items were measured equivalently among

groups (i.e., metric invariance).23 Finally, in the last model, we tested the extent to which the

item intercepts were measured equivalently among groups (i.e., scalar invariance).23 Once the

models were computed, we determined ME/I by evaluating the difference in Chi-square in

relation to change (Δ) in degrees of freedom of the model with fewer constraints Change in CFI

of less than or equal to 0.01 suggests that the invariance of an instrument should not be

rejected.24 Therefore, if the Chi-Square difference test is significant, but the CFI change is less

than 0.01, there is some evidence for the equivalence/invariance of the model structure or

parameters among groups.24

Differential Item Functioning (DIF) also known as measurement bias was also examined

for factors that failed to pass test for ME/I.25 To assess DIF in this study, we used a

multiple-indicator multiple cause (MIMIC) model.26 MIMIC models can be used to identify subgroup

differences in a latent construct.26 These models are extensions of item-response theory modes

but can include simultaneous test of several characteristics

Lastly, structural models were constructed to assess the relationship between

psychosocial constructs and time spent sitting and watching television per day Structural models

were adjusted for the following covariates: body mass index, age, years out from diagnosis, and

disease stage of diagnosis All statistical tests were two-sided and significance was determined at

p < 0.05

RESULTS

Sample Characteristics

The study population of 342 surveyed AA BCS has the mean age of 53.5 years Most

(45%) of the participants presented with stage II disease and were on average 7-years post

diagnosis Approximately half (52%) of participants were college graduates, 48% reported a

BMI in the obese category, and 43% reported watching television equal to or greater than 2

hours per day Sample characteristics are reported in Table 1

Structural validity and reliability

The measurement model for pros, cons, and self-efficacy for reducing time spent

watching television and family support for sedentary behavior reduction was a close fit to the

data (CFI ≥ 0.95, RMSEA ≤ 0.08, SRMR ≤ 0.08) Statistically significant correlations were

observed between pros and cons (r = -0.34, P < 0.01), self-efficacy and cons (r = -0.33, P <

0.01), and family support and cons (r = 0.13, P = 0.05) All factor loading, intercepts, and factor

variances were appropriate sign and magnitude Several items (i.e., TV is boring, enjoy watching

TV, watching TV is relaxing, and confidence to limit TV during) cross-loaded on other factors

(See Table 2) The overall fit of the measurement model revealed a close fit to the data for each

population sub groups (See Table 3) Internal consistency reliability for pros, cons, self-efficacy,

and family support were 0.54, 0.80, 0.80, and 0.87, respectively (data not tabled)

Table 2 Factor structure of psychosocial constructs

Item Factor 1 Factor 2 Factor 3 Factor 4

Watching TV takes time away from doing more

I would Feel lazy and sluggish if I watched TV for

Watching TV sometimes hurts my eyes and gives me

Watching TV is one of my favorite forms of

Watching TV is my way to escape from the world -0.03 0.87 0.18 -0.06

Turn off the TV even when there is a program on I

Leave the room where the TV is on even if others are

Plan ahead of time what TV shows I will watch

My family encouraged me to spend less time being

My family discussed how sedentary habits can be

My family helped me to think of ways to reduce the

Factor 1 = Pros, Factor 2 = Cons, Factor 3= Self-efficacy; Factor 4 = Family Support; Items representing

a particular subscale were reported in bold font

Test for ME/I

Age groups: The measurement model constraining the factor structure revealed a close fit

to the data (χ 2 = 186.9, df = 153, p-value = 0.03, CFI = 0.98, SRMR = 0.04) among different age

groups Subsequent nested models of the factor loading and factor means and intercepts yielded a

close fit to the data However, the change (Δ) in CFI was ≥0.01 when constraints were imposed

on the factor loadings (See Table 4) No further tests for invariance were performed

Obesity status: The measurement model constraining the factor structure revealed a close

fit to the data (χ 2 = 136.8, df = 102, p-value < 0.01, CFI = 0.98, SRMR = 0.03) among body size

groups Subsequent nested models of the factor loading and factor means and intercepts yielded a

close fit to the data and values estimating the Δ in CFI support evidence of ME/I for the

measurement model among body size groups (See Table 4)

Education: The measurement model constraining the factor structure revealed a close fit

to the data (χ 2 = 131.4, df = 102, p-value = 0.03, CFI = 0.98, SRMR = 0.03) among education

levels Subsequent nested models of the factor loading and factor means and intercepts yielded a

close fit to the data and estimates of the Δ in CFI were appropriate in magnitude suggest that the

measurement model is ME/I among educational levels (See Table 4)

Post hoc tests for differential item functioning

The MIMIC model examining the relationship between age group and the measurement

model revealed a close fit to the data (χ2 = 72.9, df = 62, p-value = 0.16, CFI = 0.99, SRMR =

0.02) Statistically significant path coefficients were observed between age group and

self-efficacy (β = -0.17, P < 0.05) and age group and pros (β = -0.27, P < 0.01), suggesting age group

differences in the measurement of these constructs (See Figure 1)