Modeling intraindividual variability with repeated measures data

T h e partner gender by subject interaction X by S / Z is the variation in the effect of gender of partner for each subject i.e., to what degree does the mean of female partners minus

Modeling Intraindividual Variability with Repeated Measures Data:

MULTIVARIATE APPLICATIONS BOOK SERIES

The multivariate Applications book series was developed to encourage the use of rigorous methodology in the study of meaningful scientific issues, and to describe the applications in easy to understand language The series is sponsored by the Society

of Multivariate Experimental Psychology and welcomes methodological applications from a variety of disciplines, such as psychology, public health, sociology, education, and business The main goal is to provide descriptions of applications of complex statistical methods to the understanding of significant social or behavior issues The descriptions are to be accessible to an intelligent, non-technical oriented readership (e.g., non-methodological researchers, teachers, students, government personnel, practitioners, and other professionals)

Books can be single authored, multiple authored, or edited volumes The ideal book for this series would take on one of several approaches: (1) demonstrate the application of several multivariate methods to a single, major area of research; (2) describe a multivariate procedure or framework that could be applied to a number of research areas; or (3) present a variety of perspectives on a controversial topic of interest to applied multivariate researchers There are currently 7 books in the series:

What $There Were No Significant Tests?, co-edited by L Harlow, S Mulaik, and

Structural Equation Modeling with AMOS, by B Byrne (2001)

Conducting Meta-Analysis Using SAS, co-authored by W Arthur, Jr., W Bennett, Jr., and A I Juffcutt (2001)

Modeling Intraindividual Variability with Repeated Measures Data: Methods and Applications, co-edited by D S Moskowitz and S L Hershberger (2002) Anyone wishing to propose a book should address the following: (1) title; (2) author(s); (3) timeline including planned completion date; (4) Brief overview of focus for the book including a table of contents and a sample chapter (or more); ( 5 ) mention

any competing publications in this area; (6) mention possible audiences for the proposed book More information can be obtained from the editor, Lisa Harlow, at: Department of Psychology, University of Rhode Island, 10 Chafee Rod., Suite 8 , Kingston, RI 02881-0808; Phone: 401-874-4242; Fax: 401-874-5562; or e-mail: Lharlow@uri.edu Information can also be obtained from one of the advisory board members: Leona Aiken (Arizona State University), Gwyneth Boodoo (Educational Testing Service), Susan Embretson (University of Kansas), Michael Neale (Virginia Commonwealth University), Bill Revelle (Northwestern University), and Steve West (Arizona State University)

Modeling Intraindividual Variability with Repeated Measures Data:

Edited by

California State University, Long Beach

LAWRENCE ERLBAUM ASSOCIATES, PUBLISHERS

The final camera copy for this work was prepared by the editors

and therefore the publisher takes no responsibility for consistency

or correctness of typographical style However, this arrangement

helps to make publication of this kind of scholarship possible

Copyright @ 2002 by Lawrence Erlbaum Associates, Inc

All rights reserved No part of the book may be reproduced in any form, by photostat, microform, retrieval system, or any other means, without the prior written permission of the publisher

Lawrence Erlbaum Associates, Inc., Publishers

Books published by Lawrence Erlbaum Associates are printed on acid-free paper,

and their bindings are chosen for strength and durability

Printed in the United States of America

10 9 8 7 6 5 4 3 2 1

Carolina, Chapel Hill, NC 27599-3270

J.O Ramsay, Department of Psychology, McGill University, 1205 Dr Penfield Avenue, Montreal, Quebec, Canada, H3A 1B1

Dennis Wallace, Department of Preventive Medicine, University of Kansas Medical Centre, 4004 Robinson Hall, 3901 Rainbow Blvd., Kansas City, KS

Fuzhong Li, Oregon Research Institute, 1715 Franklin, Blvd., Eugene, OR

Lisa A Strycker, Oregon Research Institute, 1715 Franklin, Blvd., Eu- gene, OR 97403-1983

Steven Hillmer, School of Business, University of Kansas, 203 Summer- field Hall, Lawrence, KS 66044-2003

John R Nesselroade, Department of Psychology, The University of Vir- ginia, 102 Gilmer Hall, P.O Box 400400, Charlottesville, VA 22904-4400 John J McArdle, Department of Psychology, The University of Virignia,

102 Gilmer Hall, P.O Box 400400, Charlottesville, VA 22904-4400

University, P.O Box 980710, Richmond, VA 23286-0440

Jonathan M Meyers, Department of Psychology, The University of Vir- ginia, 102 Gilmer Hall, P.O Box 400400, Charlottesville, VA 22904-4400 97403-1983

Contents

Preface

els

David A Kenny, Nial Bolger, and Deborah A Kashy

Models of Individual Growth and Change

Stephen W Raudenbush

Structural Equation Modeling of Repeated Mea-

sures Data: Latent Curve Analysis

Patrick J Curran and Andrea M Hussong

Multilevel Modeling of Longitudinal and Functional

Data

J 0 Ramsay

Analysis of Repeated Measures Designs with Linear

Mixed Models

Dennis Wallace and Samuel B Green

Fitting Individual Growth Models Using SAS PROC

viii

John R Nesselroade, John J McArdle, Steven H Aggen, and

Jonathan M Meyers

Preface

This volume began as a nightmare

Once upon a time, life for social and behavioral scientists was (relatively) simple When a research design called for repeated measures data, the data were analyzed with repeated measures analysis of variance The BMDP 2V module was frequently the package of choice for the calculations

Life today is more complicated There are many more choices Does the researcher need t o model behavior at the level of the individual as well

as at the level of the group? Should the researcher use the familiar and well-understood least-squares criterion? Should the researcher turn t o the maximum likelihood criterion for assessing the overall fit of a model? Is it possible and is it desirable t o represent the repeated measures data within structural equation modeling?

So the nightmare began as (shall we be dishonest and say) one night of deliberations among these choices The thought then arose that it would

be useful t o have the statistical experts writing in the same volume about the possibilities and some of the dimensions that are pertinent t o making these choices Hence the origin of the present volume

The issue of the analysis of repeated measures data has commonly been examined within the context of the study of change, particularly with re- spect t o longitudinal data (cf., Collins & Horn, 1991; Gottman, 1995) This volume contains three chapters whose primary focus is on the study of growth over several years time (Raudenbush, chapter 2; Curran & Hussong,

chapter 3; Duncan, Duncan, Li, & Strycker, chapter 7) Studies of change

typically imply the expectation that variation, movement in scores, is gen- erally unidirectional-generally up or generally down Not all repeated measures data are concerned with change, and change is only one aspect

of the variability that occurs within individuals To illustrate, consider an example from the study of social behavior

Personality, social, and organizational psychologists are often interested

in the effects of situations on behavior: t o what extent are individuals’ behaviors consistent across sets of situations and t o what extent does the behavior of individuals change as a function of the situation For example, the focus might be on how people’s dominant and submissive behaviors change as a function of being in a subordinate, co-equal, or supervisory work role There might also be interest in whether people’s responses t o these

ix

X p r e f a c e

situations vary as a function of their level on personality characteristics Some people, let’s say extraverts, may change more in their behavior than other individuals in responding t o these different situations

This could be studied in the laboratory in which individuals participate

in situations in which they are placed in a subordinate role, a co-equal role, and a supervisory role and their responses are recorded This would be an example of a balanced design All participants would participate in three situations These data can be analyzed in the familiar technique of repeated measures analysis of variance We might introduce the personality variable

of extraversion t o examine the interaction between individual differences and situation

However, there is considerable error variance in a measure based on

a one-occasion assessment (Epstein, 1979; Moskowitz & Schwarz, 1982) Measurements of the individual in each situation on several occasions would improve the quality of measurement This is possible but difficult in the laboratory, so sometimes researchers make use of naturalistic techniques for collecting this kind of data (see Kenny, Bolger, & Kashy, chapter 1; also see Moskowitz, Suh, & Desaulniers, 1994)

Despite whether the researcher remains in the laboratory or whether the researcher uses a naturalistic methodology, the researcher is confronted with decisions about how t o handle the data The multiple measures for each situation could be aggregated (averaged) t o provide a single measurement

in each situation for each individual If this is done within the context of the laboratory, this provides a balanced design with better measures Unfor- tunately, this strategy throws away information Some people would have less variability in their measures than other people It may be of interest

t o know who has more variability in their responses t o such situations as being the boss or being the supervisee

As an alternative to the laboratory context, the researcher might use

a naturalistic d a t a collection method such as event contingent recording (Moskowitz, 1986; Wheeler & Reis, 1991) In event contingent recording, participants are given standardized forms and asked t o record their behavior after being in certain kinds of events, such as all interpersonal events at work The form could request information about characteristics of the situation as well as the person’s behavior so interpersonal events can be categorized into situations with a boss, situations with a co-equal, and situations with a supervisee This method has appeal because it provides records of behavior in real life rather than responses t o possibly artificial situations in the laboratory

However, the structure of such data presents d a t a analytic decisions Individuals will report differing numbers of events Individuals will report differing numbers of events in different kinds of situations Individuals may report events corresponding to some of the targeted situations (e.g., in the subordinate and co-equal situations) but not t o other targeted situations such as having the supervisory responsibilities of the boss situation The d a t a structure could be simplified by aggregating across events

p r e f a c e xi

referring t o the same kind of situation to obtain one measure per situation and only including people in the sample who reported events in all three kinds of situations The simplification of the data structure would provide

a balanced design, and consequently the familiar data analytic techniques

of repeated measures analysis of variance and repeated measures analysis

of covariance could be used

However, such simplification would also eliminate information The sim- plification would (1) not take into account variability in people’s responses across events of the same type of situation; (2) throw away that portion of

a sample that has “missing data”; that is, individuals whose data do not

include the representation of all kinds of specified events, and (3) disregard

the time ordering of events

Once one becomes involved with recording multiple assessments of indi- viduals behavior and affect responses, the variability of people’s responses across events becomes salient and compels modeling For example, diur- nal and weekly rhythms have been demonstrated for affect and behavior (Brown & Moskowitz, 1998; Larsen, 1987; Watson, Wiese, Vaidya, & Tel- legen, 1999) Behavior and affect co-occur over time in ways that cannot

be identified from static assessments of these variables (Moskowitz & Cote, 1995) Similarity and dissimilarity among measures or items from occa- sion t o occasion may be of interest (see Nesselroade, McArdle, Aggen, & Meyers, chapter 9) The shape of variation can be of considerable impor- tance, such as the shape of change in response t o stress or psychotherapy

or recovery from illness (e.g., Bolger & Zuckerman, 1995; also see Ramsay, chapter 4) The time ordering of events can be used t o make inferences about antecedent-consequent relations (see Hillmer, chapter 8)

So the focus of this volume is the examination of how individuals be- have across time and t o what degree this behavior changes, fluctuates, is stable, or is not stable We call this change in individual behavior “intrain- dividual variability.” Intraindividual variability can be contrasted with “in- terindividual variability.” The latter describes individual differences among different people; the former describes differences within a single person Although most behavioral and social scientists believe that behavior does differ from one occasion to the next, sophisticated techniques for exploring intraindividual variability have been underutilized Several factors have contributed t o the reluctance of analysts t o utilize these techniques One factor is their newness, many of them having only been developed within the last few years A second factor is the perceived difficulty of implementing these techniques; descriptions tend t o be highly technical and inaccessible

t o nonmathematically trained researchers A third factor is the unavailabil- ity of computer programs to do the analyses, a situation that has recently been much improved with the release of new computer programs

The primary goal of this volume is to make accessible t o a wide audience

of researchers and scholars the latest techniques used t o assess intraindi- vidual variability The chapters of this volume represent a group of distin- guished experts who have written on a range of available techniques The

xii p r e f a c e

emphasis is generally at an introductory level; the experts have minimized mathematical detail and provided concrete empirical examples applying the techniques

The volume opens with a chapter by David Kenny, Niall Bolger, and Deborah Kashy, who contrast several procedures for the analysis of repeated measures data They note two problems with using traditional analysis of variance (ANOVA) procedures for analyzing many contemporary designs using repeated measures data The first is that research participants of- ten will not have the same number of data points The second is that the predictor variable generally does not have the same distribution across measurement points for all research participants They approach the anal- ysis of intraindividual variability within the context of multilevel analyses

in which research participant are independent units and the repeated ob- servations for each individual are not assumed to be independent They illustrate that a strength of alternative procedures to ANOVA is that they more readily permit the evaluation of random effects that reflect the extent

of variability among individuals to fixed effects They compare features of three alternative procedures for modeling the group of research participants and the variability within the group of research participants: a two-step ordinary least-squares regression procedure, a weighted least-squares varia- tion of multiple regression, and a procedure based on a maximum likelihood criterion

Stephen Raudenbush compares advantages of the hierarchical linear model (a multilevel model), structural equation modeling, and the gener- alized multivariate linear model in the analysis of repeated measures data

He argues for the flexibility of the hierarchical linear model (HLM) HLM permits the inclusion of all available data, allows unequal spacing of time points across participants, can incorporate a variety of ways of character- izing change in the data such as rate of change and rate of acceleration, and can provide for the clustering of individuals within groups such as schools or organizations He then combines ideas from the standard hier- archical linear model and the multivariate model to produce a hierarchical multivariate model that allows for different distributions within persons of randomly missing data and time-varying covariates, permits the testing of

a variety of covariance structures, and examines the sensitivity of inferences about change to alternative specification of the covariance structure The procedure discussed permits the examination of whether alternative models are consistent with key inferences about the shape of change

Patrick Curran and Andrea Hussong describe how repeated measures data can be represented in structural equation models They discuss the advantages and disadvantages of two kinds of structural equation models for representing longitudinal change: the autoregressive crosslagged panel model and the latent curve analysis model They emphasize the latent curve approach, an approach that first estimates growth factors underlying observed measures and then uses the growth factors in subsequent analy- ses Latent curve analysis provides two key advances over autoregressive

crosslagged panel models The first is the capability to model data sets with more than two time points The second is the capability to provide estimates of the extent of variability among individuals, both the extent of variability in starting points and in rates of change An applied example concerning the development of antisocial behavior and reading proficiency

is used to illustrate the latent curve analysis model The example illustrates that predictors of behavior at single time points (e.g., initial status) are dif- ferent from the predictors of the shape of change over time They also use the example to illustrate several options for incorporating nonlinear forms

of growth in structural equation models

James Ramsay provides a commentary on issues connecting the chap- ters by Raudenbush, Curran and Hussong, and Kenny, Bolger, and Kashy

He makes several points relative t o the study of longitudinal data, consid- ering the implications of missing data, the number of points necessary to define characteristics of growth curves such as level, slopes, and bumps, and the possibility that the curves for individuals are not registered such that the curves for individuals may show a similar shape but reflect different timings of events His chapter further extends the discussion of repeated measurements to the case where there are many measurements and makes the point that such data can be represented by a sample of curves using

a set of techniques referred to as functional data analysis His chapter ends on a note of caution, reminding the reader that moving to the more complex models that are sometimes presented in this book has costs that need to be considered For example, the maximum-likelihood procedures are sensitive to the mis-specification of the variance-covariance structure Moreover, adding random coefficient parameters uses up degrees of freedom leading t o a loss of power and potentially unstable estimates of fixed effects Thus, the cautious researcher who has a moderate sample size may prefer

to keep the model simple such as by remaining with a least-squares-based regression procedure (cf Kenny, Bolger, & Kashy, chapter 1)

There is considerable complexity in the analysis of the models that make use of random as well as fixed effects (see chapters 1 and 2) The chapters

by Judith Singer and by Dennis Wallace and Samuel Green present detailed description of how to analyze and interpret such models using a commonly available package, the PROC MIXED procedure from SAS

Dennis Wallace and Samuel Green’s chapter provides extensive infor- mation about how to estimate fixed and random effects They provide detailed explanations of the meaning of the underlying statistics, such as maximum-likelihood and restricted maximum-likelihood methods, and an introduction to some of the structures that may be found in the variance- covariance matrices They provide an outline of recommended steps for estimating models incorporating fixed and random effects These steps are illustrated using an example from a longitudinal study of the effect of two treatment interventions for reducing substance abuse among homeless individuals; the illustration includes an examination of whether the effec- tiveness of the treatment programs vary as a function of changing levels of

xiv p r e f a c e

depression

Judith Singer’s discussion provides practical advice for all stages of the analysis including data preparation and writing computer code She illus- trates a process that is sometimes mysterious for the novice researcher in this area Models for the representation of individuals’ variability across time are sometimes presented as single equations a t multiple levels (Bryk & Raudenbush, 1992) and sometimes by single equations that specify multiple sources of variation (cf Goldstein, 1995) She demonstrates how separate equations can be written a t multiple levels and then elements can be substi- tuted in t o arrive a t a representation in a single equation The presentation

is situated in the context of individual growth models; the presentation can also be extended and applied t o cases with repeated measures data that are unidirectional as described in the extended example presented earlier and

in the chapter by Kenny, Kashy, and Bolger

Stephen Raudenbush comments that the use of structural equation mod- eling has not typically been extended t o the case where individuals are clustered Terry Duncan, Susan Duncan, Fuzhong Li, and Lisa Strycker take the step of providing such an extension They provide an introduction

t o representing multilevel models in structural equation models using an example from an analysis of change in adolescents’ use of alcohol They compare the strengths and weaknesses of three approaches for modeling longitudinal data that are clustered and unbalanced One method, a full information hierarchical linear model (HLM), is familiar from the chapter

by Raudenbush A second method, a limited information multilevel latent

growth model (MLGM), is an extension of latent growth modeling that was presented in the chapter by Curran and Hussong The third approach

is based on a full information maximum likelihood (FIML) latent growth modeling using an extension of a factor of curves model which has not pre- viously been discussed in the book They provide examples of programming

in both HLM (Bryk, Raudenbush, & Congdon, 1996) and Mplus (Muthbn

& M u t h h , 1998)

Steven Hillmer provides a basic introduction t o using time series models

t o predict intraindividual change In a time series model, data points for the same variable are arranged sequentially in time, and a basic goal is t o identify a model that best represents the sequencing of these data Hillmer reviews the differences between the main kinds of models that might be used He contrasts two classes of models: stationary models in which the joint probability of any set of observations is unaffected by a shift backward

or forward in the time series and nonstationary models in which parts of the series behave similarly although not identically t o other parts of the series He reviews the steps of building a time series model, providing extensive graphical material for understanding the issues that might arise The chapter includes an example of an interrupted time series data in which

an intervention occurs during the course of a time series and the effect of the intervention is estimated The extended example provided is drawn from the business literature on sales Time series analyses can also be applied

technique factor analysis, which uses the common factor model t o model the covariation of multiple variables measured across time for a single in- dividual They note problems with this model in the representation of process changes over time, such as the representation of effects tha t dis- sipate or strengthen over time They present two models that allow for time-related dependencies and illustrate the application of these two dy- namic factor analysis methods using reports of daily moods T h e necessary LISREL code for conducting these analyses is included

The initial organization for this volume was done within the context of two symposia presented at the 1997 meeting of the American Psychological Association We thank Lisa Harlow, the editor of the Erlbaum Multivariate Applications Series for suggesting that we prepare a volume based on these symposia We also thank James Ramsay, Yoshio Takane, and David Zuroff for comments on drafts of these chapters We are also grateful t o Chantale Bousquet and Serge Arsenauit for their preparation of the text in I4'QjX Preparation of this volume was partially supported by funds from the Social Sciences and Humanities Research Council of Canada

We hope that the volume provides readers with a sense of the range

of reasonable options for analyzing repeated measures data and stimulates new questions and more interest in repeated measures designs that extend beyond the context of longitudinal data

Pleasant dreams

Debbie S Moskowitz Scott L Hershberger

REFERENCES

Bolger, N., & Zuckerman, A (1995) A framework for studying personality

in the stress process Journal of Personality a n d Social Psychology,

69, 890-902

Brown, K W., & Moskowitz, D S (1998) Dynamic stability of behavior: The rhythms of our interpersonal lives Journal of Personality, 66,

105-134

Bryk, A S., & Raudenbush, S W (1992) Hierarchical linear models:

Applications and data analysis methods Newbury Park, CA: Sage

xvi p r e f a c e

Bryk, A S., Raudenbush, S W., & Congdon, R T (1996) H L M : Hierar- chical linear and nonlinear modeling with the H L M / 2 L and H L M / 3 L programs Chicago, IL: Scientific Software International, Inc

Collins, L., & Horn, J (Eds.) (1991) B e s t methods f o r analysis of change:

Recent advances, unanswered questions, future directions Washing-

ton, DC: American Psychological Association

Epstein, S (1979) The stability of behavior: I on predicting most of the people much of the time Journal of Personality and Social Psychol- ogy, 37, 1097-1126

Goldstein, H (1995) Multilevel statistical models (2nd ed.) New York:

Halstead Press

Gottman, J M (1995) T h e analysis of change Hillsdale, NJ: Erlbaum Larsen, R J (1987) The stability of mood variability: A spectral analy- sis approach t o daily mood assessments Journal of Personality and

Social Psychology, 52, 1195-1204

Moskowitz, D S (1986) Comparison of self-reports, reports by knowledge- able informants and behavioral observation data Journal of Person- ality, 54, 101-124

Moskowitz, D S., & Cote, S (1995) Do interpersonal traits predict affect:

Journal of Personality and Social

A comparison of three models

Psychology, 69, 915-924

Moskowitz, D S., & Schwarz, J C (1982) The comparative validity of behavioral count scores and knowledgeable informants’ rating scores

Journal of Personality and Social Psychology, 42, 518-528

Moskowitz, D S., Suh, E J., & Desaulniers, J (1994) Situational in- fluences on gender differences in agency and communion Journal of Personality and Social Psychology, 66, 753-761

Muthdn, L K., & Muthdn, B 0 (1998) Mplus user’s guide Los Angeles:

Muthden and Muthen

Watson, D., Wiese, D., Vaidya, J., & Tellegen, A (1999) The two general activation systems of affect: Structural findings, evolutionary consid- erations, and psychobiological evidence Journal of Personality and Social Psychology, 76, 820-838

Wheeler, L., & Reis, H T (1991) Self-recording of everyday life events: Origins, types, and uses Journal of Personality, 59, 339-354

Chapter 1

New Yorlc University

Texas A &M University

Researchers often collect multiple observations from many individuals For example, in research examining the relationship between stress and mood,

a research participant may complete measures of both these variables every day for several weeks, and so daily measures are grouped within partici- pants In relationship research, a respondent may report on characteristics

of his or her interactions with a number of different friends In developmen- tal research, individuals may be measured a t many different times as they develop In cognition research, reaction times may be observed for multiple stimuli

These types of data structures have been analyzed using standard (ANOVA) methods for repeated measures designs The most important limitation of the analysis of variance (ANOVA) approach is that it requires balanced data So, in the previous examples, each person would be re- quired t o have the same number of repeated observations For example,

in the stress and mood study, everyone might have t o participate for ex- actly 14 days, and in the relationships study each respondent might report

I

2 Kenny, Bolger, and Kashy

on interactions with exactly four friends It is often the case, however,

th at da t a structures generated by repeated observations are not balanced, either because of missing observations from some participants or, more fun- damentally, because of the nature of the research design If, for instance, researchers were interested in learning about naturally occurring interac- tions with friends, they might have individuals describe their interactions with each person whom they consider to be a friend For individuals who have few friends, there would be very few observations, whereas for other individuals there would be many

An additional factor can make the design unbalanced even if the number

of observations per person is equal For the design t o be balanced, the distribution of each predictor variable must be the same for each person

So, if the predictor variable were categorical, there would need t o be the same number of observations within each category for each person If the predictor variable were continuous, then its distribution must be exactly the same for each person The likelihood of the distribution being the same for each person is possible, but improbable For example, in a study of stress and mood, it is unlikely th at the distribution of perceived stress over the 14 days would be the same for each person in the study

In this chapter we introduce the technique of multilevel modeling as a means of overcoming these limitations of repeated measures ANOVA The multilevel approach, also commonly referred t o as hierarchical linear mod- eling, provides a very general strategy for analyzing these d a t a structures and can easily handle unbalanced designs and designs with continuous pre- dictor variables In introducing multilevel modeling, we focus our attention

on traditional estimation procedures (ordinary least squares and weighted least squares) t h at , with balanced data, produce results identical t o those derived from ANOVA techniques We also introduce nontraditional esti- mation methods t ha t are used more extensively in subsequent chapters

We begin by introducing a research question on how gender of interac- tion partner affects interaction intimacy We follow this by presenting a n artificial, balanced da ta set on this topic and provide a brief overview of the standard ANOVA approach t o analyzing such a d a t a set We then intro- duce a real d a t a set in which the d at a are not balanced, and we consider an alternative to the ANOVA model, the multilevel model Finally, we com- pare the least-squares estimation approaches described in this chapter t o the maximum likelihood estimation approaches discussed in other sections

Estimating Multilevel Models 3

a set of measures, including the interaction partner’s gender and interac- tion intimacy, for every interaction that he or she has over a fixed interval

In our study, each of 80 subjects (40 of each gender) interacts with six partners, three men and three women The study permits the investigation

of the degree t o which the gender of an interaction partner predicts the level of perceived intimacy in interactions with that partner One can also test whether this relationship varies for men versus women, t h a t is, women may have more intimate interactions with male partners, whereas men have more intimate interactions with female partners

Using conventional ANOVA t o analyze the data from this study would result in a source table similar to that presented in Table 1.1 In the table, partner gender is symbolized as X , subject gender is denoted as 2 , and S

represents subjects Listed in the table are the sources of variance, their degrees of freedom, and the error terms for the F tests (the denominator of

the F ratio) t h a t evaluate whether each effect differs significantly from zero The multilevel modeling terms that correspond t o each effect are presented

in the last column of the table These terms are introduced later in the chapter It is helpful t o have an understanding of the different sources of variance The between-subject variation in Table 1.1 refers t o the variation

in the 80 means derived by averaging each subject’s intimacy ratings over the six partners This between-subject variation can be partitioned into three sources, the grand mean, subject gender (Z), and subject within gender ( S / Z ) The mean term represents how different the grand mean

is from zero, and the subject gender variation measures whether men or women report more intimacy across their interactions The third source of variation results from differences between subjects within gender Within the group of males and females, do some people report more or less intimacy

in their interactions?

The within-subject variation refers t o differences among partners for each subject: Do people differ in how intimate they see their interactions with their six partners? The partner gender effect ( X ) refers t o whether in- teractions with male versus female partners are more intimate T h e partner gender by subject gender interaction ( X by 2) refers t o whether same or opposite gender interactions are seen as more intimate T h e partner gender

by subject interaction ( X by S / Z ) is the variation in the effect of gender

of partner for each subject (i.e., to what degree does the mean of female partners minus the mean of male partners vary from subject t o subject) Finally, there is variation due t o partner ( P / X S / Z ) , and the issue is how much the intimacy ratings of interactions with partners differ from one an- other controlling for partner gender Each person reports about three male and three female partners, and this source of variance measures how much variation there is in intimacy across interactions with partners who are of the same gender Because in this example participants interact with a given

‘We use subject t o refer t o the research participants so t h a t subjects ( S ) can easily

be distinguished from partners ( P ) in our notation

4 Kenny, Bolger, and Kashy

Table 1.1

ANOVA Source Table for the Hypothetical Balanced Case

Within this model, there are three random effects: Subject ( S / Z ) , Sub- ject x Partner Gender ( X by S / Z ) , and Error ( P / X S / Z ) It is possible to

x Partner Gender, and Error variances The subject variance, symbolized

tion of this chapter, measures variation in average intimacy scores after controlling for both subject and partner gender The Subject x Partner Gender variance] symbolized as o f 2 , measures the degree to which the ef- fects of Partner Gender differ from subject to subject after controlling for the subject’s gender Denoting a as the number of levels of X ( a = 2 in this example) and b as the number of partners within one level of X ( b =

are given by

Subject: od2 = ( M S s I Z - M S p / x s / z ) / a b (1.1) Subject x Gender of Partner: o f 2 = (MSXbyS/Z - M S p / x s / z ) / b (1.2)

As noted, an exact estimate of the partner variance cannot be obtained because it is confounded with error variance] and so we represent the com- bination of partner variance and error variance as o e 2 Finally, although not usually estimated] we could compute the covariance between Subject and Subject x Partner Gender by computing the covariance between the

Estimating Multilevel Models 5

mean intimacy of the subject and the difference between his or her intimacy with male and female partners Such a covariance would represent the ten- dency of those who report greater levels of intimacy t o have more intimate interactions with female (or male) partners Although this covariance is hardly ever estimated within ANOVA, the method still allows for such a covariance

The table also presents the usual mixed model error terms for each of the sources of variance For the fixed between-subjects sources of variance,

M S s / z is the error term To test whether there are individual differences

in intimacy, M S s / Z is divided by M S p / s x / z The error term for the fixed within-subject effects is M S x x s / z Finally, the error term for M S x s / z

is M S p l s x l z , which itself cannot be tested

MULTILEVEL MODELS

Multilevel Data Structure

The ANOVA decomposition of variance just described only applies t o the case of balanced data For unbalanced data, a multilevel modeling ap- proach becomes necessary A key t o understanding multilevel models is

t o see that these data have a hierarchical, nested structure Although re- searchers typically do not think of repeated measures data as being nested,

it is the case that the repeated observations are nested within persons In hierarchically nested data with two levels, there is an upper-level unit and a lower-level unit Independence is assumed across upper-level units but not lower-level units For example, in the repeated measures context, person

is typically the upper-level unit, and there is independence from person t o person Observation is the lower-level unit in repeated measures data, and the multiple observations derived from each person are not assumed t o be independent Predictor variables can be measured for either or both levels, but the outcome measure must be obtained for each lower-level unit The following example should help to clarify the data structure

Example Data Set

As an example of the basic data structure, we consider a study conducted

by Kashy (1991) using the RIR In the Kashy study, persons completed the

RIR for 2 weeks Like the previous balanced-data example, this study inves-

tigated the degree t o which partner gender predicts the level of perceived intimacy in interactions with that partner and whether this relationship differs between men and women

Because persons often interacted more than once with the same partner,

we computed the mean intimacy across all interactions with each partner that is, for the purposes of this example, we created a two-level data set

in which subject is the upper-level unit and partner is the lower-level unit There are 77 subjects (51 women and 26 men) and 1,437 partners in the

6 Kenny, Bolger, and Kashy

study The number of partners with whom each person interacted over the data collection period ranged from 5 t o 51 The average intimacy across all interactions with a particular partner is the outcome variable, and it is measured for every partner with whom the person interacted

Partner gender, symbolized as X , is the lower-level predictor variable

Note that X can be either categorical as in the case of partner gender ( X =

-1 for male partners and X = 1 for female partners) or it can be continuous (e.g., the degree t o which the person finds the partner t o be attractive) Subject gender is the upper-level predictor variable and is denoted as 2

In repeated measures research, upper-level predictor variables may be ex- perimentally manipulated conditions t o which each subject is randomly assigned or person-level variables such as gender, a person’s extroversion, and so on If Z were a variable such as person’s extroversion, it would he a continuous predictor variable, but because 2 is categorical in the example,

it is a coded variable ( 2 = -1 for males and 2 = 1 for females) Finally, the outcome variable, average intimacy of interactions with the partner, is measured on a seven-point scale and is symbolized as Y

Because a second example in which the X variable is continuous is

helpful, we make use of the fact that Kashy (1991) also asked subjects t o evaluate how physically attractive they perceived each of their interaction partners t o be Ratings of the partner’s attractiveness were centered by subtracting the grand mean across subjects from each score (We feel that

it is generally inadvisable t o center X for each subject, so-called group centering.) The second example addresses whether interactions with part- ners who are seen as more physically attractive tend t o be more intimate

We can also use subject gender as an upper-level predictor variable, which allows us t o test whether the relationship between attractiveness and inti- macy differs for male and female subjects

So, in the example data set, subject is the upper-level unit, and subject

gender is the upper-level predictor variable or 2 Partner is the lower-level

unit and partner gender or partner’s physical attractiveness is the lower- level predictor or X Intimacy is the outcome variable or Y , and there is an average intimacy score for each partner The intimacy variable can range from 1 to 7, with higher scores indicating greater intimacy

MOST BASIC APPROACH TO MULTILEVEL MODELING: ORDINARY LEAST SQUARES

Although it is certainly possible for multilevel modeling t o be a challenging and complex data analytic approach, in its essence it is simple and straight- forward A separate analysis, relating the lower-level predictor, X , t o the outcome measure, Y , is conducted for each upper-level unit, and then the

results are averaged or aggregated across the upper-level units In this section we introduce the ordinary least squares (OLS) approach t o multi- level modeling without reference t o formulas Specific formulas describing

Estimating Multilevel Models 7

multilevel analyses follow

Using the partner’s physical attractiveness example, this would involve computing the relationship between a partner’s attractiveness and inter- action intimacy with t ha t partner separately for each subject This could

be done by conducting a regression analysis separately for each subject, treating partner as the unit of analysis In the Kashy (1991) example, this would involve computing 77 separate regressions in which attractiveness is the predictor and intimacy is the criterion

Table 1.2 presents a sample of the regression results derived by predict- ing average interaction intimacy with a partner using partner attractiveness

as the predictor For example, Subject 1 had an intercept of 5.40 and a slope of 1.29 The intercept indicates th at Subject 1’s intimacy rating for a partner whom he perceived t o be of average attractiveness was 5.40 The slope indicates t ha t, for this subject, interactions with more attractive partners were more intimate, t ha t is, one could predict t h a t , for Subject

1, interactions with a partner who was seen t o be 1 unit above the mean

on attractiveness would receive average intimacy ratings of 6.69 Subject

4, on the other hand, had an intercept of only 2.20 and a slope of -.37 So,

not only did this subject perceive his interactions with partners of average attractiveness t o be relatively low in intimacy but he also reported tha t in- teractions with more attractive partners were even lower in intimacy Note

t h a t , at this stage of the analysis, we do not pay attention t o any of the statistical significance testing results Thus, we do not examine whether each subject’s coefficients differ from zero

The second part of the multilevel analysis is t o aggregate or average the results across the upper-level units If the sole question is whether the lower-level predictor relates t o the outcome, one could simply average the regression coefficients across the upper-level units and test whether the average differs significantly from zero using a one-sample t test For the attractiveness example, the average regression coefficient is 0.43 The test

th at the average coefficient is different from zero is statistically significant

[t(76) = 8 4 8 , ~ < .001] This coefficient indicates th at there is a signifi- cant positive relationship between partner’s attractiveness and interaction intimacy such t ha t , on average, interactions with a partner who is one unit above the mean on attractiveness were rated as 0.43 points higher in inti- macy If meaningful, it is also possible t o test whether the average intimacy ratings differ significantly from zero or some other theoretical value by av- eraging all of the intercepts and testing the average using a one-sample t

test

It is very important t o note that the only significance tests used in multilevel modeling are conducted for the analyses th at aggregate across upper-level units One does not consider whether each of the individual regressions yields statistically significant coefficients For example, it is normally of little value t o tabulate the number of persons for whom the X

variable has a significant effect on the outcome variable

When there is a relevant upper-level predictor variable, 2, one can ex-

8 Kenny, Bolger, and Kashy

Table 1.2

A Sample of First-Step Regression Coefficients Predicting Interaction

Intimacy with Partner’s Physical Attractiveness

Estimating Multilevel Models 9

amine whether the coefficients derived from the separate lower-level regres-

sions vary as a function of the upper-level variable If 2 is categorical, a t

test or an ANOVA in which the slopes (or intercepts) from the lower-level regressions are treated as the outcome measure could be conducted For example, the attractiveness-intimacy slopes for men could be contrasted with those for women using an independent groups t test The average slope for men was M = 0.38 and for women M = 0.45 The t test that the two average slopes differ is not statistically significant, t(75) = 0.70,

ns Similarly, one could test whether the intercepts (intimacy ratings for partners of average attractiveness) differ for men and women In the ex- ample, the average intercept for men was M = 3.78 and for women M =

4.31, t(75) = 2.19, p = .03, and so women tended to rate their interactions

as more intimate than men Finally, if 2 were a continuous variable, the

analysis that aggregates across the upper-level units would be a regression analysis In fact, in most treatments of multilevel modeling, regression is the method of choice for the second step of the analysis as it can be applied

t o both continuous and categorical predictors

Multilevel Model Equations

In presenting the formulas that describe multilevel modeling, we return t o the example that considers the effects of subject gender and partner gender

on interaction intimacy As we have noted, estimation in multilevel models can be thought of as a two-step procedure In the first step, a separate regression equation, in which Y is treated as the criterion variable that is

predicted by the set of X variables, is estimated for each person In the

formulas that follow, the term i represents the upper-level unit, and for the Kashy example i represents subject and takes on values from 1 to 77; j

represents the lower-level unit, partner in the example, and may take on a different range of values for each upper-level unit because the data may be unbalanced For the Kashy example, the first-step regression equation for

person i is as follows:

K j = boi + b l i X i j + eij (1.3)

where boi represents the intercept for intimacy for person i, and bli

represents the coefficient for the relationship between intimacy and partner gender for person i Table 1.3 presents a subset of these coefficients for the

example data set Given the way partner gender, or X , has been coded (-1,

l ) , the slope and the intercept are interpreted as follows:

boi: the average mean intimacy across both male and female

partners

b1i: the difference between mean intimacy with females and

mean intimacy with males divided by two

10 Kenny, Bolger, and Kashy

Table 1.3 Predicting Interaction Intimacy with Partner’s Gender: Regression Coefficients, Number of Partners, and Variance in Partner Gender

Estimating Multilevel Models 11

Consider the values in Table 1.3 for Subject 1 The intercept, boi, indi- cates that across all of his partners this individual rated his interactions t o

be 5.35 on the intimacy measure The slope, b l i , indicates that this person rated his interactions with female partners t o be 1.52 (0.76 X 2) points higher in intimacy than his interactions with male partners

For the second-step analysis, the regression coefficients from the first step (see Equation 1.3) are assumed to be a function of a person-level predictor variable 2 :

boi = a0 + a122 + di

bli = co + ClZi + fi

(1.4) (1.5) There are two second-step regression equations, the first of which treats the first-step intercepts as a function of the 2 variable and the second of which treats the first-step regression coefficients as a function of 2 In general, if there are p variables of type X and q of type 2 , there would be p

+ 1 second-step regressions each with q predictors and an intercept There are then a total of p ( q + 1) second-step parameters The parameters in Equations 1.4 and 1.5 estimate the following effects:

ao: the average response on Y for persons scoring zero on both

X and 2

a l : the effect of 2 on the average response on Y

co: the effect of X on Y for persons scoring zero on 2

c1: the effect of 2 on the effect of X on Y

Table 1.4 presents the interpretation of the four parameters for the example For the intercepts ( b o i , ao, and CO) t o be interpretable, both X

and 2 must be scaled so that either zero is meaningful or the mean of the variable is subtracted from each score (i.e., the X and 2 variables are centered) In the example used here, X and 2 (partner gender and gender

of the respondent, respectively) are both effect-coded (-1, 1) categorical variables Zero can be thought of as an “average” across males and females The estimates of these four parameters for the Kashy example d a t a set are presented in the OLS section of Table 1.5

As was the case in the ANOVA discussion for balanced data, there are three random effects in the multilevel models First, there is the er- ror component, eij, in the lower-level or first-step regressions (see Equa- tion 1.3) This error component represents variation in responses across the lower-level units after controlling for the effects of the lower-level pre- dictor variable, and its variance can be represented as 0: In the example, this component represents variation in intimacy across partners who are

of the same gender (it is the partner variance plus error variance that was discussed in the ANOVA section) There are also random effects in each of

12 Kenny, Bolger, and Kashy

Table 1.4 Definition of Effects and Variance Components for the Kashy Gender of

Subject by Gender of Partner Example

Multilevel Effect Estimate Parameter Definition of Effect

all subjects and partners

interactions as more intimate than males

with female partners are seen as more intimate than those with male partners

gender effect is different for male and female subjects

a d 2 Individual differences in the typ-

ical intimacy of a subject's in- teractions, controlling for part- ner and subject gender

Individual differences in the ef- fect of partner gender, control- ling for subject gender

Wihin-subject variation in inter- action intimacy, controlling for partner gender (includes error variance)

Table 1.5

Estimates and Tests of Coefficients and Variance Components for the Kashy Gender

of Subject of Partner Example

the SAS GLM procedure, and HLM, respectively

14 Kenny, Bolger, and Kashy

the two second-step regression equations In Equation 1.4, the random ef- fect is di and it represents variation in the intercepts that is not explained

by 2 Note th at di in this context is parallel t o M S ~ I Z within the bal- anced repeated measures ANOVA context, as shown in Equation 1.1 The variance in di is a combination of o i , which was previously referred t o as Subject variance, and cr," Finally, in Equation 1.5, the random effect is fi

and represents variation in the gender of partner effect Note that f i here

is parallel t o M S X b y S / Z within the repeated measures ANOVA context, as shown in Equation 1.2 The variance in f i is a combination of 0; , which was previously referred t o as the Subject by Gender of P a r h e r variance, and

0," A description of these variances for the example is given in Table 1.4

Recall t ha t it was possible to obtain estimates of 0; and 0; for balanced designs by combining means squares As can be seen in Equations 1.1 and 1.2, in the balanced case the formulas involve a difference in mean squares divided by a constant In the unbalanced case (especially when there is a continuous X ) , this constant term becomes quite complicated Although

we believe a solution is possible, so far as we know none currently exists The multilevel model, with its multistep regression approach, seems radically different from the ANOVA model However, as we have pointed out in both the text and Table 1.1, the seven parameters of this multilevel model correspond directly t o the seven mean squares of the ANOVA model for balanced data Thus, the multilevel model provides a more general and more flexible approach t o analyzing repeated measures d a t a than th at given

by ANOVA, and OLS provides a straightforward way of estimating such models

if the computer package t ha t performs the first-step regressions can be used

Estimating Multilevel Models 15

t o create automatically a data set that contains the first-step regression es- timates Although this can be done within SAS using the OUTEST = data

s e t name COVOUT options for PROC REG, it can be rather challenging because SAS creates the output d at a set in matrix form Regardless of how the da t a set is created, the coefficients in it serve as outcome measures in the second-step regressions

Complications in Estimation with Unbalanced Data

The OLS approach t o multilevel modeling allows researchers t o analyze unbalanced d at a th at cannot be handled by ANOVA As we have noted, there are two major reasons th at da ta are not balanced First, persons may have different numbers of observations This is the case in Kashy da t a set where the number of partners varies from 5 t o 51 Second, even if the number of observations were the same, the distribution of X might vary

by person In the example, X is partner gender, and the distribution of

X does indeed vary from person to person and so the variance of X differs (see Table 1.3) As noted earlier, data are unbalanced if either the number

of observations per person is unequal or the distribution of the X variables differs by person Note th at a study might be designed t o be balanced, but one missing observation makes the da ta set unbalanced

MULTILEVEL ESTIMATION METHODS THAT WEIGHT THE SECOND-STEP REGRESSIONS

The OLS approach does not take into account an important ramification

of unbalanced data: The first-step regression estimates from subjects who supply many observations, or who vary more on X , are likely in principle

t o be more precise than those from subjects who supply relatively few observations or who vary little on X A solution t o this problem is t o weight

the second-step analyses t h at aggregate over subjects by some estimate of the precision of the first-step coefficients How best t o derive the weights

th at are applied t o the second-step analyses is a major question in multilevel modeling, and there are two strategies that are used: weighted least squares (WLS) and maximum likelihood (ML) Because the NIL approach is treated

in detail in other chapters in the volume, we focus most of our attention on the WLS solution However, we later compare WLS, as well as OLS, with

ML

Multilevel Modeling with Weighted Least Squares

Expanding the multilevel model from an OLS solution t o a WLS solution

is relatively straightforward As in OLS, in the WLS approach a separate analysis is conducted for each upper-level unit This first-step analysis

is identical t o t ha t used in OLS, as given in Equation 1.3 The second- step analysis also involves estimating Equations 1.4 and 1.5 However, in

16 Kenny, Bolger, and Kashy

the WLS solution, Equations 1.4 and 1.5 are estimated using weights that represent the precision of the first-step regression results

The key issue then is how t o compute the weights In WLS, the weights are the sums of squares for X or SSi (Kenny et al., 1998) This weight is a function of the two factors that cause data t o be unbalanced: The number

of lower-level units sampled (partners in the example), a,nd the variance of

X (partner gender in the example)

Multilevel Modeling with Maximum Likelihood

The major difference between ML and WLS solutions t o multilevel model- ing is how the weights are computed The ML weights are a function of the standard errors and the variance of the term being estimated (see chapter

5 for greater detail) For example, the weight given t o a particular boi is a

function of its standard error and the variance of d i ML weighting is statis-

tically more efficient than WLS weighting, but it is computationally more intensive There is usually no closed form solution for the estimate, that is, there is no formula that is used t o estimate the parameter Estimates are obtained by iteration and the estimates that minimize a statistical criterion are chosen In ML estimation, the first and second-step regressions are esti- mated simultaneously Several specialized stand-alone computer programs have been written that use ML t o derive estimates for multilevel data: HLM/2L and HLM/3L (Bryk, Raudenbush, & Congdon, 1994), MIXREG (Hedeker, 1993), MLn (Goldstein, Rasbash, & Yang, 1994), and MLwiN (Goldstein et al., 1998) Within major statistical packages, SAS’s PROC MIXED and BMDP’s 5V are available

PROGRAMS

The estimation of separate regression equations is awkward and computa- tionally inefficient Moreover, this approach does not allow the researcher

t o specify that the X effect is the same across the upper-level units It

is possible t o perform multilevel analyses that yield results identical t o those estimated using the LLseparate regressions” WLS approach but that are more flexible and less awkward This estimation approach treats the lower level or observation as the unit of analysis but still accounts for the random effects of the upper level We illustrate the analysis using SAS’s

GLM procedure as an example The analysis could be accomplished within most general linear model programs We use SAS because it does not re- quire that the user create dummy variables, but other statistical packages could be used The WLS analysis that we describe requires that a series

of three regression models be run, and then the multilevel parameters and tests are constructed from the results of these three models

Lower-Ievel units are treated as the unit of analysis In other words,

Estimating Multilevel Models 17

each observation is a separate data record Each record has four variables: the lower-level predictor variable X , the upper-level predictor variable Z,

the outcome variable Y , and a categorical variable, called PERSON in the example t h at follows, which identifies each individual or upper-level unit in the sample In the first run or Model 1 the setup is:

PROC GLM

CLASS PERSON

The mean square error from the model is the pooled error variance or sz

Also, the F tests (using SAS’s Type I11 Sum of Squares) for both PERSON and PERSON by X are the WLS tests of the variance of the intercepts

( s i ) and the variance of the slopes ( s : ) , respectively Note that this model

supplies only the tests of the intercept and slope variances The other tests are not WLS tests and should be ignored ’

PROC GLM

CLASS PERSON

This model gives the proper estimates for main effect of X (co) and the

Z by X interaction (c1) (see Equation 1.5) The SOLUTION option in the MODEL statement enables these estimates to be viewed Mean squares for these terms are tested using the PERSON by X mean square (SAS’s

Type 111) from Model 1 as the error term If there are multiple X variables,

Model 2 must be re-estimated dropping each PERSON by X interaction

The term INT is added so that the intercept can be viewed This model gives the estimates of the Z effect ( a l ) and the overall intercept (ao) from Equation 1.4 The mean squares for these terms are tested using the

PERSON Mean Square (Type 111) from Model 1

If there were two X variables, X1 and X z , then Model 2 would be

estimated twice In one instance, the PERSON by X 1 term would be

dropped; however, the effects of the both X1 and Xa would remain in the

equation as well as the PERSON by X 2 interaction In other instance, the PERSON by X z term would be dropped; however, the effects of the both

’The reader should be warned t h a t , in the output, the Z effect has zero degrees of

freedom This should be ignored

18 Kenny, Bolger, and Kashy

interaction If there were more than one 2 variable, they could all be tested using a single Model 3

The results from the tests of the variances of Model 1 have important consequences for the subsequent tests If there were evidence t h a t an effect (e.g., f ) does not significantly vary across upper-level units and so s$ is

not statistically significant, Model 1 should be re-estimated dropping that term In this case, instead of using that variance as an error term for other terms in Model 2, those terms can be tested directly within Model 1 using the conventional Model 1 error term So if s: is not included in the model,

co and c1 would be tested using the sz Rarely, if ever, is the variance of the

intercepts not statistically significant However, if there was no intercept variance, a parallel procedure would be used t o test no and n l

Table 1.5 presents the OLS, WLS, and ML results for the Kashy data set

described previously The ML estimates were obtained using the HLM program (Bryk et al., 1994)

Model 1 is estimated first t o determine whether there is significant vari- ance in the intercepts and slopes across persons There is statistically sig- nificant evidence of variance in the intercepts [F(75,1283) = 8 2 2 , ~ <

.001]; however, there is not evidence that the slopes significantly vary [F(75,1283) = 1 2 2 , ~ = l o ) We adopt the conservative approach and treat the slopes as if they differed

We see that the intercept is near the scale midpoint of four Because effect coding is used, effects for respondent gender, partner gender, and their interaction must be doubled t o obtain the difference between males and females We see from the subject gender effect t h a t females say that their interactions are more intimate than reported by males by about half

a scale point The p a r t n e r effect indicates that interactions with females are perceived as one tenth of a point more intimate than interactions with males Finally, the interaction coefficient indicates t h a t opposite-gender interactions are more intimate than same-gender interactions

One feature t o note in Table 1.5 is the general similarity of the estimates This illustrates how WLS and even OLS can be used t o approximate the more complicated ML estimates Of course, this is one example and there must be cases in which ML is dramatically different from the least-squares estimators We discuss this issue further in the following section

COMPARISON BETWEEN METHODS

In this section we consider the limitations and advantages of OLS, WLS, and ML estimation The topics that we consider are between and within slopes, scale invariance, estimation of variances and covariances, statistical efficiency, and generality

Estimating Multilevel Models 19

Between and Within Slopes

The coefficient b l i measures the effect of X on Y for person i In essence, OLS and WLS average these b l i values t o obtain the effect of X on Y

However, there is another way to measure the effect of X on Y We can

on mean X (again weighting in the statistically optimal way) treating per-

son as the unit of analysis So for the example, we could measure the effect having more female partners on the respondent’s overall level of intimacy

We denote this effect as bg and the average of the b l i or within-subject coefficients as bw

Figure 1.1 illustrates these two different regression coefficients There are three persons, each with four observations denoted by the small-filled circles We have fitted a slope for each person, designated by a solid line

We can pool these three slopes across persons t o compute a common, pooled within-person slope or bw This slope is shown in the figure as the dashed

line that we fitted for each person The figure also shows the three points

through which bB is fitted (the large-filled circles) The slope bB is fitted

through these points and is shown by the large dashed line

There are then two estimates of the effect of X on Y : bw and b s

In essence, bw is an average of the persons’ slopes, and bg is the slope

computed from the person means For the Kashy data set, we estimated these two slopes for the effect of partner gender on perceived intimacy The value for bw is 0.056, indicating that interactions with female partners are seen as more intimate However, the value for bB is negative being -

0.217 This indicates that people who have relatively more female partners

20 Kenny, Bolger, and Kashy

viewed their interactions as less intimate (The coefficient is not statistically significant )

The ML estimate, as we have described it, of the effect of X on Y is

a compromise of the two slopes of bw and bB whereas the WLS and OLS estimates use only a version of bw Note that in Table 1.5 the ML estimate

for this effect ( X ) is somewhat lower than the WLS estimate because ML

uses the negative between slope In our experience, these two slopes are typically different, and, as the example shows, sometimes they even have different signs So, it is a mistake t o assume without even testing that the two slopes are the same The prudent course of action is to compute both slopes and evaluate empirically whether they are equal If different, in most applications we feel t h a t bw is the more appropriate

To estimate both slopes the following must be done: create an additional predictor variable that is the mean of the X , for each person (Bryk & Rau- denbush, 1992) Thus, there are two X predictors of Y : X i j and the mean

X T h e slope for X i j estimates bw and the slope for mean X estimates bg

Alternatively, the X variables can be “group-centered” by removing the

subject mean for each variable (for more on centering in multilevel models see Kreft, de Leeuw, & Aiken, 1995)

We should note t h a t , in the balanced case, mean X does not vary, and

so bw can be estimated but bB is not identified Perhaps, the balanced case has misled us into thinking that there is just one X slope ( b w ) when in

fact in the unbalanced case there are almost always two (that may or may not be equal)

Scale Invariance

There is a serious limitation t o WLS estimation that is not present in either

ML or OLS Second-stage estimates using WLS estimation of intercepts are not scale invariant, that is, if an X variable were transformed by adding

a constant t o it, the WLS second-step solution for the intercepts cannot ordinarily be transformed back into the original solution The reason for this lack of invariance is that the weights used in the step-two equations differ after transformation The standard error for the intercept increases

as the zero point is farther from the mean Because of the differential weighting of the intercepts, estimates of cell “means,” using the intercepts, will not be the same

To illustrate this problem using the sample d a t a set, we recoded the d a t a using dummy coding (males = 0, females = 1) instead of effect coding for the both person and partner gender variables Table 1.6 presents the estimated cell means for the four conditions We see that there is a difference between the predicted ”means” and so the coding system matters

Because ML estimates the weights simultaneously, it does not t o have this problem Because OLS does not weight at all, OLS does not have

3However, if t he same equation were estimated twice (e.g., a n X variable is present

Estimating Mu1 tilevel Models 21

Table 1.6 Estimated Cell “Means” for the Four Conditions Using WLS

Person Partner Estimated Gel 1 “mean”

Gender Gender Effect Coding Dummy Coding

Estimation of Variances and Covariances

One major advantage of ML is that it directly provides estimates of vari- ances and covariances A procedure for obtaining WLS estimates of vari- ance has been developed (Kashy, 1991), but it is very complicated We know

of no appropriate method for estimating covariances within WLS Because slopes and intercepts are typically weighted differently, it is unclear how t o weight each person’s estimates t o form a covariance

It seems logically possible that estimates of both variance and covariance could be developed within OLS However, we know of no such estimates

If OLS were t o be used more in estimation in multilevel models, it would

be of value t o determine these estimators

ML has the strong advantage of providing estimates of these variances and covariances Unfortunately, we should note that all too often these terms are largely ignored in the analysis Most of the focus is on the fixed effects Very often the variances and covariances are as important as the fixed effects Knowing that X has the same effect on Y for all subjects

(i.e., sz is zero) can often be a very interesting result because it implies that effect of X on Y is not moderated by individual differences

in one equation and dropped in the other), ML is likely to weight the effect differently

in the two equations This differential weighting creates difficulties in the decomposition

of indirect effects in mediation

22 Kenny, Bolger, and Kashy

Statistical Efficiency

If we assume that the statistical model is correct, OLS is the least efficient, WLS the next, and ML the most The complex weighting of ML creates this advantage We wonder, however, whether this advantage may at times

be more apparent than real Consider the Kashy study For both ML and WLS, why should people who have more partners count more than those with fewer? Statistically, more is better, but that may not be the case in all repeated measures studies

Perhaps, if there is a disparity in the number of observations per person, the researcher might want t o test if number of observations (perhaps log transformed) is a moderating variable, that is, does the effect of X on Y

increase or decrease when there are more observations? Number of observa- tions would then become a Z variable entered in the second-step equations

We estimated such a model with the Kashy data and did not find evidence for moderation, but we did find a trend that persons with more interaction partners reported lower levels of intimacy

Generality

There are several complications of the model that we might want t o con- sider First, the outcome variable, Y , may be discrete, not continuous For instance, in prevention studies, the outcome might be whether the person has a deviant status or not Second, X or Y may be latent variables In social-interaction diary studies, there may be several outcomes (intimacy, disclosure, and satisfaction) that measure the construct of relationship qual- ity It may make sense t o treat them as indicators of a latent variable Third, we have assumed that after removing the effect of the X , the errors are independent However, the error may be correlated across time, per- haps with autoregressive structure Fourth, the distribution of errors may have some other distribution besides normal (e.g., log normal) Typically, behavioral counts are highly skewed and so are not normal Fifth, the vari- ance in the errors may vary by person Some people may be inherently more predictable than others

Increasingly, ML programs allow for these and other complications However, it would be difficult if not impossible t o add these complications

t o a least-squares estimation solution Thus, ML estimation is much more flexible than least-squares estimation

SUMMARY

Multilevel modeling holds a great deal of potential as a basic data analytic approach for repeated measures data An important choice that researchers will have to make is which multilevel estimation technique t o use Although statistical considerations suggest that ML is the best estimation technique

Estimating Mu1 tilevel Models 23

t o use because it provides easy estimates of variance and covariance com- ponents, is flexible, and provides estimates that are scale invariant, there are times that OLS might also be very useful We should note t h a t ML estimation is iterative, and sometimes there can be a failure t o converge on

a solution Moreover, ML estimation, as conventionally applied, pools the between and within slopes without evaluating their equality Therefore, when ML is used in an unsophisticated manner, it is possible t o end up confounding what may be conceptually very different effects

OLS approaches are familiar and easy t o apply, and results generated

by OLS generally agree with those produced by ML WLS has some advan- tages over OLS Its estimates are more efficient and estimates of variance components are possible However, it suffers from the problem that the intercept estimates are not scale invariant

Notably, if the d a t a set is balanced or very near balanced, there is only a trivial difference between the different techniques ML estimation still has the advantage that variance components can always be estimated, but, if the design is perfectly balanced, the variance components can be estimated and tested using least squares A major advantage of both OLS and WLS solutions is that they can be accomplished by using conventional software (although SAS’s PROC MIXED is available for ML) Thus, a researcher can use conventional software t o estimate the multilevel model

WLS and OLS may serve as a bridge in helping researchers make the transition from simple ANOVA estimation t o multilevel model estimation

It may also be a relatively easy way t o estimate multilevel models without the difficulties of convergence and iteration Finally, and most importantly,

it can provide a way for researchers who are not confident that they have successfully estimated a multilevel model using new software t o verify that they have correctly implemented their model We have generations of re- searchers who are comfortable with ANOVA and who have difficulty work- ing with multilevel regression models These people can estimate models

Regardless how the researcher estimates a multilevel model, we strongly urge the careful probing of the solution Even the use of standard ANOVA

is complicated, and errors of interpretation are all too easy t o make Re- searchers need t o convince themselves that the analysis is correct by trying out alternative estimation methods (some of which may be suboptimal), plotting raw data, and creating artificial d a t a and seeing if the analysis technique recovers the model’s structure We worry t h a t , in the rush t o use these exciting and extraordinarily useful methods, some researchers may not understand what they are doing and they will fail to make discoveries

t h a t they could have made using much simpler techniques