Assessment in counseling chapter 3

∗ Systematic versus unsystematic error∗ Reliability only takes unsystematic error into account Reliability... Reliability & Correlation∗ Reliability often based on consistency between t

Chapter 3

∗ Every observed score is a combination of true

score and error

Obs = T + E

∗ Reliability =

Classical Test Theory

s

s s

s

O

T O

E

2

2 2

2

∗ Systematic versus unsystematic error

∗ Reliability only takes unsystematic error into

account

Reliability

Reliability & Correlation

∗ Reliability often based on consistency between two sets of scores

∗ Correlation: Statistical technique used to examine consistency

Positive Correlation

Negative Correlation

∗ Correlation coefficient: a numerical indicator of the

relationship between two sets of data

∗ Pearson-Product Moment correlation coefficient is most common

Pearson-Product Moment Correlation Coefficient

r = ∑ z 1 z 2

N

∗ The percentage of shared variance between two sets of data

Coefficient of Determination

∗ Test-Retest

∗ Alternate/Parallel Forms

∗ Internal Consistency Measures

Types of Reliability

∗ Correlating performance on first administration with performance on the second

∗ Co-efficient of stability

Test-Retest

∗ Two forms of instrument, administered to same individuals

Alternate/Parallel Forms

∗ Split-half reliability

∗ Spearman-Brown formula

∗ Kuder-Richardson formulas

∗ KR 20

∗ KR 21

∗ Coefficient Alpha

Internal Consistency Measures

∗ Typical methods for determining reliability may not

be suitable for:

∗ Speed tests

∗ Criterion-referenced tests

∗ Subjectively-scored instruments

∗ Interrater reliability

Nontypical Situations

∗ Examine purpose for using instrument

∗ Be knowledgeable about reliability coefficients of other instruments in that area

∗ Examine characteristics of particular clients

against reliability coefficients

∗ Coefficients may vary based on SES, age,

culture/ethnicity, etc.

Evaluating Reliability Coefficients

r s

SEM = 1 −

Standard Error of Measurement

∗ Provides estimate of range of scores if someone were to take instrument repeatedly

∗ Based on premise that when individuals take a test

multiple times, scores fall into normal distribution

∗ Sam’s SAT Verbal = 550

∗ r = 91; s = 100

∗ SEM

SEM: Example

30

3 100

09 100

91 1 100

=

×

=

=

−

=

Determining Range of Scores Using

SEM

∗ Method to determine if difference between two

scores is significant

∗ Takes into account SEM of both scores

Standard Error of Difference

∗ Generalizability or Domain Sampling Theory

∗ Focus is on estimating the extent to which specific sources of variation under defined conditions are contributing to the score on the instrument

Alternative Theoretical Model