Summary of Research on Various Teacher Employment Screening Instruments 2

N=34 grades 1-6 TPI Gain score in math achievement No relationship Dissertation not available, thus information derived from abstract TPI scores dropped across grade levels ARAMBURO, DOR

Howard Ebmeier

University of Kansas

howard@ku.edu

8/18/2008

measure

Dependent Measure

Correlation /Effect Size

Reliability Comments

AARESTAD, JANNA SERINA (1980) ANALYSIS OF

TEACHER SELECTION/RETENTIONAND STUDENT

ACHIEVEMENT AS RELATED TO SELECTION

RESEARCH INCORPORATED'STEACHER

PERCEIVER INTERVIEW Ph.D dissertation, University

of Minnesota

N=34 grades 1-6 TPI Gain score in

math achievement No relationship Dissertation not available, thus information derived from abstract

TPI scores dropped across grade levels

ARAMBURO, DOROTHY JAMES (1981) THE

EFFICACY OF THE STRUCTURED INTERVIEWIN

THE SELECTION OF SPECIAL EDUCATION

TEACHERS Ed.D., dissertation, University of New

Orleans

N=179 special education teachers

TPI Multiple Teacher

Factor Survey

0.29 via canonical correlations among 12 TPI themes and 4 factors of MTFS

Dissertation not available, thus information derived from abstract

Not sure what the MTFS measured but apparently completed by the teacher

Antoline, Carlos Mitchell (2000) A validation study of the

Teacher Perceiver Interview Ph.D dissertation, Texas

Woman's University

N=349 employed by district teachers Teachers group into 4 categories

via TPI (not continuous variable as hoped)

Principal ratings Significant

difference but very small F value Only top vs bottom teachers were significantly different in post hoc tests

Cited in study is Gallup 0.70 reliability estimate for TPI

Student evaluations were not different

Ball, Jane Rowlett 1992) A validation study of students'

and parents' perceptions of the talent of teachers selected

using the SRI Teacher Perceiver interview Ed.D

dissertation, Kansas State University

N=11 One school TPI Parent, student

ratings

None reported Very small qualitative study Concludes that teachers

selected with TPI were liked by parents and students

—same could probably be said of all teachers however they were selected

measure Measure /Effect Size

Baskin, Maryan Kay (1994) The predictability of the

Urban Teacher Selection Interview to the classroom

success of elementary student teachers Ed.D dissertation,

The University of Memphis

N=68 student teachers

Star Teacher by Haberman

Practice Teaching Portfolio

The regression procedure found the Pass/Fail element of the Urban Teacher Selection Interview predicted success on the Communicatio

n aspect of the evaluation instrument

“Overall, however, the UTSI did not differentiate student teachers who would be evaluated as successful in other areas of teaching which were part

of the Practice Teaching Portfolio areas like planning, classroom management, and using effective teaching strategies Findings from this study indicate that Rank on the UTSI, which is emphasized by Haberman as the basis for selecting students, was not a significant predictor of the classroom success of student teachers.”

Bingham, Patrick Jerome (2000) The concurrent and

predictive validity of elementary school teacher

pre-employment success indicators Ph.D dissertation, The

University of Texas at Austin

N=138 teachers TPI Achievement on

TAAS (Texas assessment) Principal Rating

0.04 (reading) 0.03 (math) 0.07 (writing) -0.05 (principal rating)

Very small correlation that was significant but only because of large N-student (N=6600) unit of analysis

Brown, Karl Kevin (2004) The impact of the Gallup

Teacher Perceiver Interview on hiring teachers as

perceived by selected administrators in the Alamo Heights

Independent School District, San Antonio, Texas Ed.D

dissertation, Texas A&M University

Three groups Recommended by TPI (201), not recommended but hired (105), pre-TPI (602)

TPI Retention rates

Administrator perceptions

ANOVA across 3 groups was significant with difference due to not recommended different than pre and recommended

Retention rates of TPI recommended teachers were higher

Buresh, Richard John (2003) The predictive validity of the

Teacher Perceiver Interview in selecting effective

elementary teachers in amid-sized Midwestern school

district Ed.D dissertation, The University of North

Dakota

N=59 all grade levels TPI Panel of experts

scored teacher evaluations of principals

0.039 for total Also reports each scale

Contains good review of TPI previous research Reposts negative correlation with leave days

measure Measure /Effect Size

CHALKER, DONALD MERLE (1981) THE TEACHER

PERCEIVER INTERVIEW AS ANINSTRUMENT FOR

PREDICTING SUCCESSFUL TEACHING BEHAVIOR

Ed.D dissertation, Wayne State University

N=40 HS urban teachers in 3 districts

TPI Quartile rankings

by building principals Ratings by sample

of students

Significant correlations between admin ratings and TPI and student ratings and TPI

Dissertation not available, thus information derived from abstract

Correlation numbers absent from abstract

COOK, RONALD EUGENE (1981) CHIPPEWA

VALLEY LOCAL VALIDATION STUDY OF

THETEACHER PERCEIVER INTERVIEW PROCESS

Ed.D dissertation, Wayne State University

N=30 middle school teachers

High and low scoring scores were used for t-test

TPI Administrator

ratings, student ratings

Unclear, author concludes no significance but then states there was a positive t test and

correlations

Dissertation not available, thus information derived from abstract

CORNINE, LARRY LEE (1980) A VALIDATION

STUDY OF THE TEACHER PERCEIVER INTERVIEW

USING THE ILLINOIS TEACHER EVALUATION

QUESTIONNAIRE Ed.D., dissertation, University of

Kansas

N=All Ottawa, KS secondary teachers Experimental group=secondary teachers hired after TPI

Control=teachers before TPI

TPI Student ratings of

teacher competence

No relationship Dissertation not available, thus information derived from abstract

Chesley, Daryl (2004) The validity of teacher employment

success predictors Ed.D dissertation, The George

Washington University

N=587 teachers UTPI

Praxis

Principal ratings 0.144

(statistically significant but large N) Praxis not significant

Restricted range problems since they were all employed district teachers

measure Measure /Effect Size

teachers

Ventures Tenn value added

composite achievement scores

Reading was positive but rest were negative (rather strange since reading and math scores were positively correlated in study)

“there is little association between Ventures for Excellence scores and TVA scores”

Sample size too small to conclude much of anything

Delli, Dane Armand (2000) The predictive validity of the

Teacher Perceiver Interview on the teaching performance

of classroom teachers Ph.D dissertation, The Ohio State

University

N=72 teachers N=124 TPI Principal ratingsAbsence 0.37 if all themes are

added together

in one regression equation R**2 0.14

Little or none internal reliability in the various themes

Capture in article with Young Good review of the development and research behind TPI

Failed to composite score correlation

Diemer, Janaan S (1998) Identifying effective teachers of

behavior disordered students: Is the Gallup Teacher

Perceiver Interview an answer? Ed.D dissertation, New

Mexico State University

N=33 teachers (special ed) Identified

as top/bottom quadrant of TPI

TPI Supervisor

Ratings 0.00 Teachers deemed ineffective had higher average scores on the total and subscales that those deemed

effective by TPI Dunning, Christopher Michael (2005) Teacher Perceiver

interview tool: A contemporary validation study within a

Florida district Ed.D dissertation, University of South

Florida

N=40 teachers

20 NBC certified 20 not

786 Respondents for factor analysis from college interviews

NBC/non-NBC teachers (restricted range problem likely)

Cronback alpha ranged from 0.0-0.30

on the 10 scales

Factor analysis of first 24 questions on TPI (three factors suggested: interpersonal (0.35), intrapersonal (0.20), extrapersonal (0.45) Reliability of total instrument (0.63) Three factor model had good confirmatory factor fit (0.90) but one factor model was slightly better

95% of board certified teachers had TPI highly recommend

Expert panel supported 22/24 questions as important

measure Measure /Effect Size

RACHEL DAVIS (1984) THE RELATIONSHIP

BETWEEN THE TEACHERPERCEIVER INTERVIEW

AND INSTRUCTIONAL BEHAVIORS OF TEACHERS

OF LEARNINGDISABLED STUDENTS Ed.D

dissertation, University of Virginia

LD teachers N=? TPI The Classroom

Planning and Management Assessment (CPMA), based upon research of effective teaching

A non-significant correlation between the TPI total score and the CPMA total mean score

From abstract Some positive and significant correlations reported among various scales on the TPI and the dependent measure but overall no effect

Frey, Patricia (2003) Ability of the Urban Teacher

Selection Interview to identify teachers who are likely to

be retained in the Buffalo Public Schools Ed.D

dissertation, Seton Hall University, College of Education

and Human Services

N=876 teachers at all grade levels Star Teacher Retention 0.13 correlation

(low but significant given the large N)

Star Teacher instrument used in initial employment decision thus possible range restriction issues

Gillies, Terry Kim (1988) The relationship between

selection variables and subsequent performance ratings for

teachers in an Oregon school district Ph.D dissertation,

University of Oregon

N=196 first year teachers employed by one district

TPI Principal ratings Average of

0.17 (great variation over the 4 years of data collection

—one year was actually negative)

Overall correlation was significant but pretty low Range restriction effect likely since were employed

by district after results of TPI known

Howard, Mildred Anthony (1998) The relationships among

teacher selection predictors and teacher performance with

principals' opinions about a teacher selection process

Ph.D dissertation, Georgia State University

N=143 teachers (relatively new)

30 elementary principals

TPI Principal

evaluation of each teacher via a form reflecting values

of TPI

0.47 for all 6 scales entered collectively

Individual subscales correlated from 0.03-0.38 Regression was run for 6 of the theme variables but not total

measure Measure /Effect Size

JOHANNSEN, KENNETH LUDWIG (1981) THE

RELATIONSHIP OF A UNIQUE

TEACHERSELECTION METHOD TO STUDENT

ACHIEVEMENT Ed.D dissertation, Peabody College

for Teachers of Vanderbilt University

N=40 20 teacher who interviewed using TPI and 20 who were previous hires and had not used the TPI Teachers were balanced in terms of background

TPI Math gain score t-test indicated

no difference between groups

Dissertation not available, thus information derived from abstract

JONES, DAVID EDWARD, JR (1978) PREDICTING

TEACHING PROCESSES WITH THETEACHER

PERCEIVER INTERVIEW Educat D dissertation,

Virginia Polytechnic Institute and State University

Dissertation not available from KU library

Kanipe, Mary Lou Smith (1996) A study of the predictive

validity of the Urban Teacher Perceiver Interview in the

Knox County School System Ed.D dissertation, The

University of Tennessee

N=233 teachers—all grade levels

68 Principals

Urban TPI Principal

Evaluations 0.10 totalsubscales

ranged from 0.0-0.18

Correlation with minority teachers (0.39) indicated definite bias

May be Range restriction effectsince all teachers were screened with TPI prior to employment Koerner, Robert Jacob (2007) The relationship between the

Teacher Insight score and student performance as

measured by the Texas Growth Index Ed.D dissertation,

University of North Texas

N=123 3-11 grade

4500 students Teacher Insight TGI (Texas Growth Index) 0.045 English and reading

0.042 Math

Ceiling effects are present since only teachers hired are used

Kreifels, Robert G (2003) A study of selected teacher

interview characteristics and student achievement Ed.D

dissertation, Saint Louis University

N=260 elementary teachers (N=17 at 4/5 grade ?)

TPI (reported 3 subscales) 5

th reading

4th math Math—no significant

relationships with correlations ranging from 0.20 to –0.03 Reading 0.43

to 0.25 with no significant results

The sample size reported for the correlations was 17, which I cannot figure out

No relationship between ITBS and TPI found

measure Measure /Effect Size

LASHER, HUDSON FRED (1976) THE TEACHER

PERCEIVER INTERVIEW AS ANINSTRUMENT FOR

IDENTIFYING CHARACTERISTICS OF

PROSPECTIVETEACHERS Ed.D dissertation,

University of Wyoming

Students in education program

Compared college student scores with average national TPI scores

LEAHY, RICHARD EDGAR (1984) TEACHER

CHARACTERISTICS AND STUDENTACHIEVEMENT:

AN EXPLORATION OF THE EMPLOYMENT

INTERVIEW Ph.D dissertation, University of Minnesota

Document could not be located at KU

Loyd, David Eugene (2004) The effect of a screening

instrument to determine teacher effectiveness as measured

by academic gain scores Ed.D dissertation, The

University of Tennessee

N=242 teachers in

3-8 grades Urban TPI Tenn Value Added

achievement in math, reading, language

No significant main effect ANOVA F values were pretty low thus the correlation probably was close to zero

Sample was divided into 6 categories based on UTPI score and ANOVA conducted

MAUSER, PHILLIP J (1986) THE PREDICTIVE

VALIDITY OF THE TEACHERPERCEIVER

INTERVIEW TO TEACHER EFFECTIVENESS Ed.D

dissertation, Oklahoma State University

N=77 all levels TPI Principal

Evaluation 0.02 Excellent background review of the TPI development and older studies Early studies

indicated 0.7-0.8 KR internal reliability MILLS, JONATHAN KENDALL (1986) THE TEACHER

PERCEIVER INSTRUMENT AND ITSCORRELATION

WITH OBSERVER RATINGS OF TEACHER-PUPIL

RELATIONSHIPS (TEACHER INTERVIEWS,

CALIFORNIA, TEACHER EVALUATION) Ed.D

dissertation, Loma Linda University

N=? Seventh-day Adventist schools TPI Principal evaluation using

ORS McDaniel Observer Rating Scale

Concluded no relationship Dissertation not available, thus information derived from abstract

MOODY, WALTER EUGENE (1973) A COMPARISON

OF TEACHER PERCEIVER INTERVIEWGUIDE AND

MINNESOTA TEACHER ATTITUDE INVENTORY

SCORES WITH SUPERVISORRANKING OF SPECIAL

EDUCATION TEACHERS Ed.D dissertation,

University of Kansas

Document could not be located at KU

measure Measure /Effect Size

Martin, Linda J (2008) Searching for effective teachers: A

statistical analysis of the Ventures for Excellence Teacher

Interview questionnaire Ed.D dissertation, Seattle Pacific

University

Cronback

“Construct validity of the instrument was not confirmed”

Three factors were discovered: interpersonal sensitivity, engagement beliefs and strategies, and flexible thinking

Mills, Scott A (2007) An investigation of the predictive

validity of a structured teacher-interview instrument Ph.D

dissertation, Purdue University

N=61 grades 3-5 sample was used for other purposes N=38 teachers had

VE scores

VE interview Northwest

Evaluation Association’s Measure of Academic Progress in reading, LA, and math

0.36 (Reading) 0.33 (LA) 0.27 (math) 7%-13% of variance

Unclear if the teachers having VE scores were younger than those with no scores Appears to be a lot of data lost as many correlations were based on only 100 or so students

Neal, Stephen O (1997) A comparison of principals'

perceptions of teacher candidates and selected talent

profiles Ed.D dissertation, Northern Illinois University

12 TPI themes Did not report a great deal of overlap OVERMAN, WILLIAM F (1981) THE IMPORTANCE

OF THE TEACHER PERCEIVERINTERVIEW AS A

TEACHER SELECTION CRITERION TO

ADMINISTRATORS INRELATIONSHIP TO THEIR

LEADERSHIP STYLE Ed.D dissertation, University of

Northern Colorado

Dissertation not available, thus information derived from abstract

Qualitative study of how administrators perceived the value of the TPI and how they used the results

Ostlund, Charles N (2006) The Teacher Perceiver

Interview as apredictor of teacher retention in special

education Ed.D dissertation, University of Virginia

N=339 special education teachers

TPI Interview Rates of retention

of teachers

TPI explained almost no variance (full Dissertation not available, thus information derived from abstract)

Papiernik, Henry Joseph (1988) An analysis of rating

proficiency in the use of the teacher perceiver interview in

the selection of teachers Ed.D dissertation, Temple

University

N=127 principals up for recertification on the TPI

derived from abstract Original proficiency gained from the TPI training is maintained through recertification program

So what was learned? Isn’t that obvious?

measure Measure /Effect Size

Potthoff, Dennis Eugene (1991) A qualitative case study of

the relationship between the scores of selected students on

the SRI/UNL Teacher Perceiver Interview and their

subsequent student teaching success Ph.D dissertation,

The University of Nebraska - Lincoln,

Qualitative study of 5 elementary education majors Interview scores and classroom behavior were different

Ricker, Ann Marie McNeela (1999) Teacher selection:

Chicago Region One elementary neighborhood school

administrators since school reform Ed.D dissertation,

Northern Illinois University

in a candidate

Robison, Scott E (2003) An exploratory study of the

utility of a structured interview in the employment of

classroom teachers Ph.D dissertation, Purdue University

N=14 (half scored well the other half did not) 2-4 grade teachers N=263 students

VE Interview Northwest testing

data, student surveys, supervisor’s assessment

5/14 teachers moved to the other group after one year and re-interviewing using the VE

Comparison of high and low groups was significant

in LA and Math but not reading Comparison groups was not significant for student surveys Principal evaluations were handled in a qualitative manner

SIMMONS, BERTIE AUSTIN (1984) AN ANALYTICAL

AND COMPARATIVE STUDY OF THESELECTION

RESEARCH INTERVIEW AS AN INSTRUMENT FOR

SELECTING TEACHERS FORA LARGE, URBAN

SCHOOL DISTRICT Educat D dissertation, Texas

Southern University

N=164 teachers TPI Principal rating

Employment stability

No significant relationship via ANOVA 0.372 correlation that was not significant

Divided the teachers into 5 groups based on supervisor’s rating followed by ANOVA Also did a correlation which was NS

SIMMONS, JAMES ELWOOD (1976) A STUDY TO

TEST THE TEACHER PERCEIVERINTERVIEW AS AN

INSTRUMENT THAT WOULD SELECT VOCATIONAL

AGRICULTUREINSTRUCTORS THAT DEVELOP

POSITIVE RAPPORT WITH THEIR STUDENTS Ph.D

dissertation, The University of Nebraska - Lincoln

evaluations rapport Administrator ratings

0.40 with student ratings 0.37 with administrator ratings

Not all the TPI scales correlated but the total did

Sawyer, Gayle Buckheister (2005) A study using the Star

Teacher Selection Interview to predict the successful

performance of teachers in South Carolina's Program of

Alternative Certification for Educators Ph.D dissertation,

University of South Carolina

N=33 first year teachers from alternative certification program

Star Teacher Principals

evaluation

No predictive value

Dissertation not available, thus information derived from abstract

measure Measure /Effect Size

Schmidt, Wayne D (1993) The Teacher Perceiver

Interview as a predictor of teacher performance Ph.D

dissertation, University of Missouri -Columbia

N=154 hired and 144 not hired

TPI PBES-Principal

evaluation

0.015 for TPI and Principal Evaluation N=155

TPI was a good predictor if candidate was hired (TPI use as screening device thus results were obvious)

Simmons, Anne Carol Brown (1996) A validation study of

the use of the Teacher Perceiver Interview (TPI) in a

selected southeastern urban high school Ph.D

dissertation, Georgia State University

N=55 teachers TPI TPAQ-Principal

evaluation—

measure GTOI-Principal evaluation

unknown GTOI was not usable since little if any variation

TPAQ study was qualitative in nature Study of little use

THOMPSON, ALBERT RAY (1982) THE TEACHER

PERCEIVER INTERVIEW AS RELATEDTO STUDENT

ACHIEVEMENT Ed.D dissertation, University of

Northern Colorado

N=27 4th and 5th

grade teachers TPI Student scores on math, reading, and

la on the CTBS

TPI total score not related to achievement

Dissertation not available, thus information derived from abstract

Watson, Robert Lewis (1991) A study comparing the

profile of selected University of Wyoming student teachers

with the SRI Gallup Teacher Perceiver Interview Ed.D

dissertation, University of Wyoming

40 student teachers TPI Compared to TPI

national sample Of little value in assessing the TPI

ZARANEK, RICHARD JAMES (1983) A

CORRELATIONAL ANALYSIS BETWEEN

THETEACHER PERCEIVER INTERVIEW AND

TEACHER SUCCESS IN THE CHIPPEWA

VALLEYSCHOOL SYSTEM (MICHIGAN) Ed.D

dissertation, Western Michigan University

N=21 elementary N=29 secondary teachers

TPI Supervisor

Evaluation R=0.46 Elementary

teachers R=-0.12 Secondary teachers

Range restriction effect present Job satisfaction also included in results Note absolute differences between elementary and secondary teachers

Gatti, Sharon (1990) Predictors of future teacher

effectiveness for nontraditional teacher certification

candidates Ph.D dissertation, Colorado State university

Non-traditional student teachers

TPI plus a host

of other measures

Florida Performance Measurement System (FPMS)—

I assume completed by cooperating teacher

TPI not a significant predictor

Not very relevant for this study Dissertation not available, thus information derived from abstract