School-of-Education-Assessment-Handbook

6 KEY PROCESSES OF OUR ROBUST ASSESSMENT SYSTEM To enact the assessment system, we also align our assessment plan with the collection and reporting processes required by regional and nat

A SSESSMENT H ANDBOOK

Department of Human Development and Family Life Education

Department of Teacher Education Department of Curriculum and Instruction Department of Educational Leadership

Version 3.0 Approved by Faculty and Staff, November 15, 2018

2

PURPOSE OF THIS HANDBOOK

Samford University’s Orlean Beeson School of Education’s (herein, The School) mission is “to guide, develop and challenge students academically, professionally, personally and spiritually to prepare leaders who will positively influence and shape society.” The purpose of this handbook

is to articulate the importance of the role of assessment and describe the school’s assessment system Through our assessment system, we obtain objective information We use that

information to identify strengths and growth areas at the student, program, and school-wide level This information thus helps us develop the knowledge, skills, and dispositions of students

to achieve our mission

The School has been continuously accredited by NCATE since 1954, so this Handbook has been through many versions in its history Many of the previous versions were written for specific departments or shared with portions of the school We based the current Assessment

Handbook on the most comprehensive of these previous versions, incorporating revisions and best features of others It is intended to encompass all departments of the School, to ensure that assessment becomes a permanent part of our School culture, and to assist with assessment

and accountability

THE SCHOOL’S ORGANIZATIONAL STRUCTURE

Our school consists of four departments, each under the leadership of a chair: Teacher

Education (TE), Curriculum & Instruction (C&I), Human Development and Family Life Education

(HDFE), Educational Leadership (EdL) Across each department, our programs are housed under

the leadership of a director Appendix A contains a table listing the programs associated with each department and degree offered, along with each program’s associated specialized

organization (if any) Appendix B contains a short description of each specialized organization, elsewhere called Specialized Professional Associations (SPAs)

The Dean has a Leadership Committee, composed of those who serve as chairs of each

department This committee meets once a month

The School meets a minimum of once per semester for an entire “data day” devoted to

assessment In addition, every month, the School conducts a forum for faculty and staff, using assessment data to focus on continuous improvement and on maintaining relationships and communication required to facilitate changes

The Assessment Committee meets a minimum of once each semester to address concerns with

assessment, improve processes, collaborate with the School Curriculum Committee as needed, and maintain this Assessment Handbook The Assessment Committee is composed of faculty representatives from each department and staff central to assessment

The Extended Assessment Advisory Committee includes all members of the Assessment

Committee and incorporates stakeholders from the community (e.g., PreK-12 professionals, candidates, alumni, faculty, adjuncts, and support staff) The Extended Assessment Advisory

Committee meets at least once each semester to receive input from these stakeholders,

receive feedback advice, and improve processes

THE SETTING

Samford University’s main campus is located in Homewood, Alabama, near Birmingham We also have operations at off-site locations for two programs in the Department of Educational Leadership. Providing instruction at off-site locations minimizes travel for the candidates, with instructors traveling sometimes two nights per week to these locations to conduct classes These off-site programs inhabit buildings owned by other organizations and the locations may change every year, depending upon where faculty and staff focus recruitment We anticipate these regular changes, and we see a trend toward more online instruction to avoid the overhead costs associated with travel and maintenance of off-site facilities

KEY EXTERNAL ORGANIZATIONS

Samford University is accredited by the Commission on Colleges of the Southern Association of Colleges and Schools (SACSCOC; See Appendix B) As a part of that process, each program submits annual program reviews to the University Director of Assessment

Programs leading to licensure/certification require regular review to maintain approval or accreditation The Council for the Accreditation of Educator Preparation (CAEP; See Appendix B)

is a professional accreditation organization requiring rigorous review of education preparation providers Specialized professional associations (SPAs) define and review the content area standards by program Approval by a SPA results in national recognition for a program

Educational preparation programs that do not have SPA approval must have approval by the Alabama Department of Education’s Continuous Improvement in Educator Preparation (CIEP) program CIEP Program Review is defined by the 2015 Educator Preparation Chapter, Alabama Administrative Code (click link to access)

ASSESSMENT PHILOSOPHY AND GUIDING PRINCIPLES

We define assessment as the gathering of evidence from multiple data sources and using that

evidence to make recommendations about the characteristics of programs, departments, or the School

The School embraces an assessment philosophy that posits for each person ownership in

assessment as we strive for a model of excellence, embracing and promoting a culture of

assessment characterized by evidence-based decisions All faculty and staff—and, where

applicable, candidates and partners—participate in this dynamic assessment environment

We adhere to seven principles agreed upon by faculty and staff to guide our assessment system:

✓ We use multiple measures and methods in assessing our outcomes The triangulation of these sources allows us to make better inferences

4

✓ We employ flexible systems that reflect (1) our shared vision, (2) institutional, state and professional standards, and (3) program goals/outcomes The flexible systems include both formal and informal approaches, to allow us to be responsive to the changing needs of candidates we serve, the field/clinical experiences, the organizational structure

of the School, and our professional fields

✓ We commit to continuous assessment beyond accreditation and program reviews As individuals, we each share this commitment to continual program improvement based upon frequent collection and analysis of key data, and evaluation of outcomes

✓ We consistently improve the assessment system through a shared vision among the members of the Assessment Committee, the faculty, adjuncts, and staff

✓ We emphasize assessment where learning is influenced most – at the course level

✓ We ensure clarity in procedures for data collection, disaggregation, analysis,

interpretation, dissemination, and use of assessment data

✓ We maintain scientific and ethical excellence in methods and measures (see below)

TECHNICAL PROPERTIES

When we develop key assessment tools in-house, we examine them systematically for technical adequacy, focusing on validity, reliability, fairness, and freedom from bias (AREA, APA, & NCME,

2014) Test validity is the extent to which an assessment accurately measures what it is

supposed to measure Our studies focus on content validity, the degree to which an assessment

measures knowledge of the intended domain

Reliability is the degree to which an assessment tool produces stable and consistent results Our

studies focus on inter-rater reliability (also called inter-rater agreement, inter-rater

concordance, and inter-observer reliability) It is a score of how much homogeneity, or

consensus, there is in various evaluators’ ratings

Fairness, an essential quality of an assessment, is broadly defined as equitable treatment of all

test-takers during the testing process, absence of measurement bias, equitable access to the constructs being measured, and justifiable validity of test score interpretation for the intended purpose(s)

We focus on freedom from bias Bias is a tendency to lean in a certain direction, often to the

detriment of an open mind

KEY COMPONENTS OF OUR ROBUST ASSESSMENT SYSTEM

Three main parts comprise the structure of our assessment system: (1) standards we use to align the system, (2) a platform to gather, record, and report data, and (3) key assessments to generate the data We describe the structure below, and the processes of our dynamic system

in the following section

We use standards to align our assessments with curriculum, instruction, and accountability

Each specific assessment aligns with national/state/institutional standards and/or well thought- out student learner objective(s) In some cases, these standards come from the specialized professional associations (SPAs; see Appendix B)

This alignment approach allows us to thoughtfully integrate and connect components of schools

as systems Alignment is also supported by research as a best practice (see Appendix C)

An assessment platform aids in the gathering of data and presentation of data tables The

assessment platforms we use are Watermark’s Taskstream and Qualtrics (for course evaluations only)

In addition to course assessments, we have program key assessment, and CAEP key indicators Our key assessments generally include the following:

Alabama State Licensure Exams and/or Content Knowledge Assessment Samford Planning Rubric

Samford Student Learning Rubric Internship Evaluation

Educator Disposition Assessment rubric Employer Survey

Completer Survey Technology Integration Assessment Thus, the Assessment Committee evaluates assessment using the following checklist:

✓ Each assessment aligns to appropriate standards (State, SPA, CAEP/InTASC)

✓ Assessment is routine and timely

✓ Each performance-based task has a standard (outcome)-aligned three-point scoring rubric We consistently use this “1” to “3” scale for all Key Assessment scoring rubrics in our course, dispositional, and clinical experiences to allow for comparisons among programs, as deemed appropriate

✓ We examine technical properties of in-house constructed performance tasks and scoring rubrics for valid alignment with standards and reliable implementation

✓ Candidates enter artifacts tied to Key Assessments into TaskStream/Watermark for grading by instructors and clinical educators

✓ Our assessment system captures data at transition points established by each program

✓ Candidate evidence supports assessment purpose, alignment to appropriate standards, and attainment of learning outcomes

✓ Regular analysis and interpretation of data lead faculty in making data-driven decisions

✓ Annual program review encompasses assessment review and planning for continuous improvement

6

KEY PROCESSES OF OUR ROBUST ASSESSMENT SYSTEM

To enact the assessment system, we also align our assessment plan with the collection and

reporting processes required by regional and national accreditation agencies using state and

national standards Appendix B of this document includes the Assessment Operation Plan table

of standards, times, and responsible parties for conducting each assessment

Internally, the School uses the four-phase continuous improvement model

for assessment: plan, implement, analyze, and report/revise The first

phase includes a clear plan, purpose, and defined outcomes The

implement phase includes a systematic approach to data

collection involving all stakeholders The analysis phase centers

on careful distillation of the data collected The report/revise

phase includes the ongoing use of assessment to improve

programs and services often referred to as “closing the loop.” This

final stage includes evaluation and improvement of the assessment

process itself

Continuous Improvement Cycle

TRANSITION POINTS: MONITORING CANDIDATE PROGRESS

Faculty in each program establishes assessment transition points, representing gateways

through which candidates must pass to proceed to the next level of their program of study Typically, transition points correspond to admission to the program, a mid-point of the

program, and at the end of the program Programs vary based upon external requirements, duration, and level Appendices E-G include transition details for each department

ON-GOING REVIEW OF THE PLAN

The Assessment Committee reviews the assessment system and this handbook annually and conducts a comprehensive review at least once every three years A summary report of the review will contain a report of the findings, a review of accomplishments and opportunities for growth (including needs identified by the Extended Assessment Advisory Committee), each program’s self-assessment of the plan, and results of a focused discussion at the regular meeting

of the School about the status of this assessment system

APPENDIX A:

DEPARTMENTS/PROGRAMS WITH ASSOCIATED DEGREES AND SPECIALIZED ORGANIZATIONS

Type (Initial or Advanced)

Specialized Org**, if any

Elementary

Early Childhood

Early Childhood Special Education

Elementary Collaborative

ACEI NAEYC CEC Elementary with Christian Education and Missions

Concentration (K-6)

*Degree (Bachelor/Master/Specialist); Type (Initial/Advanced/Other);

** Specialized Organizations are listed below by Acronym See Appendix B for detailed description:

NCFR – National Council on Family Relations; ACEI – Association for Childhood Education International; ACTFL – American Council on the

Teaching of a Foreign Language; CEC – Council for Exceptional Children; NCSS – National Council for the Social Studies; ELCC – Educational Leadership Constituent Council; NAEYC – National Association for the Education of Young Children; NAGC – National Association for Gifted Children/Council for Exceptional Children; NASPE – National Association for Sport and Physical Education; NCTE – National Council of Teachers

of English; NCSS – National Council for Social Studies; NSTA – National Science Teachers Association;

8

APPENDIX B: DESCRIPTION OF EXTERNAL ORGANIZATIONS

THE SOUTHERN ASSOCIATION OF COLLEGES AND SCHOOLS (SACS) COMMISSION ON COLLEGES

(SACSCOC)

SACSCOC is “the regional body for the accreditation of degree-granting higher education

institutions in the Southern states It serves as the common denominator of shared values and practices among the diverse institutions in Alabama, Florida, Georgia, Kentucky, Louisiana, Mississippi, North Carolina, South Carolina, Tennessee, Texas, Virginia and Latin America and other international sites approved by the SACSCOC Board of Trustees that award associate, baccalaureate, master’s, or doctoral degrees The Commission also accepts applications from other international institutions of higher education.” (http://www.sacscoc.org/)

COUNCIL FOR THE ACCREDITATION OF EDUCATOR PREPARATION – CAEP (Formerly NATIONAL

COUNCIL FOR ACCREDITATION OF TEACHER EDUCATION – NCATE)

CAEP is a national accrediting body for schools, colleges, and departments of education

authorized by the U.S Department of Education CAEP determines which schools, colleges, and departments of education meet rigorous national standards in preparing teachers and other specialists for schools Accreditation, once granted, is continuous as long as the institution fulfills its responsibilities under the continuing accreditation process Continuing accreditation status is granted after an institution has been accredited Continuing accreditation requires institutions

to file annual reports and host an on-site Board of Examiners team in a defined cycle CAEP encompasses administrator education, school counseling, school psychology, speech, and teacher education in various fields such as elementary, English, mathematics, social studies, world languages, and special education Each of these areas has sponsoring agencies that are noted below Each area has been aligned with this Assessment Plan

ASSOCIATION FOR EARLY CHILDHOOD EDUCATION INTERNATIONAL (ACEI)

The Elementary Teacher Education program is a Specialty Professional Association (SPA) through the Association for Early Childhood Education International (ACEI), which is a constituent

member of CAEP ACEI is responsible for the Program Review process for CAEP ACEI requires that teacher preparation programs must have a broad foundation in the sciences, humanities and social sciences, with advanced study in at least one specialty area to be able to optimally expand children's abilities to grow and develop in all areas There are 11 broad areas of

preparation for elementary teachers Additionally, ACEI looks for evidence of learning in the areas of foundations, child development, and learning and teaching Perhaps most importantly, ACEI notes that teacher preparation programs for pre-service elementary teachers should provide carefully administered, sequenced and supervised clinical/field experiences in all areas

of the elementary curriculum Pre-service teachers should have gradually increased

responsibilities in the classroom They should be provided with opportunities to work with children at various grade levels, with a variety of culturally diverse backgrounds, and with different capabilities, including mainstreamed or included special education children, and in activities that link course content to practice They should be expected to critically select and use appropriate materials, resources and technology, and to have experiences with classroom

management and a variety of evaluation techniques Collaboration with other professionals in the school setting should be encouraged to develop team building skills and utilization of all resources to enhance children's learning The elementary education program at this School strives to offer each candidate experiences consistent or exceeding the ACEI standards

NATIONAL COUNCIL OF TEACHERS OF ENGLISH (NCTE)

The National Council of Teachers of English (NCTE) has articulated the standards for future English language arts teachers and is a SPA through CAEP

The NCTE Program Standards for the preparation of English language arts teachers (Grades 7-12) are comprised of four components: 1 Program Structure: Candidates follow a specific

curriculum and are expected to meet appropriate performance assessments for pre-service English language arts teachers 2 Candidate Attitudes: Through modeling, advisement,

instruction, field experiences, assessment of performance, and involvement in professional organizations, candidates adopt and strengthen professional attitudes needed by English

language arts teachers 3 Candidate Knowledge: Candidates are knowledgeable about language; literature; oral, visual, and written literacy; print and non-print media; technology; and research theory and findings 4 Candidate Pedagogy: Candidates acquire and demonstrate the

dispositions and skills needed to integrate knowledge of English language arts, students, and teaching Six to eight assessments are required as evidence for demonstrating pre-service teacher competence across the NCTE Program Standards

NATIONAL COUNCIL OF TEACHERS OF MATHEMATICS (NCTM)

The National Council of Teachers of Mathematics has supplied the standards for future

Prekindergarten-12 teachers of mathematics since 1982 and is a SPA through CAEP The NCTM identifies 10 standards of mathematic achievement to be carried through four divisions

(Prekindergarten-2, 3-5, 6-8, and 9-12) of the education system The standards are divided into both Content Standards and Process Standards that are accompanied by two to four specified goals and multiple indicators that apply across each grade level

To meet NCTM Program Standards, program reports must be submitted to NCATE and reviewed

by NCTM-trained reviewers The program report must demonstrate that 80% of indicators are addressed with at least one indicator in each of the 10 Standards Each program must also provide evidence of a state-required licensure or certification exam Unless using the Praxis II mathematics content exam, which has already been aligned to the NCTM Standards and

Indicators, programs must show evidence of alignment to the NCTM Standards and Indicators by providing a thorough description of the exam with specific explanations as to how it aligns to the NCTM Standards and Indicators This is in alignment with the School’s Assessment Plan

NATIONAL SCIENCE TEACHERS ASSOCIATION (NSTA)

The National Science Teachers Association (NSTA), a SPA through CAEP, has articulated the standards for future K-12 science teachers even before the publication of the National Science Education Standards in 1996 Because “science teacher” represents a broad array of disciplines,

10

the NSTA Standards for Science Teacher Preparation have relied upon recommendations

supplied by the American Association of Physics Teachers, the American Chemical Society, the National Association of Biology Teachers, and the National Earth Science Teachers Association Consequently, areas under review include Biology, Physics, Chemistry, Earth Science, and General Science—all at the secondary level

The NSTA Standards fall into these categories: content knowledge, instructional preparation, field experiences, as well as content specific considerations (e.g., safety and welfare, conducting scientific research)

The expectation is that teacher candidates will be assessed through a combination of

standardized test performance (Praxis), transcript analysis (for content courses), and field experience evaluations (especially student teaching and internship), along with demonstrations

of an ability to prepare instructional materials (lesson plans and EPP designs) This is in

alignment with the School’s Assessment Plan

NATIONAL COUNCIL FOR THE SOCIAL STUDIES (NCSS)

The National Council for the Social Studies (NCSS) provides the standards for future Social Studies teachers and is a SPA through CAEP The Program Standards apply to all initial programs for the preparation of secondary social studies teachers, Grades 7–12

To meet the NCSS Program Standards, pre-service social studies education students in the Teacher Preparation program at our institution are assessed through a variety of data sources including, but not limited to the Praxis Exam (standardized test performance), field experience evaluations, analysis of transcripts, demonstrations of ability to prepare instructional materials and work with diverse populations of students (lesson plans and EPP designs), and capacity for research and reflective teaching This is in alignment with the School’s Assessment Plan

NATIONAL COUNCIL ON FAMILY RELATIONS (NCFR)

The National Council on Family Relations, founded in 1938, is the oldest nonprofit, nonpartisan, multidisciplinary professional association focused solely on family research, practice, and education… “NCFR established and administers the internationally Certified Family Life Educator (CFLE) credential Approximately 125 college and university Family Science degree programs in the U.S and Canada use NCFR Family Life Education curriculum standards as guidelines for their undergraduate and graduate students.” (https://www.ncfr.org/index.php/about)

INTERNATIONAL LITERACY ASSOCIATION - ILA (Formerly INTERNATIONAL READING ASSOCIATION – IRA)

The International Literacy Association (ILA) has managed the standards for future reading professionals including paraprofessionals, classroom teachers, reading specialists, teacher educators, and administrators A SPA through CAEP, ILA has managed these standards through its partnership with NCATE/CAEP since 1980