Employer Survey Summary Report

Consistent with this long history of rigorous assessment focused on pragmatic topics, the following report summarizes the impetus, methodology, procedures, and outcomes associated with t

STATE UNIVERSITY OF NEW YORK COLLEGE AT CORTLAND

TEACHER EDUCATION EMPLOYER SURVEY SUMMARY REPORT

SUNY Cortland has a long history of rigorous assessment aimed at collecting formative and summative information; information that addresses important campus and unit mission, vision, and planning goals and objectives; and information that identifies and accentuates strengths but simultaneously highlights weaknesses that need remediation and/or additional inquiry Consistent with this long history of rigorous assessment focused on pragmatic topics, the following report summarizes the impetus, methodology, procedures, and outcomes associated with the Fall 2003 Employer Satisfaction Survey of our 1999 – 2002 teacher education graduates.

TEACHER EDUCATION EMPLOYER SURVEY IMPETUS

SUNY Cortland's conceptual framework stresses the importance of continuous and comprehensive assessment, assessment that triangulates thirteen learning outcomes (see below) at six separate checkpoints [i.e., at (1) application to the Teacher Education Program; (2) completion of 100 hours of field work; (3) eligibility to student teach; (4) during the student teaching experience; (5) completion of the program; and (6) post-graduation] using a range of measurement tools In general and articulated

in the Mission of the Unit, our goal is to ensure that our teacher education graduates "think critically, utilize technology, communication effectively, understand and value diversity, contribute to their communities, and make a difference in the lives of their students." More specifically, SUNY Cortland candidates need to successfully demonstrate thirteen learning outcomes, outcomes based on Regulations of the Commissioner of Education, the INTASC Principles, the NCATE standards, and the SUNY Chancellor's New Vision Below are the thirteen learning outcomes:

1 Demonstrate a solid foundation in the arts and sciences.

2 Possess in-depth knowledge of the subject area to be taught.

3 Demonstrate good moral character.

4 Understand how students learn and develop.

5 Manage classrooms structured in a variety of ways to promote a safe learning environment.

5 Know and apply various disciplinary models to manage student behavior.

7 Apply a variety of teaching strategies to develop a positive teaching-learning environment where all students are encouraged to achieve their highest potential.

8 Integrate curriculum among disciplines, and balance historical and contemporary research, theory, and practice.

9 Use multiple and authentic forms of assessment to analyze teaching and student learning and to plan curriculum and instruction to meet the needs of individual students.

10 Promote parental involvement and collaborate effectively with other staff, the community, higher education, other agencies, and cultural institutions, as well as parents and other caregivers, for the benefit of students.

11 Demonstrate sufficient technology skills and the ability to integrate technology into classroom teaching/learning.

12 Foster respect for individual’s abilities and disabilities and an understanding and appreciation of variations of ethnicity, culture, language, gender, age, class and sexual orientation

13 Continue to develop professionally as reflective practitioners who are committed to an going scholarly inquiry.

on-METHODOLOGY AND PROCEDURES

The Director of Institutional Research and Assessment (IRA) was requested by the Provost, Deans, Unit, and Teacher Education Council to develop and implement a systematic process for the collection

of (teacher education) employer satisfaction information

Step 1 – Survey Development The IRA Director conducted an extensive exploratory analysis

of extant employer satisfaction surveys/procedures sold by national survey groups and approximately one hundred higher education institution websites The bulk of these instruments were generic in nature and/or tailored to specific graduate outcome criteria specified in institution mission/vision/guiding statements In particular, NCATE accredited/candidate institution's instruments were examined and it was found that the bulk of these instruments asked particular questions in regard

to the institution's learning and knowledge standards Consistent with this approach, the IRA Director developed an instrument that reflected our thirteen learning outcomes and also solicited narrative feedback regarding perceived future needs and/or directions for teacher education programs (see Appendix B); this instrument also was coupled with a definition sheet (Appendix C) and a cover letter (Appendix A) to ensure that respondents would interpret the learning outcomes consistent with the Unit's perspective

Step 2 – Employer Identification The IRA Director aggregated four years (i.e., 1999 – 2002)

of employment information reported by alumni on a series of graduate follow-up surveys The employer demographic information collected revealed information gaps in regard to contact information For example, the follow-up survey respondent may have indicated they work in a large school district but did not specify the specific school As a second example, the respondent may not have provided a supervisor's name These information gaps restricted the number of teacher graduate employers that we could contact (Note In an attempt to resolve this issue, the graduate follow-up survey's employer demographic questions have been refined to collect more detailed and comprehensive information) At this point, the New York State Department of Education's (NYSED) Elementary, Middle, Secondary, and Continuing Education's School Administrator's listing of public

and private contacts (mostly Superintendents or Principals) were manually merged with our incomplete list of employer demographics; whenever possible potential matches were identified; that is, if a follow-up survey respondent indicated that they worked in the particular school district but did not specify a school, they would be linked with the actual district This information was used to generate mailing labels for survey dissemination; there were a total of 815 matches albeit 663 had incomplete information (thus our expectation was that no more than 152 would be returned) In addition, since we were often unaware of an immediate supervisor's contact information, we developed a cover letter that: (a) informed the contact about the survey; and (b) encouraged distribution supervisor knowledgeable about the teacher's performance

Step 3 – Legal Issues The Director of IRA, based on concerns levied at TEC meetings,

contacted the SUNY legal council to explore legal implications associated with providing teacher graduates' identifying information on the survey form It was determined that the school was within its right to request specific information on specific teacher graduates if the teacher education alumni's graduation information was public knowledge; our alumni's graduation information is publicly reported in various reports and thus was deemed acceptable.

Step 4 – Unit Approval The next step in our process actually started at the beginning of the

project Since the IRA Director was requested to develop a teacher education employer survey, the Provost, Deans, Unit, and TEC were consulted for feedback regarding various survey and implementation issues In addition, the TEC advisory council provided input that was integrated The final approved survey documents are found in Appendix A - C.

Step 5 – Survey Distribution While there were 815 overall matches between the follow-up

survey respondent's employer information and the NYSED's list, many of these were mailed to the same employer As such, there were 345 packets of surveys mailed to different employers with many

of these packets containing separate surveys for each individual graduate.

The results of this investigation are strikingly similar across analyses: The majority of our teacher education graduates are viewed by supervisors as either highly acceptable or acceptable on all measures of teacher quality Appendix D shows that over 96% of respondents indicated that the overall quality of Cortland teachers is highly acceptable or acceptable, with almost 60% indicating a highly acceptable rating In fact, across all 13 learning outcomes reported in Appendix E, the highest

unacceptable rating was only 6% Although there were not any significant negative outcomes, the data were still explored controlling for using various demographic variables In Appendix F, Superintendents and Principals rated more teachers as highly acceptable versus acceptable and that all Other respondents rated more teachers as acceptable versus highly acceptable For Appendix G – J, there were not any noteworthy findings; albeit, within a range of items, the proportion of highly acceptable to acceptable and vice versa varied by the controlled demographic variable Appendix K provides a list of 10 categories in which respondents believe that future teacher education programs should focus training

FUTURE DIRECTIONS

There are three future directions that have been proposed or initiated based on a comprehensive analysis of various project processes and outcomes In an attempt to increase the accuracy and completeness of respondent provided employer demographic information (on the Graduate Follow-Up Survey), the OIRA in conjunction with Career Services and Information Technology Staff have developed an online survey that requires alumni to provide detailed employer information, employer information that aligns with the New York State Department of Education Administrators List The detailed employer information will eliminate manual matching of employer – employee information and thus simultaneously ensure more matches A second direction will be to annually administer the survey – this will eliminate sending surveys to schools that no longer employ Cortland graduates The last direction will be to explore a relationship with the New York State Department of Education, a relationship that will: (a) help us identify the employment location of our graduates for future contact; and (b) create a blinded data-sharing effort to provide us feedback on the success of our graduates, feedback based on performance evaluations completed for all New York State teachers

Appendix A Survey Cover Letter

in the school district relevant evaluation information may be available in archived performance records); the names of one or more current or previous beginning teachers in your school or school district are found on the labels in the demographic sections of the scannable survey forms SUNY Cortland requires that beginning teachers demonstrate competency in 13 specific learning areas; as such, we organized our survey around these criteria; in addition, we have included a comprehensive question (14) aimed at overall teacher quality

Attached to this letter is a definition sheet to guide the respondent's interpretation of each learning area If you are going to distribute the survey(s) to a knowledgeable colleague, please make sure that they have a copy of the definition sheet Should you or the respondent have any questions or would like to provide additional information, there is an area at the end of the survey for comments or please contact me at the phone number or email address listed below Again, thank you very much for your time and consideration Please use the pre-addressed return envelope to return the completed survey.

Sincerely,

Shawn Van Etten, Ph.D.

Director of Institutional Research and Assessment

Enclosures

ald

P.O Box 2000 Cortland, New York 13045-0900 Phone: (607) 753-5565 ■ Fax: (607) 753-5590 Email: vanettens@cortland.edu

Appendix BSurvey

Appendix C Survey Definition Sheet SUNY CORTLAND EMPLOYER SURVEY DEFINITIONS

1 FOUNDATION IN THE ARTS AND SCIENCES: The teacher demonstrates a solid knowledge and

understanding in the liberal arts and sciences; a knowledge base that allows him/her to draw from central concepts, tools of inquiry, and structures of the discipline; thereby allowing the creation of meaningful learning experiences for students.

2 KNOWLEDGE OF THE SUBJECT AREA: The teacher has detailed content knowledge in the discipline(s)

he/she teaches

3 MORAL CHARACTER: The teacher promotes honesty; understands and upholds confidentiality; does not distort,

misrepresent, or misuse student or other potentially harmful information; and provides quality education to all students regardless of background.

4 UNDERSTANDS HOW STUDENTS LEARN AND DEVELOP: The teacher understands that students differ in

their (a) developmental levels; (b) approaches to learning; (c) academic motivation; and (d) background

knowledge; as such, he/she creates learning opportunities that accommodate these individual differences and thus promotes the fullest development of all students.

5 MANAGES CLASSROOMS TO PROMOTE A SAFE LEARNING ENVIRONMENT: The teacher uses an

understanding of individual and group motivation, behavior, and cognition to create learning environments that promotes positive social interaction, active learning, and self-motivation.

6 MANAGES STUDENT BEHAVIOR: The teacher is aware of a range of disciplinary models and selects and/or

adapts one or more of these models based on the particular circumstances/needs of the learning environment

7 ENCOURAGES ALL STUDENTS TO ACHIEVE THEIR POTENTIAL: The teacher understands and uses a

variety of instructional strategies to encourage students' development of critical thinking, problem solving, and performance skills.

8 INTEGRATES CURRICULUM & PERSPECTIVES: The teacher understands the importance of making

interconnections across disciplines and comparing historical and contemporary research, theory, and practice perspectives.

9 MULTIDIMENSIONAL ASSESSMENT: The teacher understands and uses multiple and authentic forms of

formal and informal assessment strategies to evaluate teaching and to ensure the continuous development of all students.

10 PROMOTES COLLABORATION: The teacher fosters and maintains relationships with school colleagues,

parents (and other caregivers), agencies, and other key stakeholders to promote student learning and well-being.

11 USES TECHNOLOGY: The teacher incorporates, when appropriate, the use of technology/technological mediums

to enhance dissemination of information and to promote student learning.

12 EMBRACES DIVERSITY: The teacher accepts and values students from diverse backgrounds; treats all students

equitably; creates a climate of mutual respect; and provides a range of activities to promote individual differences (e.g., race, culture, language, gender, age, class, sexual orientation) and experiences.

13 PROFESSIONAL DEVELOPMENT: The teacher is a reflective practitioner who continually evaluates the effects

of his/her choices and actively seeks out opportunities for professional and personal growth.

14 OVERALL QUALITY AS A TEACHER: The overall quality of the teacher as a composite of the above 13

knowledge, skills, and/or dispositions.

Appendix D

Fall 2003 Teacher Education Employer Satisfaction Survey Overall Quality Summary: Question #14

Q14 Overall quality as a teacher?

Q14 Overall quality as a teacher?

Q14 Overall quality as a teacher?

Highly Acceptable Acceptable

Unacceptable Not Applicable

Appendix E

Fall 2003 Teacher Education Employer Satisfaction Survey Summaries: Questions #1 - #13

Q1 Foundations in the arts and sciences?

Q1 Foundations in the arts and sciences?

Q1 Foundations in the arts and sciences?

Highly Acceptable Acceptable

Unacceptable Not Applicable

Q2 Knowledge of the subject area?

Q2 Knowledge of the subject area?

Highly Acceptable Acceptable

Unacceptable Not Applicable

Q3 Moral character?

Q3 Moral character?

Highly Acceptable Acceptable

Q4 Understands how students learn and develop?

Q4 Understands how students learn and develop?

Highly Acceptable Acceptable

Q5 Manages classroom to promote a safe learning environment?

Q5 Manages classroom to promote a safe learning environment?

Highly Acceptable Acceptable

Q6 Manages student behavior?

Q6 Manages student behavior?

Highly Acceptable Acceptable

Q7 Encourages all students to achieve their potential?

Q7 Encourages all students to achieve their potential?

Highly Acceptable Acceptable

Unacceptable Not Applicable

Q8 Integ rates curriculum and perspectives?

Q8 Integrates curriculum and perspectives?

Q8 Integrates curriculum and perspectives?

Highly Acceptable Acceptable

Unacceptable Not Applicable

Q9 Uses multidimensional assessment?

Q9 Uses multidimensional assessment?

Highly Acceptable Acceptable

Unacceptable Not Applicable

Q10 Promotes collaboration?

Q10 Promotes collaboration?

Highly Acceptable Acceptable

Unacceptable Not Applicable

Q11 Uses technology?

Q11 Uses technology?

Highly Acceptable Acceptable

Unacceptable Not Applicable

Q12 Embraces diversity?

Q12 Embraces diversity?

Highly Acceptable Acceptable

Q13 Professional development?

Q13 Professional development?

Highly Acceptable Acceptable

Unacceptable Not Applicable

D2 Supervisor title?

Other

Principal

Superintendent

Q2 Knowledge of the subject area?

Q4 Understands how students learn and develop?

D2 Supervisor title?

Other

Principal

Superintendent

Q5 Manag es classroom to promote a safe learning environment?

D2 Supervisor title?

Other

Principal

Superintendent

Q6 Manag es student behavior?

D2 Supervisor title?

Other

Principal

Superintendent

Q7 Encourag es all students to achieve their potential?