APAC Guidelines Revised Fall 2015 FINAL WITH APPENDICES

SUNY Oneonta Academic Program Assessment Committee Guidelines for Academic Program Assessment at SUNY Oneonta: Developing Meaningful and Efficient Assessment of Student Learning... 5 De

SUNY Oneonta Academic Program Assessment Committee

Guidelines for Academic Program Assessment at SUNY Oneonta: Developing Meaningful and Efficient Assessment of Student Learning

Table of Contents

Introduction ……… 3

Alignment Within Institution ……… 4

Assessment Cycle Timelines and Reporting Deadlines ……… 5

Developing an Assessment Plan in Four Steps (for undergraduate and graduate programs) …… 6

Step 1: Developing Student Learning Outcomes Step 2: Curriculum Mapping Step 3: Finding Measures Step 4: Closing the Loop Guidelines for Writing Annual APAC Reports ……… … 11

Programmatic Use of Assessment at SUNY Oneonta ……… ……… 12

Institutional Use of Assessment at SUNY Oneonta ……… ……… 13

Appendix A: APAC Report Checklist ……… ……… 14

Appendix B: APAC Plan Checklist, Step 1 ……… ……… 15

Appendix C: APAC Plan Checklist, Step 2 ……… ……… 16

Appendix D: APAC Plan Checklist, Full Plan ……… ……… 17

Appendix E: Sample Curriculum Maps ……… ……… 18

Appendix F: Sample Aggregated Data ……… ……… 19

Introduction

The Academic Program Assessment Committee (APAC) was established by the Provost and College Senate to facilitate academic program assessment The committee consists of nine faculty members: the College Senate elects five as representatives from each school; the Provost appoints the remaining four members APAC assists faculty and academic departments in applying best practice principles to procure meaningful assessment data in the most efficient manner APAC regards faculty in departments and programs as experts in their fields who are best able to determine meaningful educational experiences for students and are in the best position to assess the impacts of those experiences The guidance from APAC is designed to assist the institution in meeting the Middle States Commission on Higher Education Standards for Accreditation and Requirements for Affiliation.1 The SUNY Board of Trustees resolved in March 2010 that all campuses must have in place assessment plans that meet or exceed Middle States standards or those of specialized accreditors All learning experiences, regardless of modality (such as distance education), program pace/schedule, level and setting are to be

consistent with higher education expectations.2

This document guides members of all academic programs to plan and assess in a collaborative, inclusive, and participatory process It encourages alignment with SUNY’s Master Plan (a document revisited every four years as required by NYS Education Law section 354) as well as local college plans College leaders (e.g., vice presidents, deans, etc.) should communicate these comprehensive expectations to academic programs to build and sustain understanding as well as advance interactions and cooperative efforts among divisions As conditions change, these guidelines and periodic peer reviews are intended to advance the “consideration and use of assessment results for improvement of educational effectiveness.”3 Advice and assistance is

available upon request from the Academic Program Assessment Committee (APAC) via its

representatives from each academic division

1

Alignment

Assessment plans and processes are related to the annual reports required of each academic unit

at the end of the year and programmatic reviews that must be conducted at least every seven years They provide content for the annual report assessment plans to help academic programs describe goals and objectives for the year as they relate to overall campus direction and

summarize major accomplishments as well as challenges These complementary documents help academic programs plan ahead and enable them to use feedback for justifying adaptations and change Assessment also guides strategic planning, resource planning, and sustained

improvement

Assessment Cycle Timelines and Reporting Deadlines

Long-term Academic Program Assessment Timeline

Fall 2009-Spring 2011 First APAC plans created

Fall 2011-Spring 2014 – Implemented the first 3-year cycle

Fall 2014 – Created new 3-year assessment plans/timelines, with revisions if necessary Fall 2014-Spring 2017 – Implementation of 2nd 3-year cycle

Fall 2017 – Create new 3-year assessment plans/timelines, with revisions if necessary Fall 2017-Spring 2020 – Implementation of 3rd 3-year cycle

Fall 2020 – Create new 3-year assessment plans/timelines, with revisions if necessary

Annual Reporting Timeline

March 1 – Annual APAC Reports are due to APAC via Office of Institutional

Assessment and Effectiveness

April 1 – APAC members submit feedback to Deans and Department Chairs (APAC

members will schedule face-to-face meetings with departments that receive low rankings before forwarding their recommendation to the Dean)

May 1 – Final approval and feedback from Deans to Department Chairs and APAC via

Office of Institutional Assessment and Effectiveness

New 3-year Assessment Deadlines (2014, 2017, 2020, 2023, 2026, etc.)

October 15 – New or updated 3-year assessment plans/timelines are due to APAC via

Office of Institutional Assessment and Effectiveness

November 15 – APAC members submit feedback to Deans and Department Chairs

(APAC members will schedule face-to-face meetings with departments that receive low rankings before forwarding their recommendation to the Dean)

December 15 – Final approval and feedback from Deans to Department Chairs and

APAC via Office of Institutional Assessment and Effectiveness

Developing an Assessment Plan in Four Steps

(For undergraduate and graduate programs)

The Academic Program Assessment Plan establishes the knowledge base, skills, behaviors, and perhaps even attitudes that students of an academic program can be expected to

exhibit/hold/master/demonstrate upon graduation Each plan should also address how the content and design of the program’s curriculum lead to students’ achievement of program expectations, how the program has assessed the effectiveness of its curriculum, and how it has used assessment information to improve the academic program

Mission, Goals, and Objectives

Each academic department should have a clear mission statement that is publicly disseminated and aligned with the College mission Departments should have program goals that provide a focus for faculty, administrators and other constituencies on intentions, purposes, and delivery

Distinguishing Among Goals , Objectives, and Outcomes 4

Goals:

• General intentions/purposes that are broad and more long-range in scope and not

changing over the planning horizon

• May use words or phrases directly out of unit mission statement

• Not directly measurable

• Often a “process” statement (i.e., begin with verbs such as establish, provide, enhance) Objectives:

• Specific and measureable based on measures of expected outcomes

• Typically there are multiple objectives for each overall goal

• Often a change-oriented statement that shows directionality compared to moving

up/down, or maintaining high/low levels when a ceiling/floor exists (i.e., include words such as increase, decrease, improve, maintain)

Outcomes:

• Very specific statements translate into assessable measures

• Expected outcomes refer to anticipated results and include criteria for determining

success

• Actual outcomes refer to the actual results of the assessment

Step I Establishing Objectives: “What knowledge and competencies do we expect students to gain from our program?” (See Appendix B: APAC Plan Checklist, Step 1)

Faculty members should arrive at a consensus around the desired student learning outcomes associated with the programmatic objectives of their discipline as well as what it means to be in synchronicity with institutional expectations regarding students’ intellectual growth The

following question assists in developing the consensus: “What difference do we intend to make

in our students as a result of their experiences with us and our curriculum with respect to

knowledge, behaviors, skills, and attitudes?”

Faculty should:

• Examine and review existing program objectives

• Elicit and discuss faculty members’ perceptions of program objectives (both actual and aspirational)

• Analyze and compare program objectives with stated institutional expectations regarding students’ intellectual growth; the College's mission and strategic plan; the Academic Master Plan; programmatic objectives at comparable peers or aspirant institutions; expectations expressed by the field at large (e.g., as determined by examination of current textbooks, communication with national organizations in the discipline); criteria and standards of certification and accreditation agencies and/or national associations in the discipline if applicable; and results from the most recent program review

• Make the objectives understandable to students

The assessment plan should include approximately 4-8 Student Learning Outcomes (SLOs)

Note that creating too many learning objectives will make assessment formidable and threatens its success

Undergraduate programs may focus on objectives that cover:

• Demonstration of knowledge from different areas of subject matter

• Demonstration of writing and presentation skills

• Demonstration of synthesis of various theories

• Demonstration of analytical/critical thinking skills

• Demonstration of research skills and/or original thought

Graduate programs may find it useful to focus on broader objectives such as:5

Step II Activities and Strategies: “How do courses and other experiences built into the

curriculum relate to each other and contribute to programmatic goals?” (See Appendix C:

APAC Plan Checklist, Step 2)

Faculty should review all activities that are aimed at accomplishing programmatic objectives First and foremost, this step requires a focus on a different question: “Do we offer activities and experiences in our curriculum that make it possible for students to achieve programmatic

objectives?” In addition, it is important that faculty members reach consensus on the rationale for individual courses, program requirements, and program structure when undertaking this step (See Appendix E: Sample Curriculum Maps)

In attempting to accomplish this step, faculty should consider the following actions:

• Determine the extent to which program objectives are embedded in specific courses and make adjustments as appropriate (e.g., strengthening the coverage of objectives that are not sufficiently addressed, de-emphasizing objectives that are covered excessively)

• Review and analyze curricular coherence, focusing on the role individual courses are intended to serve, the rationale for all program requirements (including distribution requirements in the major and cognates), and rationales for pre-requisites

• As appropriate, review program components that serve different purposes in the

curriculum (i.e., major, minor, concentration, service courses)

• Determine strategies for assuring comparability of multiple sections of the same course with respect to programmatic objectives

• Examine the relationship of the program to other College requirements (e.g., General Education)

• Determine that curricula delivered by distance education are, “coherent, cohesive, and comparable in academic rigor to programs offered in traditional instructional formats.”6

Step III Assessment: “How do we know students are achieving programmatic goals?” (See

Appendix D: APAC Plan Checklist, Full Plan)

Collect information that will provide direct feedback regarding the effectiveness of a program in terms of its stated learning objectives Implementation involves first asking the question: “What evidence do we have to demonstrate whether students are meeting our expectations for their learning?” Each department should have clear expectations about what constitutes good

assessment practice and have strategies in place to help faculty develop or acquire effective tools for assessing learning outcomes Faculty members – especially those teaching different sections

of the same course – should be encouraged to use comparable methods for assessing student learning outcomes Relying primarily on course-embedded assessment can be the least time-and labor-intensive, is sometimes most economical, and assures student motivation to do well

All departments need to collect and compile student data that are relevant to each programmatic objective These tasks could be assigned to either an individual or a group (e.g., a departmental assessment committee) (See Appendix F: Sample Aggregated Data)

It is important to note that there are differences between undergraduate and graduate education, and in terms of assessment those differences are most likely reflected within the assessment tools In research-based graduate programs, a larger portion of student learning takes place outside of the classroom than in undergraduate programs Therefore, graduate program

assessment is seldom as course-based as undergraduate assessment may be. 7

Graduate programs may determine that there are many acceptable tools for measuring outcomes out of the classroom Some such tools are:8

• Graduate placement information

• Evaluation rubrics from preliminary exams and final defenses

• Number of student publications

• Results of certain exit interview questions

• Surveys of recent graduates

• Updated student CVs

Departments and academic programs should:

• Establish expectations for measures being used to assess student performance, relying on existing literature on good assessment practices to assure valid, reliable, and

representative data

• At most, focus on 3-4 student-learning outcomes each year

• Encourage faculty to use a wide variety of quantitative and qualitative measures to assess student performance, including senior thesis/research projects, student portfolios, pre- and post-assessments within courses, departmentally generated exams, standardized tests, oral proficiency exams, and student teaching or internship evaluations

• Leave the final selection of measures to be administered in a course-embedded fashion up

to the faculty members teaching the course

• Examine program effectiveness through comparisons with information provided by other programs or other groups of interest (e.g., certification agencies, national organizations in the discipline)

• Consider capstone courses as a good place to collect outcomes assessment data

• Evaluate student perceptions of the program through strategies such as senior exit

interviews and alumni surveys

• Ensure that assessment of student learning in distance education courses and programs follow processes used in onsite courses or programs, reflect good practice in assessment methods, and are amply supported by analysis and evidence.9

7

Step IV Closing the Loop: “How can we use assessment of student learning to improve our program?” (See Appendix D: APAC Plan Checklist, Full Plan)

The assessment process now provides the opportunity to compare expected outcomes with actual outcomes relative to objectives and activities This final step asks: “What are we doing well, what could we do better, and how can we improve?” Faculty in the program must review

assessment data and discuss findings with each other and perhaps other stakeholders Decisions should then be made on the continuation of activities that lead to the realization of program objectives and the discontinuation or revision of activities that are not It is also possible that the assessment process may lead to the revision or elimination of old objectives and/or the

development of new ones

Faculty should consider the following actions:

• Provide aggregate data to faculty for review and discussion (individual faculty data should never be shared with other faculty members)

• Reach conclusions regarding program effectiveness as revealed for each learning

objective, identifying both strengths and weaknesses revealed through the assessment

• Offer recommendations for changes in curriculum and teaching as appropriate

• The development of a new statement of departmental objectives for next assessment round as appropriate