All Faculty Meeting Presentation Rev

GENERAL EDUCATION ASSESSMENT RESULTS ALL FACULTY MEETING OCTOBER, 24, 2017 Presentation of General Education Assessment Goals, Procedures, Data, and Conclusions – Thad Meeks Chair of Gen

GENERAL EDUCATION ASSESSMENT RESULTS ALL FACULTY MEETING

OCTOBER, 24, 2017

Presentation of General Education

Assessment Goals, Procedures, Data, and

Conclusions – Thad Meeks (Chair of General

Education Assessment Committee)

Objectives for General Education and the Baccalaureate Degree

• Analytic, Problem Solving, and Decision-Making Skills

• Oral and Written Communication

• Foundations in Liberal Arts

• Value of Diversity

• Scientific Literacy

• Ethics

First Cycle of Gen Ed Assessment

First Cycle Analytic

Problem Solving

Communication (Written)

Scientific Literacy

Foundations

in Liberal Arts

Communication (Oral)

Value of Diversity

Ethics

Identify Measurable Outcomes '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 Develop rubrics '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 '14-'15 '14-'15

Map outcomes to the gen ed

curriculum

'13-'14 '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 '13-'14

Collect evidence '13-'14 '13-'14 '14-'15 '15-'16 '15-'16 '15-'16 '15-'16

Score with rubrics (team of

faculty)

'14-'15 '14-'15 '14-'15 '15-'16 '15-'16 '15-'16 '15-'16

Analyze results '14-'15 '14-'15 '14-'15 '15-'16 '15-'16 '16-'17 '16-'17

Act on findings to improve

curricular and cocurricular

'15-'16 '15-'16 '15-'16 '16-'17 '16-'17 '16-'17 '16-'17

Revise assessments '15-'16 '15-'16 '15-'16 '16-'17 '16-'17 '16-'17 '16-'17

Second Cycle of Gen Ed Assessment

Second Cycle Analytic

Problem Solving

Communication (Written)

Scientific Literacy

Foundations

in Liberal Arts

Communication (Oral)

Value of Diversity

Ethics

Collect evidence '17-'18 '17-'18 '17-'18 '18-'19 '18-'19 '19-‘20 '19-‘20

Score with rubrics (team of

faculty)

'17-'18 '17-'18 '17-'18 '18-'19 '18-'19 '19-‘20 '19-’20

Analyze results '18-'19 '18-'19 '18-'19 '19-‘20 '19-‘20 ‘20-’21 ‘20-’21

Act on findings to improve

curricular and cocurricular

'18-'19 '18-'19 '18-'19 '19-‘20 '19-‘20 ‘20-’21 ‘20-’21

Revise assessments '18-'19 '18-'19 '18-'19 '19-‘20 '19-‘20 ‘20-’21 ‘20-’21

Written Communication

• Early attempts were made to collect existing artifacts from

a variety of classes and score them on different written communication and critical thinking criteria

• The mean scores for the upperclassmen were statistically higher than for the lowerclassmen

• The primary limitations were the inconsistencies in the

types of and requirements for the written assignments

• Led the committee to seek a more standardized way to assess written communication (and perhaps other

objectives)

Critical Thinking Assessment Test (CAT)

• 15-item test designed to assess and promote the

improvement of critical thinking and real-world problem solving skills

• Developed by the Center for Assessment and Improvement of Learning at Tennessee Tech University with funding from NSF

• The test is scored by the faculty at the participating institution.

• Assesses four broad skills that align with our goals of Analytic Problem Solving, Written Communication, and Scientific Literacy

CAT Procedure

• Thad Meeks (Chair of General Education Assessment

Committee) and Zenia Agustin (Director of General

Education) trained on the CAT process

• Collected data from freshmen in NFS course (n = 141)

and seniors in IS courses (n = 105) in the spring of 2015

• Gen Ed Assessment Committee scored the freshman data over the course of the spring 2015 semester and the

senior data at the beginning of summer 2015

• At least two raters per item (a third rater settled disagreements)

• Ratings were sent to the Center for Assessment and

Improvement of Learning for scoring

CAT Results; * = Significant Group Difference

*

*

*

*

Conclusions from CAT

• The positives were that overall (and with three subscales), there were statistical differences between freshmen and seniors

• Despite this, the effects were not necessarily large and no effect in Evaluating/Interpreting Information

• The limitations of the process included class selection, the timing of testing, the motivation of the students, and the scoring process

• We are currently collecting and will soon be assessing new CAT data We are attempting to address some of these limitations.

Oral Communication Procedure

• In the spring of 2016, we collected data from recorded

student presentations in ACS 101 (n = 109) and various

other sources for upperclassmen (e.g., IS courses, senior

assignment presentations), (n = 69).

• Students’ oral presentations were scored by the General Education Assessment Committee using a rubric on a 1-4 scale (higher numbers represented better performance)

• Three subscales:

• 1 Organization

• 2 Language

• 3 Delivery

Oral Communication Results; * = Significant Group Difference

Conclusions from Oral Communication

Data

• Overall, and on all three subscales, there were significant differences favoring the upperclassmen

• The primary limitation of this process was that the variety

of presentations that were assessed made scoring

consistency difficult

Diversity and Ethics

• These data were gathered as a part of NSSE Focus

Groups in May of 2016 (60 total students)

• One question related to students’ exposure to different

types of perspectives (i.e., ethnic, political, economic, and social)

• 43 responders claimed a combination of classes as their means of exposure.

• One question related to students’ exposure that helped them develop their own code of values and ethics

• 32 responders claimed classroom experiences (e.g., discussion) as the source of this exposure.

• What can we conclude from these data? How can we

better assess this?

Overall Conclusions

• In a positive sense, we showed statistically significant differences between our lower and upperclassmen, except in one subscale of the CAT

• Challenges

• Better measures

• More student motivation

• More consistent sources of data

• More consistent scoring of data

• But all of this requires more buy-in for assessment

• Need for faculty assistance

• Need to “close the loop”