closing-the-homework-feedback-loop-using-dual-submission-with-reflection-homework-methodology

Closing the Homework Feedback Loop using Dual-Submission-with-Reflection Homework Methodology Abstract Homework in engineering courses serves many purposes critical to student learning

Paper ID #30569

Closing the Homework Feedback Loop using Dual-Submission-with-Reflection Homework Methodology

Dr Timothy Aaron Wood, The Citadel

Timothy A Wood is an Assistant Professor of Civil and Environmental Engineering at The Citadel He acquired a Bachelor’s in Engineering Physics Summa Cum Laude with Honors followed by Civil Engi-neering Master’s and Doctoral degrees from Texas Tech University His technical research focuses on the intersection of soil-structure interaction and structural/geotechnical data He encourages students pushing them toward self-directed learning through reading, and inspiring enthusiasm for the fields of structural and geotechnical engineering Dr Wood aims to recover the benefits of classical-model, literature-based learning in civil engineering education.

Dr Dan D Nale PE,

Dan D Nale is Professor of Practice in the Department of Civil and Environmental Engineering at The Citadel Dan received a BS in Civil Engineering from The Citadel and both a MS and PhD in Civil Engineering from The University of South Carolina Dan also earned a MBA from Mercer University Dan worked in the aerospace industry for Grumman on the Space Shuttle before working for Gulfstream Aerospace for 35 years in Savannah, Georgia At Gulfstream, Dan was responsible for Research and Development, Program Management, Engineering, Flight Operations & Flight Test Dan Nale retired from Gulfstream in April of 2019 as the Senior VP of Programs, Engineering & Test Dr Nale has serve

as an FAA Designated Engineer Representative for the FAA, is a professional engineering in the state of Georgia and holds a private pilot’s license.

Dr Ryan Kent Giles P.E., The Citadel

Ryan Kent Giles is an Assistant Professor in the Department of Civil and Environmental Engineering at The Citadel Ryan received both a BS in Civil Engineering and a BA in History from Rice University be-fore earning his MS and PhD in Civil Engineering from the University of Illinois at Urbana-Champaign His technical research interests are in structural health monitoring, design optimization, and historic struc-tures He is also interested in the history of engineering and integrating the liberal arts into engineering education.

c

Closing the Homework Feedback Loop using

Dual-Submission-with-Reflection Homework Methodology Abstract

Homework in engineering courses serves many purposes critical to student learning and success For the students, homework provides an opportunity for concept and procedural practice with feedback and correction, as well as support for the development and refinement of engineering mental models For the instructor, homework supplies insight on student progress through

formative assessment and identification of student challenges A dual-submission-with-reflection homework methodology comprehensively addresses each of these intended purposes In the dual-submission homework process, students submit initial homework solution attempts

followed by a second submission of corrected and assessed work Each submission is

accompanied by a reflection coversheet The instructor then evaluates both submission and the reflection coversheets The dual submission process successfully addresses the need for student practice, feedback, and correction while increasing the teaching efficiency of the instructor The reflection coversheets for each submission require students to consider the completeness of submissions, their knowledge development, areas of greatest mastery and greatest confusion, and communicate their understanding to the instructor A literature review of both homework

strategies and classroom assessment techniques shows the development of the dual-submission-with-reflection homework methodology The instructors administer the methodology through syllabus explanations, coversheet templates, and online learning management systems

Instructors discuss the effectiveness, benefits, and drawbacks of the methodology Results from student surveys illustrate the effectiveness of each component of the dual-submission-with-reflection homework methodology

Keywords

Homework, Instructor Perspectives, Student Perspectives

Introduction

Homework in engineering courses serves many purposes critical to student learning and success Yet student and instructor attitudes and interactions with homework are always changing Early

in engineering education, instructors assigned homework to guide student learning, but

homework was rarely collected or graded due to wide-spread, sink-or-swim instructor attitudes toward student success resulting in high stress examination periods As universities and colleges began to value student retention, instructors shifted to an approach of grading homework to provide students with feedback on their learning While potentially valuable to students, the increase in workload caused many instructors to shift grading effort to graduate teaching

assistants resulting in a breakdown in formative assessment where the instructor could address common problems As time has gone on, this approach has been further undermined by the wide-spread availability of solution manuals and crowd-sourced homework resources on the Internet [1] Engineering instructors have been actively attempting to address these various issues A literature review of ASEE publications will show the development of the authors’

dual-submission-with-reflection homework methodology as an attempt to optimize the effectiveness

of homework The methodology, as detailed in the course syllabus, has resulted in positive feedback from a range of instructors and students in the The Citadel School of Engineering

Homework Challenges

Homework can be a source of frustration and anxiety for both student and instructor The best students will get all they can out of homework regardless of the methodology employed Yet many students struggle to adequately engage with homework due to a lack of understanding of the goals of the course, homework and instructor feedback Some students see homework as an optional way to pad their grade Such students are highly susceptible to temptations to shortcut their learning by copying solutions from fellow students, solution manuals or online sources Failing to understand the benefits of honestly attempting homework, such students are often surprised by poor exam performance Other students see homework as simply a required step toward a certification for an ill-defined future job Treating their educational experience as a

certification experience, they seem to actively attempt to not understand, ignoring the concepts

and trying simply to blindly use equations and engineering models introduced in class Typically, these students will solely evaluate their work on the basis of the resulting grade Any passing grade will do; and if the instructor has provided detailed feedback, at best it is given a cursory glance before the assignment is destroyed or filed Time management is also a serious challenge for many students Engineering homework takes time and focus The crunch of too much work and too little time can create a compounding problem of late homework, missed homework, and skipped practice Students can struggle to engage with the homework due to online solutions, failure to understand underlying concepts, and time management

Instructors experience their own challenges with homework including time management and emotional exhaustion Instructors may assign homework simply as a matter of tradition [2] Others actively desire to assign beneficial homework but assign too much causing students to give up altogether, or too little leaving students without appropriate practice Creating homework assignments is a necessary activity, but often the time required to thoroughly grade homework can be overwhelming Instructors have many demands on their time, and hours a week on

homework grading can leave them overextended and exhausted This is further compounded by student apathy toward feedback so excruciatingly provided by the instructor Many instructors are blessed to have graders and teaching assistants, but this can undermine to ability of the instructor to assess and respond to the nuance of developing student knowledge Other

instructors simply fall back to a completion grade and merely hope that the students got

something out of the homework assignment Instructors are helped by a clear articulation of the goals of assigning homework

Homework Goals

For the students, homework provides an opportunity for concept and procedural practice with feedback and correction, as well as support for the development and refinement of engineering mental models The most common goal for beneficial homework is the opportunity to practice [2] ASCE ExCEEd Model Instructional Strategy refers to the opportunity to apply new

knowledge in an “unfamiliar” context [3] All learning requires repetition for proper

internalization Homework should provide an opportunity to explore concepts and practice procedures for solving problems This practice could take the form of simple conceptual

questions Alternatively, the homework can provide an opportunity to explore and repeat an established problem-solving procedure covered in class Homework can also push students to extend concepts, synthesizing multiple topics to solve particular problems A constructivist theory of education would argue that beneficial homework provides an opportunity to practice thinking like an engineer In a more traditionalist description of the same process, homework provides students with the opportunity to refine their mental models of how reality works and how the engineering models discussed in class connect to that objective reality Homework provides a clear opportunity for students to engage with the material leading to successful

learning through repeated exposure

Beneficial homework also provides the opportunity for observation of student knowledge by the instructor By reviewing student work, the instructor can develop responses to areas of

misunderstanding or weakness among the students [2] Sometimes this feedback is provided through the detailed feedback on homework assignments Alternatively, if the instructor observes

a pattern in student work, the conceptual area can be corrected, strengthened, or reinforced during class An effective instructor will use homework as an opportunity to understand student strengths and weaknesses to improve teaching effectiveness

Finally, beneficial homework also provides an opportunity for self-assessment of student

knowledge Fundamentally, students must assess their own work and consider for themselves whether they have mastered the content This self-assessment is a critical, but often neglected, part of developing the skills of a life-long learner [3] Too many students consider their

homework grade to be the primary measure of mastery Before the widespread availability of solutions online, homework with an assigned grade based on mastery might encourage students

to figure out what and why they made errors on their own More modern approaches seek to couple homework with formative assessment to guide the learning experience Traditionally, instructors provide detailed feedback in the grading of the homework with the hope that students will take the time to review the comments and correct their thinking

Literature Review

Considering the three-fold goals of beneficial homework: practice, instructor observation, and self-assessment, and the challenges of time management, solution availability, shallow learning and instructor fatigue, various innovative homework methodologies have been developed Many instructors have noted the decreasing effectiveness of homework assignments and the challenges posed to both student and teacher ASEE has provided an excellent environment for developing alternative homework methodologies to address these challenges and goals

Many have endeavored to use online computing power and have developed various online

homework platforms This technology addresses some of the challenges and goals Online

databases provide a host of problems, and the ability to change numbers automatically decreases the ability of students to copy work The use of endless or adaptive homework assignments increases student opportunities for practice and immediate feedback Further, the grading load is handled by the computer system, not the instructor, reducing instructor fatigue and protecting the instructor’s time Yet, these tools typically do not conscientiously engage self-assessment or

instructor observation and may fail to engage deeper conceptual learning [4]–[7]

Another approach is to use in-class quizzes or recitations to evaluate the homework learning experience This successfully cuts off student access to solutions, rewards adequate practice before the quiz, and typically decreases the instructor grading load while still providing an opportunity for instructor observation of student performance Yet, there are drawbacks to this approach Lost class time makes implementation costly in the minds of many instructors The limited time associate with an in-class quiz can also reduce the depth of learning that can be assessed Depending on the nature of the quizzes, the method may fail to enforce student self-assessment The use of Classroom Assessment Techniques along with problem-based quizzes can encourage self-assessment and instructor observation This method shows great promise particularly for flipped classrooms and other instructional models that give adequate time for homework-based quizzes [8]–[11]

Another innovative approach engages primarily at the level of metacognition The administrative demands of these models vary widely One model requires a reflection worksheet with each homework set [12] Adequate practice is coupled with a mandatory self-assessment providing the instructor with synthesized observation of student struggles The instructor engages primarily with the reflection worksheet decreasing the instructor’s grading time Another approach is simply to require students to do the things that good students do: taking notes, synthesizing reading, practicing problems, discussing challenges, etc [13] Completion grades are assigned for each task This approach can actively engage practice, instructor observation, and

self-assessment, but might not enforce deep learning or check that learning has truly taken place Alternatively, another method forces mastery by providing an all or nothing grade, instructor feedback, and unlimited resubmission until the student masters the content [14] This method does an excellent job of engaging practice, self-assessment, and instructor observation and can generate deep learning, but may not prevent the copying of solutions Furthermore, the method requires more instructor grading time than the traditional methods

The last category of innovation requires self-assessment of student work Most of these methods involve the assignment of traditional homework problems Students submit their initial attempt and then the instructor provides solutions that the students use to assess their work Grades are often applied based on the quality of the initial attempt and the timeliness of each submission substantially reducing student anxiety about homework and the temptations to rely on third-party solutions Some approaches ask the student to provide a grade on their original work [15], [16] Others simply ask the student to correct the work [17], [18] One study asks the students to perform an autopsy of their work [19] These approaches each provide practice opportunities and self-assessment Though students may still use solutions, the temptation to do so is decreased The instructor may choose to spend similar time evaluating and observing student performance,

or can grade for completion, trusting the self-assessment process, though at the cost of instructor observation This said, the authors have observed that when this approach is used, some students are tempted to barely attempt the problem, knowing they can work to a right answer with the instructor solution and get the grade they want For these students, the effectiveness of self-assessment is likely also compromised

While online homework and in-class quizzes have significant strengths, the dual-submission-with-reflection approach attempts to optimize the strengths of the metacognitive and

self-assessment approaches

Dual-Submission-with-Reflection Homework Methodology

The following syllabus excerpt outlines the dual-submission-with-reflection homework

methodology The appendices noted in this section are provide in the appendices Typically, homework represents 15%-20% of the overall grade in classes using this methodology and frequent exams, firmly graded, make up the remaining grade The syllabus excerpt is provided to illustrate the context established by the dual-submission-with-reflection methodology

Individual Homework

Homework is for the student, for his or her learning, practice and assessment Many of the

homework problems represent intentionally challenging, real-world problems Working

engineers and engineering students must practice problem formulation, problem solving, and solution documentation Therefore, a proper solution format is required (see Appendix A)

Students may work together on homework assignments to gain additional understanding More

than any other academic activity, continuous practice of concepts establishes long-term mastery

The assigned homework is the minimum required practice

Please consult the following book on problem formulation, solving, and documentation:

Polya, G., and Conway, J H (1945) How to Solve It: A New Aspect of Mathematical Method Princeton

University Press, Princeton, NJ [20]

Getting the most from the homework requires at least four separate events First, students should strive to use the mental and mathematical models discussed in class to solve the problem

Second, while attempting the problem, students should consider why they are working the problem: What principle does the instructor intend them to practice or explore? Why does the homework seem easy or hard? What questions remain after attempting the problem? Is the homework solution complete? The initial attempt coversheet (Appendix B) explores and

documents the answers to these questions

Third, students must check their work against the solution The solution should help answer remaining questions about the principles and processes explored in the homework The student must make the correct processes and techniques his or her own so they can tackle similar

problems on later homework and exams

Finally, the student must consider how to align future homework attempts with the expectations

of the instructor, and whether the questions about the concepts remain The self-assessment coversheet (Appendix C) provides this opportunity for reflection

Documentation

Students must document any help received from supplemental instruction, classmates, reference books, or the internet Information from the course textbook (equations and outlines of

procedures), class notes, or the instructor is immediately available to all students and requires no documentation For written homework, state who and/or what helped immediately after the provided content

Solutions

The use of solutions during homework attempts is strongly discouraged Relying on solutions

from previous classes, the textbook, or the internet will result in poor performance during the exams Nevertheless, if published solutions reveal errors, subsequent corrections require proper documentation

Grading

Homework grading by problem emphasizes effort, completeness, timeliness and accuracy Each homework problem can earn up to 10 points The grade is composed of 6 points for a complete, on-time homework attempt, 1 point if the attempt is correct, and 3 points for self-assessment of the homework attempt Students will submit each problem twice:

1 Initial Attempt:

The student will post their initial attempt on the LMS as a single PDF file

Each problem will be evaluated based on timeliness, effort and completeness for up to 60% credit An additional 10% will be earned for accurate initial attempts A problem

missing any sections (see Appendix A), appropriate diagrams, or a good faith effort at the solution in the required homework format, even with a correct answer, may receive no credit

• Each solution attempt must follow the format, including a figure and an answer

• All problems in a homework set must be uploaded in a single PDF document PDFs may

be created using personal scanners, smartphone scanning apps or the document scanners

at the library

• The first page must be a complete initial attempt coversheet (Appendix B)

2 Self-Assessment:

The student will self-assess their homework and submit a hardcopy in class

The remaining 30% credit will be earned by submitting a hardcopy of each complete and

self-assessed homework problem The instructor will provide a minimum of 36 hours access

to the solutions before requiring homework assessment in class An incomplete hardcopy problem will receive no additional credit

• Assessment should provide clear documentation of corrections made in a different color

• Assessment should identify errors and comprehensively lead to the final correct answers

• A minimum of a check mark in a different color next to the correct answer is required

• The first page must be a complete self-assessment coversheet (Appendix C)

• The second page must be the complete initial attempt coversheet from the initial attempt Typically, six potential grades can be earned per problem, as seen in Table 1

Table 1 Potential grades per problem

Initial Attempt (LMS) Self-Assessment (hardcopy) Maximum Grade

Instructor Perspective

The instructors have enjoyed the dual-submission-with-reflection homework methodology The methodology provides the opportunity to explicitly discuss the purpose of homework, good learning practices, and the importance of concept mastery The methodology also reduces the grading load while increasing cognitive interaction with students Instructors only evaluate the homework after both submissions, quickly recording the number of problems attempted,

corrected and/or correctly answered for the gradebook and interacting with the student responses

to reflection questions This closes the feedback loop allowing instructors to see where the students are understanding and struggling without the painful process of interpreting poorly completed homework Instructors have also found the methodology gives them more time and emotional energy to invest in the students who want in-depth feedback during office hours with less time wasted on students who appear not to want to master the content in the class This benefit has led to increased rapport and positive interaction between students and instructor while removing the often-adversarial tone taken by students attempting to understand how homework credit has been assigned More instructors at The Citadel see the value of the dual-submission process and are incorporating it in their classes

Student Perspective

A total of 560 engineering students in classes using a dual-submission homework methodology responded to survey questions These students were in a range of engineering mechanics classes, class years, instructors, and engineering disciplines though the majority of students were civil and mechanical engineering students in sophomore level engineering mechanics courses

The first set of questions asked about the student’s self-perceived engagement in learning during the initial attempt, the self-assessment and as compared to previous experiences with

instructor-graded learning Figure 1 shows that 90% of the students felt engaged in their learning during

each phase of the homework process Most interestingly, they felt most strongly that their

engagement was better using the dual-submission process compared to traditional instructor

graded homework (Figure 1(c)) This data supports the idea that the methodology provided

meaningful practice and meaningful self-assessment

Figure 1 Student survey responses to (a) “During the dual-submission homework process, I feel

engaged in my learning while attempting the homework the first time” (n=560) (b) “During the

dual-submission homework process, I feel engaged in my learning while assessing my

homework against the instructor's solution” (n=560), and (c) “During the dual-submission

homework process, I feel more engaged in learning compared to previous experiences with

instructor-graded homework.” (n=521)

277

299

301

228

205

165

45

43

40

10

13

15

(a)

(b)

(c)

% of surveyed students

Strongly Agree Somewhat Agree Somewhat Disagree Strongly Disagree

Students also responded to questions related to the effectiveness of the initial attempt and self-assessment coversheets These coversheets represent a later improvement to the methodology, so the sample size of students is smaller Figure 2 shows that around 60% of the students found the opportunity to reflect on what they learned and how to improve helpful Conscientious self-reflection and metacognition are developed skills that take time to appreciate The remaining students tended to think of the coversheets as a nuisance, something to be done at the last minute Instructors found the coversheets to expedite grading and provide the necessary feedback

required to understand student performance

Figure 2 Student survey responses to (a) “The Initial Attempt Coversheet helped me consider

what I learned and how to improve” (n=124) and (b) “The Self-Assessment Coversheet helped

me consider what I learned and how to improve” (n=137)

37

53

72

83

54

47

33

26

(a)

(b)

% of surveyed students

Strongly Agree Somewhat Agree Somewhat Disagree Strongly Disagree