4.2 Course Descriptions and Summary of
4.2.2 Assessments and Assignments for Engagement
With any course, assessments are a key component to determining the degree of mastery of topics. In all three courses a variety of low-stake assessments or assignments were used to try to ensure that students were watching the lecture material in a timely manner, because these were automatically graded or checked on Canvas, and to enhance students’ mastery of the fundamental material and concepts presented in the asynchronous lectures.
4.2.2.1 Chem 105/106
Since Chem 105/106 were offered fully online during the 2020–2021 academic year, all lectures were presented asynchronously and the instructors incorpo- rated low-stake assignments, including lecture review questions, and graded online homework; these are adaptations to the online instructional modal- ity that have not been used by Chem 105/106 previously. The implementa- tion of the lecture review questions is one of the highlights that is unique in Chem 105/106. It contributes 6% to students’ overall course grade (30 out of 500 points), and the instructors perceive it as helping the students’ engage- ment with the course content and possibly as leading to positive learning outcomes from students.
Each set of lecture review questions is worth 1 point, with two attempts.
It is composed of five short-answer questions (i.e., multiple choice, multiple answer, true/false, fill in the blanks, etc.). They are directly relevant to the associated lecture video content, and work at the “remember and Under- stand” levels or occasionally at the “apply” level, according to Bloom’s taxon- omy.5 Each question asks about one concept that is introduced in a roughly 10 min segment of the lecture video. For example, Question 1 is for a concept introduced between 0 and 10 min, Question 2 is for another concept intro- duced between 10 and 20 min, etc. This is designed to deter students from playing the video at a fast speed or skipping content, which could lead to missed details. however, these questions are open book/note, meaning stu- dents can complete them while watching the video rather than after watching the video. On average, students should be able to complete these questions in no more than 10 min.
The time window given for completion (more than 48 h) is to encourage students to watch a lecture video before the next one is posted because of the cumulative nature of lecture content. however, this window also maintains a flexible due time for international students in significantly different time
Downloaded from http://books.rsc.org/books/edited-volume/chapter-pdf/1746400/bk9781839165238-00031.pdf by RMIT University user on 06 February 2024
39 Adapting Large Intro-level Chemistry Courses to Fully Remote or Hybrid Instruction zones. In addition, the asynchronous style gives students plenty of time to digest a newly learned concept before putting it into application. Based on our previous in-person synchronous lecture experience, we noticed that not all students are comfortable solving at the “apply” level of questions after just hearing the concept in the lecture. By providing a time window sepa- rated from the lecture, we eliminate the anxiety for this cohort of students, which enhanced the low-stakes nature of this assignment.
Since unexpected interruptions can occur in the midst of the pandemic for students, we also incorporated a practical drop policy regarding lecture review questions. Each semester has three mid-term exams and one final exam. Each mid-term exam includes content from 11 to 13 lectures. We allow students to drop 11 sets of lecture review questions in total: three sets relating to material from each mid-term exam and two sets from post-exam 3 material. This is to prevent students from dropping 11 consecutive sets resulting in them missing some major topics necessary for their learning.
(note: a similar drop policy is also applied to the graded online homework.) Based on the data collected from 2020–2021, among 299 students in Chem 105 Fall 2020 and 259 students in Chem 106 Spring 2021, only one student did not attempt any lecture review questions. Over 88% of students received at least 25 points (out of 30 points) for this assignment (88.3% for Chem 105 and 91.5% for Chem 106). While more studies should be done in this area, it is promising that a positive correlation between the lecture review question score to the total course score was observed for both Chem 105 and 106 (Figure 4.1).
From the end-of-semester course evaluations, which are provided by the university to all students enrolled in courses and then made available to instructors the following semester, students left comments indicating a certain degree of appreciation of the lecture review questions. For both semesters, we asked students to comment on how each of the instructional approaches or strategies was helpful in learning the course content in a vol- untary, anonymous end-of-semester course evaluation questionnaire. In Chem 105, 64 students answered this question. among them, 27 students specifically commented on the usefulness of the lecture review question: 16 of them were positive, 7 were negative, and 4 were neutral. While in Chem 106, 30 students answered this question. among them, 13 students specif- ically commented on the usefulness of the lecture review questions: 9 were positive and 4 were negative. Selected representative quotes from these course evaluation responses are shown below.
● “The lecture review questions every other day were really great to keep me motivated to stay on task during lectures and keep me from falling behind.”
● “Lecture review questions were very beneficial for making sure I under- stood the topics presented in the lectures.”
● “Lecture review questions—incredible! [They] really helped me to suc- ceed in the class! I don’t think I would have kept pace without them.”
● “The lecture review questions allow me to pay more attention to the vid- eos and take more notes.”
Downloaded from http://books.rsc.org/books/edited-volume/chapter-pdf/1746400/bk9781839165238-00031.pdf by RMIT University user on 06 February 2024
Chapter 4 40
● “Lecture review questions were helpful as a quick review on the content we just learned so I didn’t just write it down and forget about it.”
● “Lecture review questions were not helpful. It just moved our grade down if they asked a weirded question.”
● “The lecture review questions were a bit more stressful than helpful in my opinion, because of all the other opportunities we have that don’t have point values assigned to them.”
Figure 4.1 Correlation between the lecture review question score and the total course score (Chem 105 – top; Chem 106 – bottom). Each blue dot rep- resents one student. red lines are the trend lines to show the positive correlation.
Downloaded from http://books.rsc.org/books/edited-volume/chapter-pdf/1746400/bk9781839165238-00031.pdf by RMIT University user on 06 February 2024
41 Adapting Large Intro-level Chemistry Courses to Fully Remote or Hybrid Instruction
● “The lecture review questions were pretty hard, but I wish they had been more for participation than graded so that we wouldn’t be so scared when submitting and could focus on the problem solving.”
● “While the lecture review questions were not super helpful in regard to the material, they kept me accountable for watching lectures on time.”
● “I liked the change from having one chance to two. That really helped me realize what misconceptions I have coming out of the particular lecture.”
In addition, when being asked if there were any features of this remote course they would encourage the instructor to retain when the course returns to an in-person format, five out of 44 responses mentioned lecture review questions and commented on keeping them in Chem 105, while eight out of 22 expressed the same desire in Chem 106. Considering this assignment only contributes six percent to a student’s overall course grade, we believe this assignment may have a positive impact on students’ learning. We acknowl- edge that this evidence is not robust, therefore at this point we are inferring from anecdotal evidence.
4.2.2.2 Chem 111A
as part of watching the asynchronous lectures for this course, students were asked to watch the recorded videos in sequence, each focusing on one sin- gle topic. after the first video presenting one topic of each daily series, sub- sequent videos were only accessible when students would answer pathway questions which were presented in the form of a Canvas quiz. Figure 4.2
Figure 4.2 Sample of how the videos were presented to students on Canvas. Stu- dents had to watch videos in sequence, and answer pathway questions, before obtaining access to the next video. The parts that are inaccessi- ble appear in faint font.
Downloaded from http://books.rsc.org/books/edited-volume/chapter-pdf/1746400/bk9781839165238-00031.pdf by RMIT University user on 06 February 2024
Chapter 4 42
shows a sample of how the videos were presented to students on Canvas.
This technique allowed students to be actively engaged with the material pre- sented in the videos and sought to avoid fatigue, both through the use of shorter videos (10–15 min long), and through opportunities for students to put the new knowledge into practice.
4.2.2.3 Summer Courses
For both the summer general chemistry and organic chemistry series, low- stakes and frequent student quizzes were given to ensure that students were completing lectures at a pace appropriate for an accelerated course. For each course, the quizzes consisted of a combination of auto-graded questions such as multiple choice and fill-in-the-blank questions, and manually graded questions, which required students to upload images of their hand-written work.
The summer general chemistry (Chemistry 105/106) quizzes were open- note and open-book to help develop students’ skills on fundamental chemi- cal concepts and chemical calculations. Students were then expected to apply these skills on closed-note, closed-book exams as a final test of mastery with application to new situations—namely, to examine the transfer of learning.6 This was done to ensure that students were proficient in fundamental chem- ical concepts needed for future chemistry courses.
In organic chemistry (Chem 251/252), quizzes were closed-note and closed- book to make sure that students had a solid understanding of basic reac- tions and mechanisms before being asked to apply and extend these skills on exams. In general, students seemed to appreciate these small assessments along the way, and when asked in course evaluations what they liked about the course one student commented “Frequent quizzes” and another said,
“Frequent quizzes also helped me keep up with the material.”
While quizzes were closed-note and closed-book in organic chemistry (Chem 251/252), the five exams were open-note/open-book. Scientific areas are collaborative in nature, areas where researchers look for what they do not yet know. It is therefore important to document and cite the references that one uses when doing scientific work. To help improve information literacy in chemistry and teach students how to use information ethically and responsi- bly, as part of the course they were taught to evaluate the authority, reliabil- ity, and reputability of different kinds of sources, and also how to use these sources as citations. On open-note/open-book exams, they were required to provide citations for any references or resources they used on their exams.
In general, this improved the academic integrity surrounding course exams since using books was not considered dishonest. anecdotally, there were no discovered postings of exam questions to outside websites from closed-note/
closed-book course exams, whereas some of the closed-note quizzes from the course had been shared to outside sites to obtain answers. as part of the course, students had to determine which references were going to be appro- priate and useful for answering a question. By engaging with references,
Downloaded from http://books.rsc.org/books/edited-volume/chapter-pdf/1746400/bk9781839165238-00031.pdf by RMIT University user on 06 February 2024
43 Adapting Large Intro-level Chemistry Courses to Fully Remote or Hybrid Instruction they often learned more than they would from their notes alone and gained skills with respect to evaluating sources. Since they were asked to cite their sources, the importance of ethical use of information as part of the scientific process may be more emphasized for them.